The hourly archiving service is ready to proceed for all IBM Log Analysis with LogDNA and IBM Cloud Activity Tracker with LogDNA accounts.
Earlier in 2020, we shared a preview of a new feature for our LogDNA based offerings — a switch from daily to hourly archiving. Since then, we have worked closely with several clients to improve the capability. The rollout to hourly archive has been long-awaited and the service is ready to proceed for all IBM Log Analysis with LogDNA and IBM Cloud Activity Tracker with LogDNA accounts. The transition is starting now and is expected to complete over the remaining week.
What is changing?
This feature will allow you to receive archived logs faster, and you’ll be able to unzip just a portion of your logs instead of an entire day’s worth. Additionally, the new folder format better supports Hive partitions.
As we transition you to the new format, you might observe a small overlap between the older daily format archives and the new archives. Expect to see hourly archives in your provider within 24 hours of the logs being received by us. These details can also be found in our documentation for IBM Log Analysis with LogDNA and IBM Cloud Activity Tracker with LogDNA.
The new archive directory format for all providers will look like this:
year=YYYY/month=MM/day=DD/<accountID>.<YYYY>-<MM>-<DD>.<HH>00.json.gz (where HH is hours in 24 format)
The current archive directory format looks like this:
/<accountID>.<YYYY>-<MM>-<DD>.<clusterId>.json.gz
The current archive directory format for S3 looks like this:
YYYY/MM/<accountID>.<YYYY>-<MM>-<DD>.<clusterId>.json.gz
The content of the archive files will remain the same as before.
What you need to do
For most customers, no reconfiguration is needed. We will be rolling out these changes to accounts with archiving enabled.
If you have systems that consume your archived files (like a data warehouse), you’ll have to update those systems to expect the new archiving directory structure. Nothing in the Export API or other LogDNA app experiences will change. You can learn more about archiving in our archiving guide.
Learn more
IBM Log Analysis with LogDNA and IBM Cloud Activity Tracker with LogDNA are two IBM Cloud services delivered in partnership with LogDNA to deliver full Log Management Service capabilities. Both services are hosted on IBM Cloud in many worldwide regions, allowing you to co-locate your logging and cloud activity events.
Setup is simple and allows you to quickly gain insights into your workloads. Click on the Observability link to gain visibility into the performance and health of your workloads today.
Realize additional value from your data
Archiving your data to IBM Cloud Object Storage opens your data in new exciting ways to realize additional value from your operational logs. Learn more about three of them here: