IBM Support

How to export more than 10,000 lines of analytics data?

Troubleshooting


Problem

Users are not able to export all our API Analytics and are limited to only 10,000 rows of data that can be exported in 200s.

Resolving The Problem

Pre 10.0.5.3: your option is to upgrade to 10.0.5.3 post, otherwise your export is limited to 10,000.
Post 10.0.5.3: Bulk export of Analytics data that uses the Scroll operation:
What is Scroll operation:

The Scroll operation is designed to retrieve many results from Analytics. This number can get large, easily reaching millions of events and GBs of data, which is why the operation is designed to provide batches of results.

This is achieved by returning a pointer, called scroll ID, which needs to be included in the follow-up request to get the next batch of results. This process can be repeated any number of times until the desired for timeframe have been retrieved from Analytics. It is important to note that a time snapshot is created internally at the time of the first scroll search and any changes to the documents will only be reflected that uses a new scroll context.

The Scroll operation also supports query parameters the same way the paginated API does, except for  timeframe parameters, start, and end.

Please review the blog link above for steps on how to export the analytics data.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSMNED","label":"IBM API Connect"},"ARM Category":[{"code":"a8m50000000CeBlAAK","label":"API Connect-\u003EManagement and Monitoring (MM)-\u003EAnalytics"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"10.0.5"}]

Document Information

Modified date:
06 September 2023

UID

ibm17030791