IBM Support

QRadar: How to export Ariel saved searches from the API from the command line (curl)

How To


Summary

This article provides a step-by-step guide on how to export Ariel saved searches from the API from the command line (curl).

Steps

Before you begin
The command cURL is used to interact with the QRadar API from the command line. To run this command, the administrator needs to generate an access token first, or use an existing one with admin or Saasadmin rights.
Steps
  1. Use the following command to search for the events you are looking for and take note of the search_id value. You must replace the $SEC_TOKEN, $AQL_QUERY, and $CONSOLE_IP with your information. 
    curl -s -X POST -H "SEC: $SEC_TOKEN" -k -d "query_expression=$AQL_QUERY" https://$CONSOLE_IP/restapi/api/ariel/searches
    • $SEC_TOKEN: Is an authentication token created in the authenticated services window of the admin tab.
    • $AQL_QUERY: The AQL query needed to find the data. For more information about AQL queries, see our AQL guide.
    • $CONSOLE_IP: The IP of the console you want to post the search to.

    The search_id n this example output is "a95c100d-1a23-4c57-b545-04b47aaee4d7": 
    "cursor_id": "a95c100d-1a23-4c57-b545-04b47aaee4d7",
"status": "WAIT",
"compressed_data_file_count": 0,
"compressed_data_total_size": 0,
"data_file_count": 0,
"data_total_size": 0,
"index_file_count": 0,
"index_total_size": 0,
"processed_record_count": 0,
"desired_retention_time_msec": 86400000,
"progress": 0,
"progress_details": [],
"query_execution_time": 0,
"query_string": "select * from events last 5 minutes",
"record_count": 0,
"size_on_disk": 0,
"save_results": false,
"completed": false,
"subsearch_ids": [],
"snapshot": null,
"search_id": "a95c100d-1a23-4c57-b545-04b47aaee4d7"}
    Note: For this example, the AQL query is returning all the fields from the events table that were sent in the last 5 minutes.
  2. Confirm the search status is completed. It might take a long time to complete, so rerun the following command to keep checking the search status, and when the status is ("status": "COMPLETED"), take note of the record_count that is returned. It is needed to page through the data in the next step. 
    curl -s -X GET -H "SEC: $SEC_TOKEN" -k https://$CONSOLE_IP/restapi/api/ariel/searches/$SEARCH_ID
    • $SEC_TOKEN: Is an authentication token created in the authenticated services window of the admin tab.
    • $CONSOLE_IP: The IP of the console you want to post the search to.
    • $SEARCH_ID: The search ID obtained from the prior step.

    cURL command example and result: 
    "cursor_id": "a95c100d-1a23-4c57-b545-04b47aaee4d7",
"status": "COMPLETED",
"compressed_data_file_count": 0,
"compressed_data_total_size": 0,
"data_file_count": 10,
"data_total_size": 2139534,
"index_file_count": 0,
"index_total_size": 0,
"processed_record_count": 153867,
"desired_retention_time_msec": 86400000,
"progress": 100,
"progress_details": [],
"query_execution_time": 1196,
"query_string": "select * from events last 5 minutes",
"record_count": 153867,
"size_on_disk": 46736682,
"save_results": false,
"completed": true,
"subsearch_ids": [],
"snapshot": null,
"search_id": "a95c100d-1a23-4c57-b545-04b47aaee4d7"}

  3. When the search status is "COMPLETE", download the event data. The supported formats for download are 'application/json', 'application/csv', 'application/xml', and 'text/table': 
    curl -S -X GET -H "SEC: $SEC_TOKEN" -H 'Range: items=0-49' -H 'Accept: application/csv' 'https://$CONSOLE_IP/api/ariel/searches/$SEARCH_ID/results' -o $FILE_NAME
    • $SEC_TOKEN: Is an authentication token created in the authenticated services window of the admin tab.
    • Accept: Change the 'Accept' header to the wanted format ('application/json', 'application/csv', 'application/xml', or 'text/table').
    • $CONSOLE_IP: The IP of the console you want to post the search to.
    • $SEARCH_ID: The search ID obtained from the first step.
    • -o: cURL flag use to save the results to a file.
    • $FILE_NAME: Name of the file where the results are saved.  

    cURL command example and result: 
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4165    0  4165    0     0   3125      0 --:--:--  0:00:01 --:--:--  3126
    The command can be run multiple times with different data ranges. For example, if the search has a record_count of 100, the data can be exported in 3 different segments: 
    curl -S -X GET -H "SEC: 5c9af508-9ffe-XXXXXXXXX" -H 'Range: items=0-32' -H 'Accept: application/csv' 'https://x.x.x.x/api/ariel/searches/a95c100d-1a23-4c57-b545-04b47aaee4d7/results' -o example.csv
curl -S -X GET -H "SEC: 5c9af508-9ffe-XXXXXXXXX" -H 'Range: items=33-65' -H 'Accept: application/csv' 'https://x.x.x.x/api/ariel/searches/a95c100d-1a23-4c57-b545-04b47aaee4d7/results' -o example1.csv
curl -S -X GET -H "SEC: 5c9af508-9ffe-XXXXXXXXX" -H 'Range: items=66-99' -H 'Accept: application/csv' 'https://x.x.x.x/api/ariel/searches/a95c100d-1a23-4c57-b545-04b47aaee4d7/results' -o example2.csv
    This technique creates three threads that would each grab a third of the events in the search. In comparison to the single-threaded export in the UI, this method is faster to complete.
  4. Once all data from the search is downloaded, the search can be deleted to preserve space on /store/transient: 
    curl -s -X DELETE -H "SEC: $SEC_TOKEN" -k https://$CONSOLE_IP/restapi/api/ariel/searches/$SEARCH_ID
    • $SEC_TOKEN: Is an authentication token created in the authenticated services window of the admin tab.
    • $CONSOLE_IP: The IP of the console you want to post the search to.
    • $SEARCH_ID: The search ID obtained from the first step.

    cURL command example and result: 
    "cursor_id": "a95c100d-1a23-4c57-b545-04b47aaee4d7",
"status": "COMPLETED",
"compressed_data_file_count": 0,
"compressed_data_total_size": 0,
"data_file_count": 10,
"data_total_size": 2139534,
"index_file_count": 0,
"index_total_size": 0,
"processed_record_count": 153867,
"desired_retention_time_msec": 86400000,
"progress": 100,
"progress_details": [],
"query_execution_time": 1196,
"query_string": "select * from events last 5 minutes",
"record_count": 153867,
"size_on_disk": 46736682,
"save_results": false,
"completed": true,
"subsearch_ids": [],
"snapshot": null,
"search_id": "a95c100d-1a23-4c57-b545-04b47aaee4d7"}

    Result
    The administrator can use the QRadar API to download events by using an AQL query.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwsyAAA","label":"Admin Tasks"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions"}]

Document Information

Modified date:
01 April 2023

UID

ibm16964840

Page Feedback