IBM Support

QRadar: Best practices when using the Content Management Tool to export custom data

News


Abstract

The Content Management Tool (or CMT tool) is a CLI-based script that allows users to export or import custom content in a QRadar environment. Administrators can export rules, reports, dashboards, log sources, and custom event properties, and other data. There is a v1 version of the script that is available in all versions of QRadar, with a v2 version in 7.4.2 and later. This technote focuses only on CMT v1.

Content

Before you begin:
  • The Content Management Tool might not work as expected with QRadar 7.4.x versions.
  • Do not attempt to export all types of content at the same time unless you run a small environment. For example, either of the following commands can take a significant amount of time to complete: 
    /opt/qradar/bin/contentManagement.pl --action export --content-type all
    ./contentManagement.pl --action export -c all
  • The -c option allows administrators to use either the content name or the numerical value associated with the content. For example, Reports have a value 10.
What to know before you export data with the CMT tool:
  • Exporting data from the command line is not a replacement for a proper QRadar environment migration. The supported and best method to backup and restore data in QRadar is to create a configuration restore from the Console (Admin > Backup and Restore). For more information, see Restore QRadar configuration and data.
  • The CMT tool is best used for a scenario where a customer created content in a development or QA environment, and would like to export a portion of their content to their production environment without having to go through a full configuration restore.
  • There is no progress meter during the export. The command line does not indicate what step of the export it is on.  Depending on environment size, it could take hours, days, or in some cases weeks for the export to complete in large environments.
  • If you export a rule or log source from the source environment with a name that also exists with the exact name in the target environment, the import fails. You need to either rename the log source or rule on the receiving environment or delete it before you import the content.
  • If you are attempting to export log sources or customization for a log source, the DSM Editor can export log sources, custom properties, parsing behavior overrides, and mapping information as a package. If you are testing or working with log sources in a lab or QA environment, it is best to export your log sources from the DSM Editor. For more information, see Exporting content from the DSM Editor.

Content Management Tool export best practices

  1. Use SSH to log in to the QRadar Console as root user.
  2. Type screen to enter screen mode. 
    image 10045
    Note: If you lose your SSH connection while CMT is exporting data, screen ensures the operation does not stop.
  3. From the command-line use Command Management Tool to export your content in sections.
    It is best to do content in sections by Content type. The option -c can be used with either the Context text identifier or Numeric Identifier.
    Content Command Use case
    Custom Rules /opt/qradar/bin/contentManagement.pl -a export -c 3 --id all The value 3 is the numeric identifier for Custom Rules.
    Custom Rules /opt/qradar/bin/contentManagement.pl -a export -c customrule --id all -o /store/IBM_Support The -o option creates a .zip file with all custom rules in the specified directory
    Log Sources /opt/qradar/bin/contentManagement.pl -a export -c 17--id all The value 17 is the numeric identifier for Log Sources.
    Log Sources /opt/qradar/bin/contentManagement.pl -a export -c sensordevice --id all
    Reports /opt/qradar/bin/contentManagement.pl -a export -c 10 --id all --global-view The value 10 is the numeric identifier for reports.
    The option --global-view is only for reports.
    Reports /opt/qradar/bin/contentManagement.pl -a export -c report --id all --global-view Note the string value is singular "report".
    Dashboards /opt/qradar/bin/contentManagement.pl -a export -c 4 --id all The value 4 is the numeric identifier for dashboards.
    Dashboards /opt/qradar/bin/contentManagement.pl -a export -c dashboard --id all Note the string value is singular "dashboard".
    Reference data /opt/qradar/bin/contentManagement.pl -a export -c 28 --id all -e
    The value 28 is the numeric identifier for reference data. 
    The option -e if only for reference data.
    Reference data /opt/qradar/bin/contentManagement.pl -a export -c referencedata --id all --include-reference-data-elements Instead of -e, you can use --include-reference-data-elements to include the elements inside the reference data.
    1. The export is written to disk as a .zip file. 
      /opt/qradar/bin/contentManagement.pl -a export -c customrule --id all -o /store/IBM_Support
[INFO] Initializing Content Management Tool...
You have new mail in /var/spool/mail/root
[root@qradar733 store]# /opt/qradar/bin/contentManagement.pl -a export -c customrule --id all -o /store/IBM_Support
[INFO] Initializing Content Management Tool...
[INFO] (ContentManagementCLI) Start Time: 2021-05-21 15:13:28
[INFO] Starting export process
[INFO] Processing Export: content-type customrule id all
[INFO] Exporting content of type [custom_rule] with id [all]
[INFO] Export Summary:
[INFO]  Content Type - [Number of items exported]
[INFO]          - qidmap - [87]
[INFO]          - reference_data_rules - [15]
[INFO]          - sensordevicetype - [42]
[INFO]          - sensordeviceprotocols - [39]
[INFO]          - sensordevicecategory - [1]
[INFO]          - ariel_property_expression - [1]
[INFO]          - ariel_regex_property - [1]
[INFO]          - reference_data - [25]
[INFO]          - offense_type - [6]
[INFO]          - custom_rule - [350]
[INFO] SUCCESS: Compressed exported bundle can be found here /store/IBM_Support/custom_rule-ContentExport-20210521151330.zip

    Importing and updating content best practices

    When administrators import content they need to be aware that they can overwrite Custom Rules, Log Sources, Dashboards. Rules or log sources that have the same name overwrite existing values during the import and there is no prompt to skip or ignore duplicates as the import runs. Overwriting content from the command line can create more issues when they export and import from different versions of QRadar. It is always best to keep frequent backups of your deployment.

    Before you begin

    • Ensure you have a recent configuration file for your Console before you attempt to import data with the CMT tool. 
    • When you import content from your test environment, ensure that you do not use duplicate rule or log source names between production and your lab or QA environments, unless you explicitly use the update function.
    • If the content contains overridden system rules, use the update action instead of the import action to ensure that the rules are imported correctly.

     
    Updating content
    By using the update function, existing content with the same name is updated rather than overwritten.
    1. Use SSH to log in to the Console as root user.
    2. Navigate to the .zip file for the content you choose to update.
    3. To update content type the command: 
      ​/opt/qradar/bin/contentManagement.pl --action update -f custom_rule-ContentExport-20210521151330.zip
      Results
      The content of the same naming structure is updated.
    Importing Content
    When you import content, there is the potential to overwrite an existing entry rather than updating it. Use the import function with caution. A recent nightly configuration backup is required before you begin an import in case your CMT import overwrites data or behaves unexpectedly.
    1. Log in to the QRadar Console as root user.
    2. Navigate to the .zip file you choose to import.
    3. To import content, type the command: 
      /opt/qradar/bin/contentManagement.pl --action import -f custom_rule-ContentExport-20210521151330.zip
      Result
      Content is imported and any current content of the same name is overwritten.

    [{"Type":"SW","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwsyAAA","label":"Admin Tasks"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)"}]

    Document Information

    Modified date:
    19 December 2023

    UID

    ibm16450761

    Page Feedback