QRadar: Data Synchronization App FAQ

Question & Answer

Question

What are the features and requirements for the IBM QRadar Data Synchronization App?

Answer

Administrators are advised to read the QRadar: Data Redundancy (DR) and support policies and IBM® QRadar Data Synchronization app Documentation to familiarize themselves with these deployments.

For readability, the content in this technical note is divided into categories

Basic Terminology
Requirements
Features
Using the App.

Section 1: Basic terminology

What is the Data Synchronization App?

The IBM® QRadar Data Synchronization App is a licensed product that is used to keep Ariel data and some configurations synchronized between a "Main Site" and a "Destination Site".

This application enables administrators to have two nearly identical QRadar deployments in separate geographic environments and reduce the recovery time in a disaster scenario.

Refer to Section 4, question: "Is there a list to check which configuration settings are transferred from the main site to the destination site?" question in this article to get a full list of settings synchronized.

What is the main site?

The main site is your primary production site. By default is the "Active" site.

Note: Do not confuse the IBM® QRadar Data Synchronization app Active and Standby status with IBM® QRadar HA status. Although the term is the same, they are not equivalent.

What is the destination site?

The destination site is deployed in a remote location to mirror the main site. By default is the "Standby" site.

Important: Do not confuse the IBM® QRadar Data Synchronization app Active and Standby status with IBM® QRadar HA status. Although the term is the same, they are not equivalent.

Section 2: Requirements

Do the main site and destination site need to be on the same QRadar version, including the patch version?

Yes, both sites need to have the same QRadar version.

Also, the administrators must ensure to always run the latest available version of IBM® QRadar Data Synchronization app.

On which QRadar versions can the IBM® QRadar Data Synchronization app run?

DSAPP version 2.1.x, requires QRadar version 7.4.0 Fix Pack 3 as the minimum version to be installed.

DSAPP version 3.0.x, requires QRadar version 7.4.1 Fix Pack 2 as the minimum version to be installed.

For more information, see Supported versions of QRadar for detailed information about other version requirements.

There is a 1:1 host-mapping requirement for hosts that process Ariel data (Event or Flow data). Which hosts does this requirement apply to?

Refer to Table1. Mapping requirements for QRadar components for the list of hosts that require and don't require 1:1 mapping.

Important: Hosts that don't require 1:1 mapping can exist on any of the sites because IBM® QRadar Data Synchronization app does not interact with them.

Is the App Host Appliance permitted in IBM® QRadar Data Synchronization app?

App Hosts are permitted in the main site and destination site deployments, but App data is not synchronized. The administrator must manually configure the apps on each deployment separately.

If I have a smaller deployment (fewer devices) on my destination site will this application work?

No, refer to Table1. Mapping requirements for QRadar components for the list of hosts that require and don't require 1:1 mapping.

Can I have a console in HA on the main site paired with a console without HA on the Destination Site?

Yes, the same is true for other managed hosts.

Can I choose which managed hosts to synchronize and not have a 1:1 mapping for all hosts?

No, it is not possible. All the required managed hosts need to have a 1:1 mapping.

Administrators interested in this feature can raise a Request for enhancement.

Refer to Table1. Mapping requirements for QRadar components for the list of hosts that require and don't require 1:1 mapping.

Does IBM® QRadar Data Synchronization app work with DLCs (Disconnected Log Collectors)?

Yes, The DLC is the recommended solution for log collection in future releases and doesn’t require a 1:1 mapping.

Is IBM® QRadar Data Synchronization app a paid-for service?

Yes, the IBM® QRadar Data Synchronization app solution is a paid-for service. To be within IBM license compliance and for the IBM® QRadar Data Synchronization app to be effective, you must purchase QRadar Data Synchronization licenses.

Licensing is handled on a per-node basis. Administrators need one license per node in the Destination deployment and must contact their sales representative for more information.

If I got the previous DR license model in place, can I use the new IBM® QRadar Data Synchronization app?

Yes. The existing (old) DR parts can be used to deploy IBM® QRadar Data Synchronization app instead.

Administrators must contact their sales representative for more information.

Will the application work in a Managed Security Service Provider (MSSP) deployments with Domains and Tenants?

Yes. Since version 2.0.0 of IBM® QRadar Data Synchronization app, restoring domains and tenants is supported.

Can I use the IBM® QRadar Data Synchronization app in a Federal Information Processing Standards (FIPS) enabled environment?

No, QRadar deployments that use FIPS are not currently supported by IBM® QRadar Data Synchronization app.

Administrators interested in this feature can raise a Request for enhancement.

Does IBM® QRadar Data Synchronization app work in cloud environments?

Yes. The QRadar Data Synchronization and Console-only failover works with QRadar on-prem and also environments where administrators deploy appliances in the cloud from marketplace images, such as AWS or Azure. For more information about marketplace images, see QRadar cloud marketplace appliance images. The QRadar Data Sync app is not QRadar on Cloud ready.

To use the Data Sync app with marketplace cloud images, administrators must ensure comply with the following mandatory requirements:

Root user access to all appliances on both main and destination sites.
Network connectivity between the paired hosts.

For more information, see What are the required ports for IBM® QRadar Data Synchronization app to communicate between main and destination sites.

Can I use the IBM® QRadar Data Synchronization app on an All-in-One appliance?

Yes, it is possible. The Administrator must deploy another All-in-One appliance on the destination site to meet the 1:1 mapping requirement.

Does the IBM® QRadar Data Synchronization app verify hardware compatibility?

No. The app checks only the appliance type. The administrator must ensure hardware compatibility. For example, a Console (31xx) must exist on the main site and the destination site for the pairing process to occur.

For more information, see QRadar: How to determine the appliance type for each host in a distributed deployment to identify the appliance types on a deployment.

Does the IBM® QRadar Data Synchronization app check interfaces name, for example, eth0?

No. Interface names are not considered.

What is the minimum bandwidth required for the IBM® QRadar Data Synchronization app?

Bandwidth requirements must be assessed by IBM®Security Expert Labs as this calculation depends on many factors: EPS, average payload size, average record size, the kind of log sources being ingested, how often you want to synchronize the data, what other custom settings the deployment has in QRadar, and so on.

What are the required ports for the IBM® QRadar Data Synchronization app to communicate between main and destination sites?

From	To	Port and Protocol	Description
Main site Console	Destination Console	443 TCP	API and failover communication
Main site Console	Destination Console	22 TCP	Configuration and Data Sync
Main site managed hosts	Destination site managed hosts	22 TCP	Data Sync
Destination Console	Main site Console	443 TCP	API and failover communication
Destination Console	Main site Console	22 TCP	Reverse Data Sync
Destination site managed hosts	Main site managed hosts	22 TCP	Reverse Data Sync

Section 3: Features

Does the IBM® QRadar Data Synchronization app copy ariel indexes?

No, the destination site automatically re-creates the indexes.

Does the IBM® QRadar Data Synchronization app synchronize Offenses?

No, the offenses are not synchronized between sites.

Does the destination site process events?

The destination site, when in standby state, has only tlssyslog protocol active, which allows the destination console to process internal health monitoring events. The administrators are advised to send only external syslog events when the destination site becomes active. When active, the destination site processes all events.

Important: Any syslog event sent to the destination site while in standby is processed, but is not advisable nor supported.

Is it possible to search and report on the destination site when it is not active?

Yes, when the destination site is on standby the administrators are able to search and report on the data that is synchronized.

Can the destination site generate offenses while on Standby?

No, correlation services on the destination are suppressed while on standby.

Is the IBM® QRadar Data Synchronization app similar to offsite forwarding?

No, they are different. The IBM® QRadar Data Synchronization app allows administrators to copy both config and data (Event and Flow data) and configure the intervals and bandwidth.

How often is data synchronized?

By default, data synchronizes at 1-minute intervals. These intervals are configurable from the app settings.
For more information, see Editing the configuration of the main site and destination site.

image-20220428161658-1

How does ariel copy decide the order of data files to synchronize?

The ariel copy process goes from oldest to newest based on the start date the administrator configures in the app settings. It synchronizes payloads first, then records.

Section 4: Using the App

How do I synchronize configuration changes from the main site to the destination site?

Administrators can configure the IBM® QRadar Data Synchronization app to automatically restore the configuration backup from the main to the destination site on a schedule or on-demand.

For more information, see syncing the configuration from the main site to the destination site.

Is there a list to check which configuration settings are transferred from the main site to the destination site?

Yes. Here is the list:

Asset Identity Exclusions
Custom Event and Flow Properties
Custom Event Mappings
Custom Log Source Types
Custom QIDs
Custom Rules
FGroup Data
Flow Sources
Forwarding Destinations
Historical Rules and Searches
Log Source Extensions
Log Sources
Network Hierarchy
Obfuscation Data
Reference Data
Reference Sets
Report Templates
Retention Settings
Routing Rules
Saved Searches
System Settings

Is there an option to customize which configuration settings (for example, custom rules, reports or searches) to synchronize?

No. All the items mentioned in Section 4, question: "Is there a list to check which configuration settings are transferred from the main site to the destination site?" are synchronized across and restored on the destination. Specific settings cannot be selected individually.

Which items are NOT synchronized from the main site to the destination site?

The following items are not synchronized:

Users
User roles
Security profiles
Authentication Settings.
Offenses
Report data (past generated reports)
Trusted Certificates (on the managed hosts)
Keystores (on the managed hosts)
Apps and app data

If there is a failure on the main site, does the destination site activate automatically?

No, to activate the destination site, the Administrator must perform a manual failover. This action can be initiated only from the destination site app.

For more information, see Activating the destination site after a failure of the main site.

Is it possible to upgrade QRadar on both, the DC and the DR site, without any downtime?

The Disaster Recovery (DR) server can be upgraded first, followed by the upgrade of the Data Center (DC) server. However, there will be downtime during the DC server upgrade. During this process, failover to the DR server cannot be performed due to the version mismatch between the DC and DR sites.

What happens when the destination site becomes active?

When the administrator activates the destination site the following occurs:

On the main site
Ariel data synchronization and all Ariel copy profiles are disabled, but no services are suppressed.

On the destination
The services that were suppressed while on 'STANDBY' are enabled upon activation, the destination site behaves like a normal QRadar Deployment and is ready to collect and process events and generate offenses and reports.

Can the destination site become active even if the main site is available?

Yes, but the "Ariel Data Copy" feature from the main site to the destination site is disabled to avoid data duplication. This copy resumes once the main site is "Reactivated".

Does the main site get disabled when you activate the destination site?

No, the IBM® QRadar Data Synchronization app does not stop or suppress any services on the main site. The services on the main site remain active and running at all stages and new data can be ingested as normal.

Are "data sources" automatically moved to the destination site when the failover to the destination site occurs?

No, data sources are not automatically moved. When activated, the destination site starts the event and flow collection services on all the required hosts and starts listening for events and flows.

How are the "data sources" redirected to the destination site?

For events and flows to reach the destination site, the administrator must redirect the data sources to the collector appliances in the destination site.

Log sources that pull data (for example, AWS, Kafka, Azure, other APIs, JDBC, and so on.) start pulling data automatically into the destination site when activated. A successful configuration restore is required before the destination site is activated.

Log sources that push data (for example, regular syslog, TLS syslog, TCP multiline, or UDP multiline, and so on.) require the administrator to reconfigure each source to the destination site collector appliances in order to continue receiving live logs when activated.

There are 3 possible methods to accomplish the log source reconfiguration:

Manual reconfiguration
DNS Resolution
Using a Load Balancer

Review the Protocol Configuration options page from the QRadar DSM Configuration guide for a group list of pull and push log sources common examples.

Check with IBM®Security Expert Labs for details on which option would work for your environment.

Do WinCollect (managed or stand-alone) agents automatically reconnect moving between sites?

No, WinCollect agents can be configured to send their logs to a DNS or load balancer to make it easy to reconfigure their destination collector.

WinCollect managed agents require a new token created on the destination site (required to push changes to the WinCollect agents) and agents must be manually reconfigured to the destination site.

Does the Data Synchronization have any performance or bandwidth impacts?

There is a configurable bandwidth limit to throttle the synchronization of Ariel data from the main site to the destination site that prevents high disk IO and high-bandwidth utilization. This setting can be configured on a 'per-host-pair' basis.

For more information, see Changing the bandwidth for paired hosts.

How long does it take to perform a failover to the destination site?

A failover process can take up to 15 minutes. A configuration deploy change is required for the destination site to become active.

Note: This time does not consider the configuration deploy changes duration or data sources reconfiguration.

What does the "Fail Back to main site" option on the destination site do?

This option starts the "Reverse Ariel Copy" feature that can be initiated only when the destination site is the active site.

This feature copies events and flows data collected by the destination site (since becoming active) back to the main site. This feature must be activated manually to ensure that data is only sent back to the main site that is fully operational.

What are the steps to failback to the main site when the main site is ready to resume operations?

The Administrator needs to start the failback to the main site task from the destination site's app.
The failback process starts the "Reverse Ariel Copy" of all the data received on the destination since it was activated up to the top of the hour on which you started the failback process.
For example, if the failback starts at 07:35 PM ariel copy selects ALL data received on the destination, starting from the time it was activated, up to 07:59 PM. Depending on the amount of data, this process can take a long time to complete.
The Administrator must wait until the "Ariel Copy Complete" notification is received on the destination app UI.
Once the notification is received, proceed with the "Reactivate the main site" step.

For more information, see Restoring the main site.

If configuration changes are done on the destination site while Active, can these changes be restored to the main site?

The IBM® QRadar Data Synchronization app currently does not have the functionality to restore a backup from the destination site to the main site.

The “failback” feature of the IBM® QRadar Data Synchronization app currently copies only the event and flow data that was collected in the destination site back to the main site, but the configuration is not synchronized back. The Administrator needs to manually update the configurations on the main site.

Important: Do not perform a regular backup and restore on the main site.

I am having issues with IBM® QRadar Data Synchronization app, what logs can I check?

Apart from the usual qradar logs, administrators can check the following log files:

Log file Location	Description
/var/log/qradar/qdr/qdr.log	Ariel Copy status
/var/log/qradar/qdr/qdr_rsync.log	Rsync used by Ariel Copy
/store/docker/volumes/qapp-<id>/log/app.log	General app logs
/store/docker/volumes/qapp-<id>/log/error.log	App Error logs
/store/docker/volumes/qapp-<id>/log/access.log	Host access to the app
/store/docker/volumes/qapp-<id>/log/supervisord.log	Supervisord startup and child process
/store/docker/volumes/qapp-<id>/log/startup.log	App setup logs

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwt3AAA","label":"QRadar Apps"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"7.4.0;and future releases"}]

Tips