Troubleshooting
Problem
The IBM QRadar SOAR app runs what is called the "poller" to process offenses, incidents and notes. In some circumstances, the poller can run for some time which can cause delay in escalating offenses and creating incidents in IBM SOAR.
Symptom
It might be observed that offenses created in IBM QRadar are not escalated to IBM SOAR in a timely manner.
Cause
The poller does a number of things such as searching for offenses, searching for incidents that have an IBM QRadar offense ID, escalating offenses to IBM SOAR, creating notes and closing offenses. Once it finishes these actions it waits, and then repeats the process.
In one instance it was found that the poller was taking a long time when searching for incidents in IBM SOAR. The total duration of the poller was largely made up of the time it took to search for open incidents.
Diagnosing The Problem
After enabling debug and analysing the circuits.log during the poller process the following entries were found. These entries show the amount of time it take for the poller to search IBM SOAR for all open incidents.
Notice the number of organizations and the period of time elapsed from "Getting incidents for org ...." and "No more QRadar incidents found ..." This is the period of time it took to search for incidents in IBM SOAR.
2021-03-24 08:38:09,983 DEBUG [resilient_helpers] Getting incidents from org 207
2021-03-24 08:38:15,078 DEBUG [resilient_helpers] No more QRadar incidents found in org 207
2021-03-24 08:38:49,777 DEBUG [resilient_helpers] Getting incidents from org 209
2021-03-24 08:38:55,645 DEBUG [resilient_helpers] No more QRadar incidents found in org 209
2021-03-24 08:38:55,646 DEBUG [resilient_helpers] Getting incidents from org 210
2021-03-24 08:40:26,976 DEBUG [resilient_helpers] No more QRadar incidents found in org 210
2021-03-24 08:40:26,976 DEBUG [resilient_helpers] Getting incidents from org 211
2021-03-24 08:40:32,297 DEBUG [resilient_helpers] No more QRadar incidents found in org 211
2021-03-24 08:40:32,298 DEBUG [resilient_helpers] Getting incidents from org 212
2021-03-24 08:44:48,215 DEBUG [resilient_helpers] No more QRadar incidents found in org 212
2021-03-24 08:44:48,215 DEBUG [resilient_helpers] Getting incidents from org 213
2021-03-24 08:45:56,170 DEBUG [resilient_helpers] No more QRadar incidents found in org 213
2021-03-24 08:45:56,170 DEBUG [resilient_helpers] Getting incidents from org 214
2021-03-24 08:46:35,194 DEBUG [resilient_helpers] No more QRadar incidents found in org 214
2021-03-24 08:46:35,194 DEBUG [resilient_helpers] Getting incidents from org 215
2021-03-24 08:46:40,591 DEBUG [resilient_helpers] No more QRadar incidents found in org 215
2021-03-24 08:46:40,592 DEBUG [resilient_helpers] Getting incidents from org 216
2021-03-24 08:46:45,964 DEBUG [resilient_helpers] No more QRadar incidents found in org 216
2021-03-24 08:46:45,964 DEBUG [resilient_helpers] Getting incidents from org 217
2021-03-24 08:46:51,371 DEBUG [resilient_helpers] No more QRadar incidents found in org 217
2021-03-24 08:46:51,372 DEBUG [resilient_helpers] Getting incidents from org 218
2021-03-24 08:47:00,839 DEBUG [resilient_helpers] No more QRadar incidents found in org 218
2021-03-24 08:47:00,840 DEBUG [resilient_helpers] Getting incidents from org 219
2021-03-24 08:47:23,204 DEBUG [resilient_helpers] No more QRadar incidents found in org 219
2021-03-24 08:47:23,204 DEBUG [resilient_helpers] Getting incidents from org 220
2021-03-24 08:47:28,583 DEBUG [resilient_helpers] No more QRadar incidents found in org 220
2021-03-24 08:47:28,583 DEBUG [resilient_helpers] Getting incidents from org 221
2021-03-24 08:47:47,466 DEBUG [resilient_helpers] No more QRadar incidents found in org 221
2021-03-24 08:47:47,466 DEBUG [resilient_helpers] Getting incidents from org 222
2021-03-24 08:47:52,653 DEBUG [resilient_helpers] No more QRadar incidents found in org 222
2021-03-24 08:47:52,653 DEBUG [resilient_helpers] Getting incidents from org 223
2021-03-24 08:49:40,594 DEBUG [resilient_helpers] No more QRadar incidents found in org 223
2021-03-24 08:49:40,595 DEBUG [resilient_helpers] Getting incidents from org 224
2021-03-24 08:49:58,023 DEBUG [resilient_helpers] No more QRadar incidents found in org 224
2021-03-24 08:49:58,024 DEBUG [resilient_helpers] Getting incidents from org 225
2021-03-24 08:51:08,531 DEBUG [resilient_helpers] No more QRadar incidents found in org 225
As mentioned, the poller will search for open incidents in IBM SOAR that have a value added to the custom field qradar_id.
The client was asked to run the following command on the IBM SOAR server.
sudo -u postgres psql co3 -c "SELECT org_id, count(*) from monapp.incidents where istatus_code='A' group by org_id order by org_id"
This command returned around 68,000 open incidents, all of which the app retrieved from API calls to IBM SOAR. The output showed that some organizations had more open incidents than others and that is why for those organizations the poller took longer to retrieve the open incidents.
Resolving The Problem
The client was asked to close incidents that did not need to remain open. After the number of open incidents decreased to 2500 across all organizations, the poller ran for a fifth of the time it ran for previously.
This example was of an MSSP client where by they will naturally have more organizations. The premise is the same for non-MSSP regardless of the number of organizations.
Related Information
Document Location
Worldwide
[{"Type":"SW","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSIP9Q","label":"IBM Security SOAR"},"ARM Category":[{"code":"a8m0z0000001gyGAAQ","label":"Integrations-\u003EQRadar app"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)"},{"Type":"SW","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEGM63","label":"IBM Security QRadar SOAR on Cloud"},"ARM Category":[{"code":"a8m0z0000001gyGAAQ","label":"Integrations-\u003EQRadar app"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)"}]
Was this topic helpful?
Document Information
Modified date:
17 February 2022
UID
ibm16442961