IBM Support

MustGather: Collecting logs for IBM Security QRadar SOAR

Troubleshooting


Problem

Use this document to collect logs for IBM Security QRadar SOAR.

Resolving The Problem

IBM Security QRadar SOAR

You can collect all the log files required by the support team to troubleshoot problems with IBM Security QRadar SOAR by using the command resPackageLogs. resPackageLogs is a general-purpose script. It gathers and packages all the logs necessary for troubleshooting into a single file, and stores the file in /root/res-logs-<date>_<time>.tar.gz. The script has a few optional arguments:

Package logs to help troubleshoot problems within the Resilient appliance
  -n, --num-thread-dumps    The number of thread dumps to create
  -d, --thread-dump-delay   The number of seconds between each thread dump
  -s, --stats               Run rstats.sh to gather additional database information
  -r, --restart-service     Restarts the Resilient service after collecting
                            the logs
  -l, --num-daily-logs      How many of the most recent daily archived log files to retrieve.
                            For example : resPackageLogs --num-daily-logs 7, means collecting 7 days log before today.
  -v, --verbose             Display progress info about the collected info
  -t, --target-directory    Specify a target directory to save log files. If the target directory does not exist, it will be created. Default value is the location of your home directory.
Some of the previously mentioned options are not available to older versions of IBM Resilient so run sudo resPackageLogs -h for a full list of the options.

Collecting logs for performance problems

If a performance problem is observed, then the optional arguments can be used:

sudo resPackageLogs -n 6 -d 5

This command tells the script to take 6 thread dumps 5 seconds apart. The total time to dump the thread stacks would be 25 seconds. (6 - 1 dumps) x (5 seconds). If a certain action takes 35 seconds to complete, then these values can be changed so that it takes at least 35 seconds, for example:

sudo resPackageLogs -n 9 -d 5

This dump takes 40 seconds. Run the script with these values, then immediately reproduce the problem.

Log files collected

Log files collected vary depend on the version of IBM QRadar SOAR.

System log files

  • /usr/share/co3/logs/catalina.out
  • /usr/share/co3/logs/catalina.err
  • /usr/share/co3/logs/client.log
  • /usr/share/co3/logs/client_access_log<YYYY-MM-DD>.log
  • /usr/share/co3/logs/monitoring.log
  • /usr/share/co3/logs/update_database.log
  • /usr/share/co3/bin/jvmenv.sh (generated in V28+) 
  • /var/lib/pgsql/9.x/data/pg_log/postgresql-<day of week>.log or /var/lib/pgsql/12/data/log/postgresql-<day of week>.log
  • /var/log/elasticsearch/elasticsearch.log
  • /proc/meminfo
  • /proc/cpuinfo
  • /proc/loadavg
  • /var/log/resilient-messaging/resilient-messaging.out
  • /var/log/resilient-messaging/resilient-messaging.err
  • /var/log/resilient-messaging/resilient-messaging.log
  • /var/log/resilient-email/resilient-email.out
  • /var/log/resilient-email/resilient-email.err
  • /var/log/resilient-email/resilient-email.log
  • /var/log/resilient-scripting/resilient-scripting.out
  • /var/log/resilient-scripting/resilient-scripting.err
  • /var/log/resilient-scripting/resilient-scripting.log
  • /var/log/resilient-scripting/resilient-scripting-monitoring.log
  • /var/log/resilient-app-manager/resilient-app-manager.log

Database details

  • Quartz job details
  • Connection details
  • Locks
  • Lock dependencies
  • Backup, table and database information (rstats.txt)

System info (system_info.txt):

  • Resilient version
  • Time when the script was run

Document Location

Worldwide

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSIP9Q","label":"IBM Security SOAR"},"Component":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB24","label":"Security Software"}},{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSA230","label":"IBM Security QRadar SOAR"},"ARM Category":[{"code":"a8m0z000000cwJWAAY","label":"Support"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
07 October 2022

UID

ibm11846545

Page Feedback