How To

Summary

FindExpensiveCustomProperties.sh is a utility script provided to review the performance characteristics of custom event properties (CEPs).

Objective

Use findExpensiveCustomProperties.sh to troubleshoot performance at the device parsing stage of the QRadar event pipeline.

Steps

1- Depending on whether you are using QRadar SaaS or QRadar SIEM, use one of the following methods to generate the performance report.

Generating the report on QRadar SaaS

To generate the performance log bundle, follow the steps from this document. Select the desired managed host and the findExpensiveCustomProperties script.

Result
The performance log bundle is downloaded, which includes the reports with the expensive Custom Event Properties.

Generating the report on QRadar SIEM

The findExpensiveCustomProperties.sh script is a utility to review the performance impact and characteristics of the CEPs in ecs-ec on any system running that service.

Notes:

The full path to the script is /opt/qradar/support/findExpensiveCustomProperties.sh
The command output is put in the current working directory where the command is run.
It is recommended to run the command in the /store/ibm_support/ path to keep troubleshooting information in one location.

Follow the next steps to run the script and generate the report:

Use SSH to log in to the QRadar console.
Run the next command to create the folder /store/ibm_support/ in case it doesn't exist and to move to it after the creation:
```
mkdir -p /store/ibm_support/
cd /store/ibm_support/
```
Run the command:
```
/opt/qradar/support/findExpensiveCustomProperties.sh
```
By default runs for 120 seconds, it can be modified with the -i option (values are in seconds).
Once completed, a file called CustomProperties-<date>.tar.gz is put in the working directory.
Output Example:
```
CustomProperties-2022-10-24-17-37.tar.gz
```

Result
The report of the expensive CEP is generated.

2- Reviewing and interpreting the generated report on a Linux host.

Decompress the file by using the next command, replace file_name with file name that was created on step four:
tar -xvzf <file_name>

Command example:

tar -xvzf CustomProperties-2022-10-24-17-37.tar.gz

Move to the new decompressed folder, for example:
```
cd CustomProperties-2022-10-24-17-37
```
In this folder, there are files like CustomProperties-<date_stamp>.tabular, which has valuable information and also can be found the reports directory:
```
# ls
CustomProperties-2022-10-24-17-37.tabular  
CustomProperties-2022-10-24-17-37.xml  
reports
```

To get the most expensive CEP in nanoseconds, check the file CustomProperties-<date_stamp>.tabular.
The file might be a little difficult to understand, use the following command to read the information in a shorter and readable format and sort the output based on the TotalCPUTimeNanoseconds:

awk -F $'\t' 'BEGIN {OFS="\t"; print "Pattern\tAverageNanoSeconds\tTotalCPUTimeNanoseconds"} {print $4 "\t" $5 "\t" $2*$5}' CustomProperties-*.tabular | sort -t$'\t' -gk3 | column -s$'\t' -t | less -iS

Output Example:

# awk -F $'\t' 'BEGIN {OFS="\t"; print "Pattern\tAverageNanoSeconds\tTotalCPUTimeNanoseconds"} {print $4 "\t" $5 "\t" $2*$5}' CustomProperties-*.tabular | sort -t$'\t' -gk3 | column -s$'\t' -t | less -iS
Pattern                             AverageNanoSeconds  TotalCPUTimeNanoseconds
Pattern                             AverageNanoSeconds  0
CommandExecuted\]\s\:\s+([^\r\n]+)  52730.333333333336  158191
Value=(\S+)                         2706.48             7.54188e+07
MetricID=(\S+)                      43033.99            1.19919e+09
(?:subject_cn=")([ -~]+?)(?:")      67567.15            2.51395e+10
Destination IP: ([^s]..........)    141219.77           5.25432e+10

You can also use this command to sort the output based on AverageNanoSeconds:

awk -F $'\t' 'BEGIN {OFS="\t"; print "Pattern\tAverageNanoSeconds\tTotalCPUTimeNanoseconds"} {print $4 "\t" $5 "\t" $2*$5}' CustomProperties-*.tabular | sort -t$'\t' -gk2 | column -s$'\t' -t | less -iS

Note: Note: Our Regex monitor metrics consider a CEP expensive if it takes more than 2 milliseconds to parse. The admin can easily identify any CEPs that average more than 500,000 nanoseconds (5 milliseconds) per event. Once the CEPs taking more than 500,000 nanoseconds are addressed, you can proceed with those taking 200,000 nanoseconds.

To correlate the regex pattern to its Property Name, copy the regex pattern and use that pattern to replace 'REGEX_PATTERN' portion at the end of the following query:

Note: For QRadar SaaS, SSH to a Data Gateway and run the psql command from there.
```
psql -U qradar -c "select propertyname,devicetypedescription,regex from ariel_property_view ;" | grep -F 'REGEX_PATTERN'
```
Output example for this regex pattern (?:subject_cn=")([ -~]+?)(?:"):
```
# psql -U qradar -c "select propertyname,devicetypedescription,regex from ariel_property_view ;" | grep -F '(?:subject_cn=")([ -~]+?)(?:")'
 VPN Subject CN                                                       | Cisco Adaptive Security Appliance (ASA)                   | (?:subject_cn=")([ -~]+?)(?:")
```
From the output, we can see that the Property Name is VPN Subject CN, and that it has the Log Source Type of Cisco Adaptive Security Appliance (ASA).
Log in to the QRadar Console.
Click the Admin tab.
Click the Custom Event Properties button.
Use the search bar to look for the Property Name identified in step 8. If there are multiple results, use Ctrl + f to search with the regex pattern. Ensure that it also has the correct Log Source Type:

Result
The administrator can find the expensive CEP by using the script by using the information provided by the findExpensiveCustomProperties.sh script and tune it as needed.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtiAAA","label":"Performance"}],"ARM Case Number":"TS010135184","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions"}]

Tips

QRadar: How to interpret the script findExpensiveCustomProperties.sh

How To

Summary

Objective

Steps

1- Depending on whether you are using QRadar SaaS or QRadar SIEM, use one of the following methods to generate the performance report.

Generating the report on QRadar SaaS

Generating the report on QRadar SIEM

2- Reviewing and interpreting the generated report on a Linux host.

Document Location

Was this topic helpful?

Document Information

UID

Share your feedback

Need support?