Identifying DSM and optimized custom property issues

To help you troubleshoot performance degradation, identify issues with any DSM extensions that were recently installed or custom property that was recently enabled.

About this task

A DSM extension creates custom parsing methods by using regex pattern matching to extract event data from unsupported or incomplete log sources. Optimized custom properties use regular expression patterns to extract data from events as they are parsed.

The regex patterns that are used in your DSM extension or optimized custom property can impact event processing in IBM® QRadar®. Inefficient regular expressions can incorrectly route data directly to storage, degrade QRadar performance, and affect event processing.

DSM and optimized custom property issues can cause the following system notification:

Performance degradation has been detected in the event pipeline. 
Events were routed directly to storage.

Procedure

Disable any DSM extension or custom property that is recently installed or enabled.
Choose one of the following options:
- If QRadar stops dropping events and you receive a system notification, then review your DSM extensions or custom properties to identify and improve the inefficient regex patterns.
- If QRadar continues dropping events, then multiple DSM extensions or custom properties might be causing a problem with the event pipeline.
Use SSH to log in to the QRadar Event Processor that is dropping events and type the following command:
```
/opt/qradar/support/threadTop.sh –p 7777
```
The command displays the data processing engine activity. The following table describes the columns in the output:

Table 1. Data processing engine columns

Columns Description

Server Port or process.

ID Process ID.

MSecs CPU time.

Name Process name.
If parser threads run longer than 1500 milliseconds, review the Java thread stacks by typing the following command:
```
/opt/qradar/support/threadTop.sh -p 7777 -s -e ".*Event Parser.*" | less
```

Table 1. Data processing engine columns
Columns	Description
Server	Port or process.
ID	Process ID.
MSecs	CPU time.
Name	Process name.

What to do next

If the Java thread stack contains java.util.regex.Pattern$Curly.match, then the performance degradation might be caused by your expensive DSM extensions or custom properties. For more information, see Expensive DSM extensions were found or Expensive custom properties found.

If the Java thread stack doesn't have expensive regular expressions, then your DSM extensions or custom properties might have parsing issues. For more information, see the parsing issues topic in the IBM QRadar Log Sources User Guide.