Troubleshooting
Problem
Resolving The Problem
- Event Collector (EC)
- Event Processor (EP)
- Event Correlation Ingress (ECS-EC-Ingress)
- Magistrate Processing Core (MPC)
Event Collector (EC)
QRadar accept events from various sources such as syslog, syslog-TCP, SNMP, and others. QRadar can establish outbound connections and retrieve events, by using SCP, SFTP, FTP, JDBC, Checkpoint OPSEC, and SMB/CIFS. As events are received, they are placed into input queues for processing. The queue sizes varies based on the protocol and method used, and from these queues, events are fed to the Event Parsers and DSMs.
Known Log Sources (based on the source IP address or hostname in the header) are parsed and coalesced into records, and unknown sources are redirected to Traffic Analysis for Log Source autodetection. If new sources are found, a configuration request message is sent back to the QRadar console to have the new source added, unless you disable Autodetection, or if you have reached your Log Source limit, which is controlled by your licenses.
EPS and License Rates
Events are taken from the source queues at a rate dictated by your QRadar licenses, and each event processor can have a distinct licensed event rate. If you are receiving events at higher rate than your license, they wait in the source queues until the rate drops. However, if you continue to send at a higher rate than your license and the queue is filled, your system drops excess events with a warning that you have reached your license limit.
For more information on Event and Flow buffering, see the Event and Flow buffering guide.
Data Collection and Sources
Each Log Source type has its own queue:
- Syslog: 100000
- Syslog (TCP): 100000
- Windows Event Log: 100000
- Windows IIS/Exchange/DHCP: 10000 each
- JDBC: 10000
- SDEE/EStreamer: 10000 each
- Log File Protocol (scp/sftp/ftp): 50000
The queues are processed in sequence with parsing servicing each queue in series. For example, if a JDBC connection was to burst to 5000 EPS for a few seconds, it's possible that the queue would fill and drop events, while the syslog queue, at a burst of 10000 EPS for 5-10 seconds would not, since the queue size is much larger. The size varies because most sources come in over UDP.
Events are parsed and then coalesced based on common patterns across events. Once 4 events are seen with the same source IP, destination IP, destination port, and username, subsequent messages for up to 10 seconds of the same pattern are coalesced together. Coalescing reduces the amount of duplicate data stored.
Traffic Analysis and Auto Detection
Events that come into the system from new sources that were not previously detected are redirected to the traffic analysis (autodetection) engine. These events are run through log sources types that are defined in traffic analysis. A minimum number of messages per minute are required to create a Log Source, and this number might be different for multiple devices. A number of devices are common from one sort to another, such as AIX, Solaris, Linux, and might not be enabled in traffic analysis. In these instances, or in cases where event rates are low from particular devices, we recommend you create the Log Sources manually.
If devices are detected incorrectly, especially if they appear as Log Source types not present on your network, it is possible to remove some of these devices from Traffic Analysis with the assistance of IBM Support.
Identity Updates
In addition to parsing normalized fields, some log sources and event types also update identity. Identity records are typically generated from messages that indicate a successful connection or authentication to a resource, such as an SSH login or windows domain authentication. Other events can still have a username parsed out, such as an antivirus update from a windows host or program started from windows, but do not indicate an authentication, so these kinds of events would not generate an identity update.
Identity updates are sent from the event collectors parsing code directly back to the console and do not go through the remaining event pipeline. These updates are stored with the host information under the Asset tab of the QRadar user interface.
Vulnerability Assessment (VA) data does not come into the event pipeline. VA and Scanner information can be retrieved and processed on any system, which is then parsed by the VIS component, sent back to the console, then stored with the Asset information as well.
Event Processor (EP)
The Event Processor accepts normalized messages from the EC and runs them through the custom rules as defined in the Rules section of the UI. As events are processed and matched, the EP generates notifications, syslog, SNMP, email messages, new events, and offenses as required.
Custom Rules Engine
The Custom Rules Engine (CRE) is responsible for processing events received by QRadar and comparing them against defined rules, keeping track of systems involved in incidents over time, generating notifications to users, and generating offenses. When a rule is confirmed as match, the event processor executes what is defined in the rule response section.
If you configure your system to generate an email or SNMP message, these messages are delivered from the Event Processor where the events are processed, not the console itself. In this situation, you must ensure that the machine has access to deliver these messages, either by an email relay or to the SNMP server that is used.
Context & Routing
The context and routing engine is responsible for passing off events to storage and on to the Magistrate Processing Core (MPC) when required.
Event (Ariel) Storage
Ariel Storage is where data gets written from the event pipeline out to disk. Data is written out to disk on any system that has an event processor. Ariel storage settings are configured on the console in System Settings - Ariel Database Settings. Ariel retention setting is a global configuration value, so all event processors are configured with the same retention.
QRadar deletes data only as required to maintain 85% free disk space. The system collects data until 85% is reached then begins compressing that data. After all data up to 6 hours old is compressed, the system then goes back and starts deleting the oldest data, compressing and deleting until the retention threshold is hit. When QRadar hits 95% usage on the /store/ partition, it shuts down and stops collecting.
Event Correlation Ingress (ECS-EC-Ingress)
ECS-EC-Ingress is a self-contained service that collects events in a buffer that which allows services to be deployed while not interrupting event collection. After ECS-EC and ECS-EP services restart, ECS-EC-Ingress spools the data back to ECS-EC and ECS-EP. Other functionality such as reports and searches might still require an administrator restarting after changes are deployed.
Magistrate Processing Core (MPC)
The Magistrate Processing Core (MPC) is responsible for correlating offenses with event notifications from multiple event processors, monitoring active offenses, generating email notifications when required, and providing access to offense information to the users through the Offenses tab.
Offense Rules:
Similar to the Event CRE, Offense rules have the ability to monitor offenses in QRadar and take action, typically email messages, once they occur.
Offense Management:
The system can monitor up to 2500 active offenses at any one time and has recall slots for 500 offenses that were idle for more than 30 minutes. Inactive (no updates for 5 days) offenses are limited to 100,000. As offenses go periodically dormant (no updates in 30 minutes) and later updated, the offense manager can update up to 500 of these offenses.
Offense Storage:
Offense information is stored in the Postgres DB.
Was this topic helpful?
Document Information
Modified date:
01 February 2023
UID
swg21622228