Troubleshooting
Problem
When the /var partition in QRadar® SIEM does not have enough space, it can affect the regular functioning of QRadar. The purpose of this article is to help the administrator with the removal of files and directories when the /var partition has not enough available disk space.
Symptom
Lack of available space in the /var partition can cause the following issues:
- Alerts about Process monitor application failed to start multiple times.
- Searches reporting I/O errors.
- Services not starting.
- Configuration deployment changes due to critical disk space.
[tomcat.tomcat] /console/JSON-RPC/QRadar.scheduleDeployment QRadar.scheduleDeployment] com.q1labs.configservices.util.ConfigServicesUtil: [INFO] [-/--] Deployment is blocked due to critical disk space issue
Cause
By default, the QRadar disk sentry check runs every 60 seconds and looks for high disk usage across the partitions. When a partition goes beyond the critical warning threshold, an alert is triggered for administrators to investigate.
Diagnosing The Problem
Administrators can identify the largest directories and files by following the steps in Troubleshooting disk space usage problems. Once these large directories are identified, follow the instructions in Resolving the Problem to remove them.
Resolving The Problem
Use the following instructions to identify safe to remove files and regain space.
Depending on the directory reported during diagnosis, follow the suggestions provided. You might follow some or all of the suggestions, depending on your needs.
- Emails queued and not delivered in /var/spool/postfix.
- Test your email server reachability.
- Troubleshoot your email delivery and flush the queue to release space.
postsuper -d ALL postfix flush
- OS Kernel crash dumps in /var/crash.
- Remove the file filling up the partition.
rm -fv /var/crash/*
- Remove the file filling up the partition.
Result
The /var partition no longer has disk space constraints. If the partition reached the point of critical services stop, restart the services in the proper order and wait 5 mins with the following commands:
IMPORTANT: When QRadar core services restart, the QRadar UI, event processing, and database are not available to all users. Administrators with strict outage policies are advised to complete the next step during a scheduled maintenance window for their organization.
systemctl stop hostcontext
systemctl stop tomcat
systemctl restart hostservices
systemctl start tomcat
systemctl start hostcontext
If the partition does not decrease its usage or the services do not start properly, contact QRadar Support for assistance.
Related Information
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwsyAAA","label":"Admin Tasks"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
19 October 2022
UID
ibm16826597