IBM Support

QRadar: Delete files or directories to gain space in /tmp partition

Troubleshooting


Problem

When the /tmp partition in QRadar does not have enough space, it can affect the regular functioning of QRadar® SIEM. The purpose of this article is to help the administrator with the removal of files and directories when the /tmp partition has not enough available disk space.

Symptom

Lack of available space in the /tmp partition can cause the following issues:
 

Cause

By default, the QRadar disk sentry check runs every 60 seconds and looks for high disk usage across the partitions. When a partition goes beyond the critical warning threshold, an alert is triggered for administrators to investigate.

Diagnosing The Problem

Administrators can identify the largest directories and files by following the steps in Troubleshooting disk space usage problems. Once identified, compare them with the following list.
  
drwxrwxrwt. 20 root     root      36K Oct 12 17:19 .
dr-xr-xr-x. 22 root     root     4.0K Oct 12 17:19 ..
drwxrwxrwt   2 root     root       61 Oct 12 17:15 .com_ibm_tools_attach
drwxrwxrwt.  2 root     root        6 Jun 28  2019 .font-unix
drwxrwxrwt.  2 root     root        6 Jun 28  2019 .ICE-unix
drwxr-xr-x   2 root     root       66 Oct 12 15:56 loginmsg.lck
drwx------   3 root     root       17 Oct  6 21:23 namespace-dev-X3Ec3I
drwxr-----   3 root     root       19 Oct  6 21:07 .pki
drwxr-xr-x   2 root     root        6 Oct  6 20:53 postgres_96_to_11_upgrade
drwxr-xr-x   2 root     root       58 Oct  6 20:53 regex12446
drwx------   3 root     root       17 Oct  6 21:23 systemd-private-ee5e1a7b55514bd4aeddba74d853b317-chronyd.service-YMnrhR
drwx------   3 root     root       17 Oct  6 21:32 systemd-private-ee5e1a7b55514bd4aeddba74d853b317-container@10839994909734928660.service-PeSCpA
drwx------   3 root     root       17 Oct 11 16:57 systemd-private-ee5e1a7b55514bd4aeddba74d853b317-container@12286838862138095125.service-3gGtF0
drwx------   3 root     root       17 Oct  6 21:26 systemd-private-ee5e1a7b55514bd4aeddba74d853b317-httpd.service-cR42p0
drwx------   3 root     root       17 Oct  6 21:23 systemd-private-ee5e1a7b55514bd4aeddba74d853b317-jitterentropy.service-XZY1yN
drwx------   3 root     root       17 Oct 12 04:54 systemd-private-ee5e1a7b55514bd4aeddba74d853b317-postfix.service-oUpROE
drwxrwxrwt.  2 root     root        6 Jun 28  2019 .Test-unix
drwx------   2 root     root        6 Oct  6 20:54 tmp.FSVjnKvRNS
drwxrwxrwt.  2 root     root        6 Jun 28  2019 .X11-unix
drwxrwxrwt.  2 root     root        6 Jun 28  2019 .XIM-unix
The following example shows the /tmp/backup20220929/ directory using 2.8GB. This directory is not in the list, therefore, it's likely a directory that can be deleted.
[root@qradar ]# du -xch -d 1 /tmp | sort -h | tail -n 5
20K     /tmp/loginmsg.lck
24K     /tmp/regex12446
2.8G    /tmp
2.8G    /tmp/backup20220929
The following example shows the /tmp/backup.tar.gz file using 2.8GB. This file is likely a leftover that can be deleted. By default, no file exists in /tmp outside the directories in the previous list.
  
[root@qradar ]# find /tmp -type f -size +100M -exec ls -lah {} \;
-rw-r--r-- 1 root root 2.8G Oct 12 16:33 /tmp/backup.tar.gz
Once these large directories and files are identified, follow the instructions in Resolving the Problem to remove them.

Resolving The Problem

Use the following instructions to identify safe to remove files and regain space.
 
Depending on the directory reported during diagnosis, follow the suggestions provided. You might follow some or all of the suggestions, depending on your needs.
  • Move or remove user leftover files.

    To move the file: 
    mkdir -pv /store/IBM_Support/
mv -v /tmp/<file> /store/IBM_Support/
    Output Example: 
    ‘/tmp/backup.tar.gz’ -> ‘/store/IBM_Support/backup.tar.gz’
removed ‘/tmp/backup.tar.gz’

    To remove the file: 
    rm -fv /tmp/<file>
    Output Example: 
    removed ‘/tmp/backup.tar.gz’
  • Move or remove the conflicting directory.

    To move the directory: 
    mkdir -pv /store/IBM_Support/
mv /tmp/<directory> /store/IBM_Support/
    Output Example: 
    ‘/tmp/backup20220929/’ -> ‘/store/IBM_Support/backup20220929’
removed directory: ‘/tmp/backup20220929/’

    To remove the directory: 
    rm -rfv /tmp/<directory>
    Output Example 
    removed directory: ‘/tmp/backup20220929/’
  • Verify the partition usage decreased. 
    df -Th /tmp
    Output Example 
    Filesystem               Type  Size  Used Avail Use% Mounted on
/dev/mapper/rootrhel-tmp xfs   3.0G   45M  3.0G   2% /tmp
Result
The /tmp partition no longer has disk space constraints. If the partition reached the point of critical services stop, restart the services in the proper order and wait 5 mins with the following commands:
 
IMPORTANT: When the QRadar core service restart, the QRadar UI, event processing, and database are not available to all users. Administrators with strict outage policies are advised to complete the next step during a scheduled maintenance window for their organization.
  
systemctl stop hostcontext
systemctl stop tomcat
systemctl restart hostservices
systemctl start tomcat
systemctl start hostcontext
If the partition does not decrease its usage or the services do not start properly, contact QRadar Support for assistance.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwsyAAA","label":"Admin Tasks"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions"}]

Document Information

Modified date:
19 October 2022

UID

ibm16829035

Page Feedback