IBM Support

Guardium Collector Down /var full - Large Docker containers overlay

Troubleshooting


Problem

Error Msg : Warning: Can't connect to local MySQL server
The cli database login failed
Welcome cli - your last login was Wed Mar  1 01:30:03 2023
Checking system components usability...
MySQL is not ready
Automatic retry 10 times with 30 seconds wait between tries.
Press Ctrl-C to go into CLI recovery mode anytime.
Retry 1 of 10...
MySQL is not ready

Symptom

mysql not running and /var data partition at 100%
image-20230809093520-1
image-20230809093558-2
image-20230809093737-3
There were no issues with Guardium occupying excessive space.

Resolving The Problem

There were no issues with Guardium occupying excessive space.
We ran the following command to recover some space (98%) to allow mysql to start:
tune2fs -m 2 /dev/sda3
To reclaim another 17GB (95%)....We dropped table AGG_EXPORT_TMP_DB;
We discovered an large amount of files slated for deletion outside of Guardium in /var.
root> sudo lsof +L1
image-20230809094755-7
After restarting the appliance, the reboot reclaimed this space.
======================2023-03-01 11:58:22 ... Output of df command:======================
Filesystem     1024-blocks      Used Available Capacity Mounted on
/dev/sda3        576566832 146162572 418670060      26% /var

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSMPHH","label":"IBM Security Guardium"},"ARM Category":[{"code":"a8m0z000000Gp0VAAS","label":"SYSTEM REQUIREMENTS"}],"ARM Case Number":"TS012304975","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
09 August 2023

UID

ibm17024943

Page Feedback