IBM Support

Intermittently, regularly, all containers are in 'Exit' state and they have to be manually restarted

Troubleshooting


Problem

Intermittently, regularly, all Planning Analytics Workspace containers seem to stop by themselves.
The "/scripts/paw.ps1 ps" (Windows) command or "/scripts/paw.sh ps" (Linux) command shows all containers in 'Exit' state.
None of the containers is running ('Up').
Nevertheless, they can be manually restarted with no problem by running one of these commands:
/scripts/paw.ps1 start (Windows)
/scripts/paw.sh start (Linux)
or
/scripts/paw.ps1 (Windows)
/scripts/paw.sh (Linux)
Which means the containers don't have any issue.
So, what can be the cause of this behaviour?

Cause

Although the easy solution is to manually restart the containers, this behaviour must be taken seriously.
Indeed, it indicates that the backup is failing.
You probably scheduled the backup.ps1 (Windows) or backup.sh (Linux), but this backup is failing for some various possible reasons.
ATTENTION:
That means all your recent backups are incomplete, and would harm your application if you try to restore one of them.
DON'T RESTORE ANY OF THESE BACKUPS!

Resolving The Problem

Step 1:
In order to determine what the reason is, you need to review the backup_YYYY_MM_DD_hh_mm_ss.log files in <PAW>/log/backup directory (default directory)
If you find a clear error message, then search for this message in the IBM knowledge base, or contact IBM Support.
Step 2:
Once the cause of the issue is found, and corrected, run the backup script again:
/scripts/backup.ps1 (Windows)
/scripts/backup.sh (Linux)
Step 3:
Verify the last backup log file. A successful backup log ends with something like:
Backup complete in xxx.xxxxxxx seconds
Step 4:
Verify that all databases are correctly backed up. Here is the list of files that we should see in <PAW>/backup/backup_YYYY_MM_DD_hh_mm_ss/:
On Windows:
 Name        Length (bytes)
 ----        --------------
 bss.zip             379008    
 couchdb.zip         104727    
 mongo.zip         32386846    
 mongo_db.zip     111347692    
 mysql.zip             9660    
 redis.zip             1219   
On Linux:
 Name        Length (bytes)
 ----        --------------
 bss.tgz             287127
 couchdb.tgz           1407
 mongo.tgz         29889801
 mysql.tgz             4011
 redis.tgz              871
Step 5:
Delete all previous backups that are incorrect:
-If there are missing files
-If all files are present but the size of some of them is abnormally small compared to a correct backup

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB10","label":"Data and AI"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSCTEW","label":"IBM Planning Analytics Local"},"ARM Category":[{"code":"a8m50000000KzIzAAK","label":"Planning Analytics-\u003EPlanning Analytics Workspace"},{"code":"a8m50000000KzKvAAK","label":"Planning Analytics-\u003EPlanning Analytics Workspace-\u003EInstall Configure Upgrade Backup"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"},{"code":"PF033","label":"Windows"}],"Version":"All Versions"}]

Document Information

Modified date:
07 April 2023

UID

ibm16982083