Troubleshooting
Problem
When trying to start PAW containers using ./scripts/paw.ps1 the following error occurs for one or several containers :
Cannot Start service <service-name>: Container is marked for removal and cannot be started
In addition to this error, when diplaying the list of containers using "./scripts/paw.ps1 ps" command, it shows some temporary containers, looking like this :
bf16f3c7faa2_couchdb
32784fd3b06d_share-app
78fd50cfc238_pa-gateway
Cause
Temporary containers can be generated in various situations, for example if a containers becomes unavailable for a too long period, of if running paw.ps1 command again while some containers were already running but some other were not or were missing. This can also be caused by timing issues (containers still not ready after a long time, while other containers are already trying to access them). This also can be caused by containers that were not properly stopped by last reboot.
Resolving The Problem
When this issue happens, we can get rid of these temporary containers and recreate correct containers in Powershell by running these commands :
cd <PAWpath>
./scripts/paw.ps1 ps [to list all containers and see their names]
./scripts/paw.ps1 stop [to stop all containers]
docker rm <name_of_temporary_containers_previously_listed>
./scripts/paw.ps1 [with no argument. So that to recreate missing containers and restarting everything]
./scripts/paw.ps1 ps [to list the containers again and verify everything is ok. All containers should be started, except bss-init that should stopped by itself after a little while]
cd <PAWpath>
./scripts/paw.ps1 ps [to list all containers and see their names]
./scripts/paw.ps1 stop [to stop all containers]
docker rm <name_of_temporary_containers_previously_listed>
./scripts/paw.ps1 [with no argument. So that to recreate missing containers and restarting everything]
./scripts/paw.ps1 ps [to list the containers again and verify everything is ok. All containers should be started, except bss-init that should stopped by itself after a little while]
Once that problem solved, we have to take actions to prevent this issue from happening again. The solution have two aspects :
1) Increase Docker wait time, so that to let containers stop smoothly : create a daemon.json in C:\ProgramData\Docker\config to increase the timeout (in seconds), for example :
{
"shutdown-timeout": 900
}
Once done, run "./scripts/paw.ps1 stop" in Powershell so that to prevent Docker from stopping the containers by itself, this is just as a precaution because the new parameter is not yet taken into account.
Now stop and restart Docker service so that to let docker use this new parameter. From here, next restarts should be ok.
{
"shutdown-timeout": 900
}
Once done, run "./scripts/paw.ps1 stop" in Powershell so that to prevent Docker from stopping the containers by itself, this is just as a precaution because the new parameter is not yet taken into account.
Now stop and restart Docker service so that to let docker use this new parameter. From here, next restarts should be ok.
In addition to this, go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control, find parameter "WaitToKillServiceTimeout" (which is in milliseconds) and increase it so that Windows does wait some more time before killing docker.
2) When a machine is restarting, it is already very busy even without docker running. So with docker starting all containers at the same time (more than 20), the machine gets even more busy, which can cause timing issues in PAW : once a container is started, it tries to access other containers, so it can wait some time for a response but not too much time ; if waiting gets too long, you may get exited containers or temporary containers getting created again. To avoid this, go to the properties of the "docker" or "docker engine" service, and change startup type from "Automatic" to "Automatic (Delayed start)".
If this is not sufficient (meaning : we can see some containers that are still in Exit state), then we may have to schedule a little Powershell script (*.ps1) few minutes after reboot (there is such an option in Windows Task Scheduler, like "x minutes after reboot") ; in this script, just execute this command :
cd <PAWpath>
./scripts/paw.ps1 start
This will restart all exited containers
cd <PAWpath>
./scripts/paw.ps1 start
This will restart all exited containers
Document Location
Worldwide
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSD29G","label":"IBM Planning Analytics"},"Component":"PAW;Planning Analytics Workspace;docker","Platform":[{"code":"PF033","label":"Windows"}],"Version":"Windows Server 2016","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
Was this topic helpful?
Document Information
Modified date:
08 October 2019
UID
ibm11079373