Direct links to fixes
8.1.1.200-IBM-SPSRV-WindowsX64
8.1.1.200-IBM-SPSRV-Linuxx86_64
8.1.1.200-IBM-SPSRV-Linuxs390x
8.1.1.200-IBM-SPSRV-Linuxppc64le
8.1.1.200-IBM-SPSRV-AIX
7.1.9.000-TIV-TSMCMS-Windows64
7.1.9.000-TIV-TSMCMS-Windows32
7.1.9.000-TIV-TSMCMS-Linuxx64
7.1.9.000-TIV-TSMOC-LinuxS390
7.1.9.000-TIV-TSMOC-LinuxPPC64
7.1.9.000-TIV-TSMOC-Windows
7.1.9.000-TIV-TSMOC-Linuxx64
7.1.9.000-TIV-TSMOC-AIX
7.1.9.000-TIV-TSMSRV-WIN
7.1.9.000-TIV-TSMSRV-SolarisSPARC
7.1.9.000-TIV-TSMSRV-Linuxx86_64
7.1.9.000-TIV-TSMSRV-Linuxs390x
7.1.9.000-TIV-TSMSRV-Linuxppc64
7.1.9.000-TIV-TSMSRV-HP-UX
7.1.9.000-TIV-TSMSRV-AIX
8.1.5.000-IBM-SPCMS-WindowsX64
8.1.5.000-IBM-SPCMS-WindowsI32
8.1.5.000-IBM-SPCMS-Linuxx86_64
8.1.5.000-IBM-SPOC-WindowsX64
8.1.5.000-IBM-SPOC-Linuxx86_64
8.1.5.000-IBM-SPOC-Linuxs390x
8.1.5.000-IBM-SPOC-LinuxPPC64le
8.1.5.000-IBM-SPOC-AIX
8.1.5.000-IBM-SPSRV-WindowsX64
8.1.5.000-IBM-SPSRV-Linuxx86_64
8.1.5.000-IBM-SPSRV-Linuxs390x
8.1.5.000-IBM-SPSRV-Linuxppc64le
8.1.5.000-IBM-SPSRV-AIX
IBM Spectrum Protect Server V8.1 Fix Pack 5 (V8.1.5) Downloads
IBM Spectrum Protect Server V7.1 Fix Pack 9 (7.1.9.000) Downloads
IBM Spectrum Protect Server V8.1.1.X interim fix downloads
APAR status
Closed as program error.
Error description
There is a potential for a server hang if the QUERY PROCESS command is issued while a local PROTECT STGPOOL operation is in the process of being cancelled. The problem can only occur if the PROTECT STGPOOL operation is multi-threaded; the output from the SHOW THREADS command will show contention between the QUERY PROCESS command and 2 or more PROTECT STGPOOL threads: Thread 571785, Parent 571782: SmAdminCommandThread, Storage 4043, AllocCnt 51 HighWaterAmt 74031 tid=c289, ptid=13c86, det=0, zomb=0, join=1, result=0, sess=0, procToken=0, sessToken=346544 Stack trace: 0x090000000051483c _global_lock_common 0x0900000000522108 _mutex_lock 0x0000000100007cd8 pkAcquireMutexTracked 0x0000000100e76ea8 SdReplQueryProcess 0x00000001000ca5ec procQueryProcess 0x00000001010ca8c4 AdmQueryProcess 0x00000001003b073c AdmCommandLocal 0x00000001003ae0f8 admCommand 0x0000000100b88cd4 SmAdminCommandThread 0x000000010000e114 StartThread Holding mutex PROCV->mutex (0x110f5a1d8), acquired at process.c(1152) Holding mutex descP->tableMutex (0x12f23ddd8), acquired at output.c(1923) Acquiring mutex ctlP->mutex (0x13ed0e158) at sdrepl.c(7334) Thread context: COMMAND: QUERY PROCESS COMMMETHOD: Tcp/Ip THREAD_TYPE: SESSION SESSION_TYPE: ADMIN ADMIN_NAME: ADMINISTRATOR Thread 571761, Parent 571753: SdReplicateBatch, Storage 0, AllocCnt 0 HighWaterAmt 0 tid=18971, ptid=18669, det=0, zomb=0, join=0, result=0, sess=0, procToken=203, sessToken=346530 Stack trace: 0x090000000051483c _global_lock_common 0x0900000000522108 _mutex_lock 0x0000000100007cd8 pkAcquireMutexTracked 0x0000000100e878ac SdAcquireChildCtlDesc 0x0000000100e8d644 LocalProtectBatch 0x0000000100e83e1c SdReplicateBatch 0x0000000100729e40 PcConsumerThread 0x000000010000e114 StartThread Holding mutex ctlP->mutex (0x13ed0e158), acquired at sdrepl.c(5267) Acquiring mutex ctlP->mutex (0x12e6f64d8) at sdrepl.c(5294) Thread context: COMMAND: PROTECT STGPOOL SCHEDULE_TYPE: ADMIN SCHEDULE_NAME: PROTECT_LOCAL_AM SCHEDULED: YES Thread 571764, Parent 571751: SdProtectManageChildCancel, Storage 0, AllocCnt 0 HighWaterAmt 0 tid=18574, ptid=18367, det=1, zomb=0, join=0, result=0, sess=0, procToken=202, sessToken=346530 Stack trace: 0x0900000000537260 _cond_wait_global 0x0900000000537df8 _cond_wait 0x0900000000538ae0 pthread_cond_wait 0x00000001000093b4 pkWaitConditionTracked 0x00000001000c9174 procEndProcessEx 0x0000000100e88988 SdProtectManageChildCancel 0x000000010000e114 StartThread Holding mutex ctlP->mutex (0x12e6f64d8), acquired at sdrepl.c(5882) Awaiting cond procP->cancelCond (0x11902a140), using mutex PROCV->mutex (0x110f5a1d8), at process.c(698) Thread context: COMMAND: PROTECT STGPOOL SCHEDULE_TYPE: ADMIN SCHEDULE_NAME: PROTECT_LOCAL_AM SCHEDULED: YES The output above shows a deadlock condition in which the SmAdminCommandThread (QUERY PROCESS), SdReplicateBatch and SdProtectManageChildCancel threads are all holding mutexes required by one of the other threads while also attempting to acquire mutexes held by one of the other threads. The hang condition will also affect any other session or process threads attempting to acquire a mutex held by one of these threads. A restart of the Spectrum Protect server will be required to recover from this condition. Spectrum Protect Server Versions Affected: All 7.1.7.0 and higher versions of Spectrum Protect server code Initial Impact: Medium
Local fix
Avoid or minimize issuing the QUERY PROCESS command while a local PROTECT STGPOOL operation is in the process of being cancelled. A restart of the Spectrum Protect server instance will be required to recover from this condition.
Problem summary
**************************************************************** * USERS AFFECTED: * * All IBM Spectrum Protect server users using PROTECT STGPOOL * * with TYPE=LOCAL * **************************************************************** * PROBLEM DESCRIPTION: * * See ERROR DESCRIPTION. * **************************************************************** * RECOMMENDATION: * * Apply fixing level when available. This problem is currently * * projected to be fixed in levels 7.1.9 and 8.1.5. Note that * * this is subject to change at the discretion of IBM. * ****************************************************************
Problem conclusion
This problem was fixed. Platforms fixed: AIX, Linux, Solaris, and Windows.
Temporary fix
Comments
APAR Information
APAR number
IT23795
Reported component name
TSM SERVER
Reported component ID
5698ISMSV
Reported release
81A
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2018-01-22
Closed date
2018-02-19
Last modified date
2018-02-19
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
TSM SERVER
Fixed component ID
5698ISMSV
Applicable component levels
R71A PSY
UP
R71L PSY
UP
R71S PSY
UP
R71W PSY
UP
R81A PSY
UP
R81L PSY
UP
R81W PSY
UP
Document Information
Modified date:
28 September 2021