APAR status
Closed as program error.
Error description
The Tivoli Storage Manager server can hang after multiple client-side deduplication enabled sessions are canceled at or near the same time. This can occur with sessions canceled via the CANCEL SESSION administrator command, or via a client session timeout (i.e. IDLETIMEOUT). Customer/L2 Diagnostics: Dumping the active threads for the dsmserv process will show multiple client side deduplication sessions being canceled. One thread should include the pkAcquireMutexTracked() function, and the other should contain the AdmLockNode() function (example below is from the AIX procstack utility): Thread 1: pth_spinlock._global_lock_common() pth_mutex._mutex_lock() pkmon.pkAcquireMutexTracked() <----------------- smcancel.CancelDedupSession() <----------------- smcancel.CancelSessionNumDedup() smcancel.CancelSessionNum() smcancel.smCancelSession() admcmd.AdmCommandLocal() admcmd.admCommand() smadmin.SmAdminCommandThread() pkthread.StartThread() Thread 2: pth_cond._cond_wait_global() pth_cond._cond_wait() pth_cond.pthread_cond_wait() pkmon.pkWaitConditionTracked() tmlock.tmLockTracked() admutil.AdmLockNode() <----------------- admnode.admSetNodeSessCanceled() smcancel.CancelDedupSession() <----------------- smcancel.CancelSessionNumDedup() smcancel.CancelSessionNum() smcancel.smCancelSession() admcmd.AdmCommandLocal() admcmd.admCommand() smadmin.SmAdminCommandThread() pkthread.StartThread() Initial Impact: Medium Tivoli Storage Manager Versions Affected: All 6.2.4 & 6.3.1 or higher Tivoli Storage Manager servers. Additional Keywords: ZZTSM62 TSM HANG HUNG 76952 DEDUP CLIENT CANCEL SESS DEADLOCK MUTEX SPINLOCK LOCKNODE NODE
Local fix
Recycle the dsmserv process to clear the hang. Avoid canceling multiple client deduplication sessions at the same time until a fixing level can be applied.
Problem summary
**************************************************************** * USERS AFFECTED: All Tivoli Storage Manager server users of * * CANCEL SESSION command. * **************************************************************** * PROBLEM DESCRIPTION: See ERROR DESCRIPTION. * **************************************************************** * RECOMMENDATION: Apply fixing levels when available. * * This problem is currently projected * * to be fixed in level 6.2.6 and 6.3.4. * * Note that this is subject * * to change at the discretion of IBM. * **************************************************************** *
Problem conclusion
This problem was fixed. Affected platforms: AIX, HP-UX, Solaris, Linux, and Windows.
Temporary fix
Comments
APAR Information
APAR number
IC89747
Reported component name
TSM SERVER
Reported component ID
5698ISMSV
Reported release
62A
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2013-01-22
Closed date
2013-02-26
Last modified date
2013-02-26
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
TSM SERVER
Fixed component ID
5698ISMSV
Applicable component levels
R61A PSY
UP
R61H PSY
UP
R61L PSY
UP
R61S PSY
UP
R61W PSY
UP
R61Z PSY
UP
R62A PSY
UP
R62H PSY
UP
R62L PSY
UP
R62S PSY
UP
R62W PSY
UP
R62Z PSY
UP
R63A PSY
UP
R63H PSY
UP
R63L PSY
UP
R63S PSY
UP
R63W PSY
UP
R63Z PSY
UP
[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSGSG7","label":"Tivoli Storage Manager"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"62A","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]
Document Information
Modified date:
26 February 2013