A fix is available
APAR status
Closed as program error.
Error description
Frequent messages "ANR0538I A resource waiter has been aborted." may be reported in the actlog during client backups. This is especially the case during database intensive processes like EXPIRATION which do hold required locks for a longer time finally causing client backups to terminate after the resourcetimeout has been reached. It is not a best practice to let a client backup run concurrently with the expiration process but that may happen (as not possible otherwise) for a heavily used Tivoli Storage Manager server. A server instrumentation trace shows very slow "DB2 MFtch Exec" processing for the expiration thread. Analysis has shown the following select as being responsible for this high value : "select BFID, OFFSET, LENGTH, OWNER, DIGEST_TYPE,DIGEST_VALUE,FLAGS,UNCOMP_LENGTH from BF_AGGREGATED_BITFILES where SRVID=? and SUPERBFID=? and OWNER=?" A special runstat needs to be done to allow DB2 to use a better access plan for this specific table: BF_AGGREGATED_BITFILES Tivoli Storage Manager Versions Affected: All platforms 6.1 , 6.2 , 6.3 , 7.1 Customer/L2 Diagnostics : - Servermon script instrumentation trace show the following statistics for the expiration thread : Thread 691414 ExpirationProcessThread parent=691411 00:16:44.567-->00:41:08.037 Operation Count Tottime Avgtime Mintime Maxtime InstTput Total KB --------------------------------------------------------------- DB2 Fetch Exec 5 0.082 0.017 0.000 0.042 DB2 MFtch Exec 1 1391.037 1391.037 1391.037 1391.037 DB2 Inser Exec 121537 70.183 0.001 0.000 2.580 DB2 Updat Exec 5 0.082 0.017 0.000 0.042 DB2 Fetch 5 0.000 0.000 0.000 0.000 DB2 Commit 1 0.161 0.162 0.161 0.161 DB2 Reg Prep 2 0.006 0.003 0.000 0.005 DB2 Reg Exec 3 1.268 0.423 0.000 0.714 DB2 Reg Fetch 3 0.000 0.000 0.000 0.000 Tm Lock Wait 1 0.003 0.003 0.003 0.003 Unknown 0.644 --------------------------------------------------------------- Total 1463.470 Please note the considerably high value of time being spent in "DB2 MFtch Exec". Initial Impact: Medium Additional Keywords: BFBF BFBE TSM
Local fix
To reduce the frequency of the ANR0538I messages to appear (may be not eliminate all of them as they can still appear due to other reasons) caused by the current runstats command, please do: - First be sure that the table BF_AGGREGATED_BITFILES and it's indexes does not need reorg. This technote explains how to check current reorg status for the tables and indexes using "db2 reorgchk" command. And it also explains how to do offline reorg if needed. "Database size, database reorganization, and performance considerations for Tivoli Storage Manager V6 and V7 servers" http://www.ibm.com/support/docview.wss?uid=swg21452146 - Run the following runstats command (the command may run when server is up and running) : db2 "runstats on table TSMDB1.BF_AGGREGATED_BITFILES ON ALL COLUMNS AND COLUMNS ((OWNER,SUPERBFID)) with distribution and sampled detailed indexes all" The server should not be halted until the runstats completes.
Problem summary
**************************************************************** * USERS AFFECTED: * * All Tivoli Storage Manager server users. * **************************************************************** * PROBLEM DESCRIPTION: * * See ERROR DESCRIPTION. * **************************************************************** * RECOMMENDATION: * * Apply fixing level when available. * * This problem is currently projected to be fixed in levels * * 6.3.6 and 7.1.1. * * Note that this is subject to change at the discretion of * * IBM. * ****************************************************************
Problem conclusion
This problem was fixed. Affected platforms: AIX, HP-UX, Solaris, Linux, and Windows.
Temporary fix
Comments
APAR Information
APAR number
IC99117
Reported component name
TSM SERVER
Reported component ID
5698ISMSV
Reported release
63L
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2014-02-03
Closed date
2014-04-16
Last modified date
2014-04-16
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
TSM SERVER
Fixed component ID
5698ISMSV
Applicable component levels
R63A PSY
UP
R63H PSY
UP
R63L PSY
UP
R63S PSY
UP
R63W PSY
UP
R71A PSY
UP
R71H PSY
UP
R71L PSY
UP
R71S PSY
UP
R71W PSY
UP
[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSGSG7","label":"Tivoli Storage Manager"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"63L","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]
Document Information
Modified date:
16 April 2014