Direct links to fixes
7.1.7.400-TIV-TSMSRV-WIN
7.1.7.400-TIV-TSMSRV-SolarisSPARC
7.1.7.400-TIV-TSMSRV-Linuxx86_64
7.1.7.400-TIV-TSMSRV-Linuxs390x
7.1.7.400-TIV-TSMSRV-Linuxppc64
7.1.7.400-TIV-TSMSRV-HP-UX
7.1.7.400-TIV-TSMSRV-AIX
8.1.1.100-IBM-SPOC-WindowsX64
8.1.1.100-IBM-SPOC-Linuxx86_64
8.1.1.100-IBM-SPOC-Linuxs390x
8.1.1.100-IBM-SPOC-AIX
8.1.1.000-IBM-SPSRV-WindowsX64
8.1.1.000-IBM-SPSRV-Linuxs390x
8.1.1.000-IBM-SPSRV-AIX
8.1.1.000-IBM-SPSRV-Linuxx86_64
IBM Spectrum Protect Server V8.1 Fix Pack 1 (V8.1.1) Downloads
IBM Spectrum Protect Server V7.1 Fix Pack 8 (7.1.8.000) Downloads
IBM Spectrum Protect Server V7.1.7.X interim fix downloads
APAR status
Closed as program error.
Error description
Index for table SD_CHUNK_LOCATIONS grows very fast after online index reorganization fails e.g. due to a database deadlock, which can lead to the DB containers not having enough free space for the next online index reorganization. This will then fail with "DIA8312C Disk was full." For example: ACTLOG shows: ANR0317I Reorganization of indices for table SD_CHUNK_LOCATIONS started. ANR0162W Supplemental database diagnostic information: -911:40001:0 (32). ANR0318I Reorganization of indices for table SD_CHUNK_LOCATIONS ended with sqlCode -911. ANR0159E tbrsql.c(1450): Database deadlock detected on 379:12. db2diag.log shows: FUNCTION: DB2 UDB, relation data serv, sqlrreorg_indexes, probe:1000 MESSAGE : ADM9504W Index reorganization on table "TSMDB1 .SD_CHUNK_LOCATIONS" (ID "4") and table space "DEDUPTBLSPACE3" (ID "22") failed on this database partition with SQLCODE "-911" reason code "2". FUNCTION: DB2 UDB, catalog services, sqlrlCatalogScan::postUpdateCacheHandling, probe:10 RETCODE : ZRC=0x80100002=-2146435070=SQLP_LDED "Dead lock detected" DIA8002C A deadlock has occurred, rolling back transaction. The next online index reorganization can fail due to lack of space in the database containers. db2diag.log shows: FUNCTION: DB2 UDB, relation data serv, sqlrreorg_indexes, probe:400 MESSAGE : ADM9501W Index reorganization has started for table "TSMDB1.SD_CHUNK_LOCATIONS" (ID "4") and table space "DEDUPTBLSPACE3" (ID "22"). FUNCTION: DB2 UDB, relation data serv, sqlrreorg_index_obj, probe:550 MESSAGE : ADM9503W Reorganizing index IID "1" (OBJECTID "4") in table space "DEDUPIDXSPACE3" (ID "23") for table "TSMDB1.SD_CHUNK_LOCATIONS" (ID "4") in table space "DEDUPTBLSPACE3" (ID "22"). FUNCTION: DB2 UDB, buffer pool services, sqlbGrowNewStripeSet, probe:20 MESSAGE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full." DIA8312C Disk was full. DATA #1 : String, 108 bytes An error caused the allocation of a new stripe set of containers to an automatic storage tablespace to fail. FUNCTION: DB2 UDB, buffer pool services, sqlbAllocateExtent, probe:980 MESSAGE : ADM6044E The DMS table space "DEDUPIDXSPACE3" (ID "23") is full. If this is an autoresize or automatic storage DMS tablespace, the maximum table space size may have been reached or the existing containers or storage paths cannot grow any more. Additional space can be added to the table space by either adding new containers or extending existing ones using the ALTER TABLESPACE SQL statement. If this is an autoresize or automatic storage DMS table space, additional space can be added by adding containers to an autoresize table space or by adding new storage paths to the storage group it is using. FUNCTION: DB2 UDB, buffer pool services, sqlbObtainDataExtent, probe:800 MESSAGE : ZRC=0x85020021=-2063466463=SQLB_END_OF_CONTAINER "DMS Container space full" DATA #1 : Object descriptor, PD_TYPE_SQLB_OBJECT_DESC, 104 bytes Obj: {pool:23;obj:4;type:33} Parent={22;4} FUNCTION: DB2 UDB, relation data serv, sqlrreorg_indexes, probe:1000 MESSAGE : ADM9504W Index reorganization on table "TSMDB1 .SD_CHUNK_LOCATIONS" (ID "4") and table space "DEDUPTBLSPACE3" (ID "22") failed on this database partition with SQLCODE "-289" reason code "DEDUPIDXSPACE3". IBM Spectrum Protect Server Versions Affected: IBM Spectrum Protect Version 7.1.3 and above on all supported platforms Initial Impact: High Additional Keywords: TSM, inline deduplication, DB2, reorg, IBM Spectrum Protect
Local fix
Under normal circumstances SD tables should not require offline reorganization. However as a local fix for this issue the offline reorganization of the SD_CHUNK_LOCATION table will address the issue for the given online index reorg failure. Perform offline index reorg for table SD_CHUNK_LOCATIONS as described in Technote: "Resolving and preventing issues related to database growth and degraded performance in Tivoli Storage Manager V7.1.1.200 and later servers" http://www-01.ibm.com/support/docview.wss?uid=swg21683633
Problem summary
**************************************************************** * USERS AFFECTED: * * All IBM Spectrum Protect server users. * **************************************************************** * PROBLEM DESCRIPTION: * * See error description. * **************************************************************** * RECOMMENDATION: * * Apply fixing level when available. This problem is currently * * projected to be fixed in levels 7.1.8 and 8.1.1. Note that * * this is subject to change at the discretion of IBM. * ****************************************************************
Problem conclusion
This problem was fixed. Affected platforms for reported release: AIX, HP-UX, Solaris, Linux, and Windows. Platforms fixed: AIX, Solaris, Linux, and Windows.
Temporary fix
Comments
APAR Information
APAR number
IT18356
Reported component name
TSM SERVER
Reported component ID
5698ISMSV
Reported release
71W
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2016-12-15
Closed date
2017-02-10
Last modified date
2017-04-24
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
TSM SERVER
Fixed component ID
5698ISMSV
Applicable component levels
Document Information
Modified date:
24 April 2017