Troubleshooting
Problem
File systems are added to the IBM Spectrum Protect server database by using the "EXTEND DBSPACE" command but the rebalance phase of the extend dbspace process does not redistribute the database across all file systems as expected.
Symptom
A manual execution of the database rebalance results in an error from Db2, for example:
xxxxxxx:/home/xxxx$db2 alter tablespace DEDUPIDXSPACE3 rebalance
SQL2094W The rebalance of table space "DEDUPIDXSPACE3" either did not add or
drop containers, or there was insufficient disk space to create all of the
containers. Reason code: "2". SQLSTATE=01690
SQL2094W The rebalance of table space "DEDUPIDXSPACE3" either did not add or
drop containers, or there was insufficient disk space to create all of the
containers. Reason code: "2". SQLSTATE=01690
In this case, the database originally had 8 DBspaces (file systems), a further 8 DBspaces were added to make a total of 16 DBspaces.
The "QUERY DBSPACE" command output shows:
The "QUERY DBSPACE" command output shows:
Protect: xxx>q dbspace
Location Total Space of Used Space on Free Space(MB)
File System File System
(MB) (MB)
------------------------------ --------------- --------------- ---------------
/tsm/xxx/db/db01 673,280.00 625,627.62 47,652.38
/tsm/xxx/db/db02 673,280.00 625,627.67 47,652.33
/tsm/xxx/db/db03 673,280.00 625,627.81 47,652.19
/tsm/xxx/db/db04 673,280.00 625,627.57 47,652.43
/tsm/xxx/db/db05 673,280.00 625,626.95 47,653.05
/tsm/xxx/db/db06 673,280.00 625,626.88 47,653.12
/tsm/xxx/db/db07 673,280.00 625,626.99 47,653.01
/tsm/xxx/db/db08 673,280.00 625,626.86 47,653.14
/tsm/xxx/db/db09 523,776.00 331,181.46 192,594.54 <
/tsm/xxx/db/db10 523,776.00 331,181.46 192,594.54 <
/tsm/xxx/db/db11 523,776.00 331,181.46 192,594.54 <
/tsm/xxx/db/db12 523,776.00 331,181.47 192,594.53 < New
/tsm/xxx/db/db13 523,776.00 331,181.46 192,594.54 < DBspaces
/tsm/xxx/db/db14 523,776.00 331,181.46 192,594.54 <
/tsm/xxx/db/db15 523,776.00 331,181.47 192,594.53 <
/tsm/xxx/db/db16 523,776.00 331,181.46 192,594.54 <
File System File System
(MB) (MB)
------------------------------ --------------- --------------- ---------------
/tsm/xxx/db/db01 673,280.00 625,627.62 47,652.38
/tsm/xxx/db/db02 673,280.00 625,627.67 47,652.33
/tsm/xxx/db/db03 673,280.00 625,627.81 47,652.19
/tsm/xxx/db/db04 673,280.00 625,627.57 47,652.43
/tsm/xxx/db/db05 673,280.00 625,626.95 47,653.05
/tsm/xxx/db/db06 673,280.00 625,626.88 47,653.12
/tsm/xxx/db/db07 673,280.00 625,626.99 47,653.01
/tsm/xxx/db/db08 673,280.00 625,626.86 47,653.14
/tsm/xxx/db/db09 523,776.00 331,181.46 192,594.54 <
/tsm/xxx/db/db10 523,776.00 331,181.46 192,594.54 <
/tsm/xxx/db/db11 523,776.00 331,181.46 192,594.54 <
/tsm/xxx/db/db12 523,776.00 331,181.47 192,594.53 < New
/tsm/xxx/db/db13 523,776.00 331,181.46 192,594.54 < DBspaces
/tsm/xxx/db/db14 523,776.00 331,181.46 192,594.54 <
/tsm/xxx/db/db15 523,776.00 331,181.47 192,594.53 <
/tsm/xxx/db/db16 523,776.00 331,181.46 192,594.54 <
Cause
There is insufficient space in the new file systems to create the database containers required for the rebalance.
Environment
All supported IBM Spectrum Protect server versions
Diagnosing The Problem
The problem tablespace, DEDUPIDXSPACE3, is ID 25.The db2pd -tablespaces output has:
Which shows the tablespace has a page size of 8KB.
The statistics for the tablespace are:
The tablespace DEDUPIDXSPACE3 has 300760832 pages at 8KB each which = 2,463,832,735,744 bytes, or around 2.3TB.
There was an expectation that each dbspace needs 154GB of free space (2.3TB/16) for a rebalance to 16 dbspaces (1 container per filesystem)
It would appear there is ample free space (~192GB) on the new DBspaces for the new containers.
The issue arises because Db2 creates the containers on the new DBspaces by using the capacity of the original containers, prior to the extend.
The 2.3TB tablespace had 8 containers before the extend, so the rebalance requires the new file system have free space equal to:
~2.3TB/8 = ~293GB
~2.3TB/8 = ~293GB
In this example, the ~520GB in the new DBspace file systems was expected to be sufficient to rebalance the database.
The problem resulted because of the order in which the tablespaces were rebalanced, tablespace ID 25 was rebalanced last.
The problem resulted because of the order in which the tablespaces were rebalanced, tablespace ID 25 was rebalanced last.
Resolving The Problem
Enlarge the new file systems to cater for the capacity required for the rebalance process.
When the database rebalance process is complete, excess space in the database can be released by using the "db2 ALTER TABLESPACE ?? REDUCE MAX" command.
The "Managing inventory capacity" section of the documentation has further information:
https://www.ibm.com/docs/en/spectrum-protect/8.1.13?topic=mds-managing-inventory-capacity
Tablespace utilization can be reviewed with the command Db2 command:
db2 connect to tsmdb1
db2 "select * from SYSIBMADM.TBSP_UTILIZATION "
db2 connect to tsmdb1
db2 "select * from SYSIBMADM.TBSP_UTILIZATION "
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB26","label":"Storage"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSER5J","label":"IBM Spectrum Protect Extended Edition"},"ARM Category":[{"code":"a8m3p000000hAZuAAM","label":"Server"}],"ARM Case Number":"TS007567985","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
25 February 2022
UID
ibm16554448