Troubleshooting
Problem
After a restart of the TM1 server, TM1RunTI executes processes in serial on first run but executes them in parallel on all subsequent runs.
Symptom
Chore that should take minutes is now taking hours as TI processes called by TM1RunTI are being run in serial.
Cause
During investigation, it was noted that lock contention was causing all but one TI to be blocked and put in a WAIT state. When the first TI finished, the next TI's status would flip to run and complete. This process repeated until all TIs had completed.
Diagnosing The Problem
1) After a restart, run Tm1top against the server in exhibiting the issue with debug lock contention set in tm1_logs.properties
2) Execute the TM1RunTI process that should call several TIs in parallel
3) Note in the Tm1top logs that all threads are in WAIT except one which is running
Resolving The Problem
Using the information in the tm1top log, the lock contention can be narrowed down.
For example,
8888 Admin TM1RunTI Wait:IXC ProcessExecuteEx Process xxxxxx:Prolog (147), WaitingOn (18516) 1
This indicates the issue is in the Prolog tab on line 147 of Process xxxxxx.
Looking at the code:
vDim = 'xxxxxxx';
IF( SUBSETEXISTS( vDim, vZeroOutSubset ) = 0 );
SUBSETCREATE( vDim, vZeroOutSubset); <---- line 147
ELSE;
SUBSETDELETEALLELEMENTS( vDim, vZeroOutSubset );
ENDIF;
The lock contention is happening because TM1 cannot create a subset.
In this case, because subsets are being created in Prolog and destroyed in Epilog, we can create them as temporary subsets:
vDim = 'xxxxxxx';
IF( SUBSETEXISTS( vDim, vZeroOutSubset ) = 0 );
SUBSETCREATE( vDim, vZeroOutSubset,1 ); <---- line 147
ELSE;
SUBSETDELETEALLELEMENTS( vDim, vZeroOutSubset );
ENDIF;
IF( SUBSETEXISTS( vDim, vZeroOutSubset ) = 0 );
SUBSETCREATE( vDim, vZeroOutSubset,1 ); <---- line 147
ELSE;
SUBSETDELETEALLELEMENTS( vDim, vZeroOutSubset );
ENDIF;
This resolved the lock contention and now TM1RunTI runs in parallel on all executions.
Related Information
Document Location
Worldwide
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSD29G","label":"IBM Planning Analytics"},"Component":"","Platform":[{"code":"PF033","label":"Windows"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
Was this topic helpful?
Document Information
Modified date:
08 February 2019
UID
ibm10870966