Fixes are available
Operational Decision Manager V8.9.2.2: Interim Fix 7
IBM Operational Decision Manager V8.10.3 Download Document
IBM Operational Decision Manager V8.10.3 Mod Pack
Operational Decision Manager for z/OS 8.10.3 Mod Pack
Operational Decision Manager V8.10.2.0: Interim Fix 4
Operational Decision Manager V8.10.2.0: Interim Fix 5
Operational Decision Manager V8.9.2.2: Interim Fix 9
Operational Decision Manager V8.9.2.1: Interim Fix 20
Operational Decision Manager V8.10.2.0: Interim Fix 6
Operational Decision Manager V8.9.2.2: Interim Fix 10
Operational Decision Manager V8.9.2.2: Interim Fix 11
Operational Decision Manager V8.10.2.0: Interim Fix 8
Operational Decision Manager V8.10.2.0: Interim Fix 9
IBM Operational Decision Manager V8.9.2.3 Fix Pack
IBM Operational Decision Manager for z/OS 8.9.2.3 Fix Pack
Operational Decision Manager V8.9.2.1: Interim Fix 22
Operational Decision Manager V8.9.2.1: Interim Fix 23
Operational Decision Manager V8.10.2.0: Interim Fix 10
Operational Decision Manager V8.10.2.0: Interim Fix 11
Operational Decision Manager V8.9.2.1: Interim Fix 24
Operational Decision Manager for z/OS 8.10.2.2 Fix Pack
IBM Operational Decision Manager V8.10.2.2 Fix Pack
Operational Decision Manager V8.10.2.0: Interim Fix 13
Operational Decision Manager V8.9.2.1: Interim Fix 26
Operational Decision Manager V8.9.2.1: Interim Fix 27
Operational Decision Manager V8.9.2.2: Interim Fix 13
Operational Decision Manager V8.9.2.1: Interim Fix 28
Operational Decision Manager V8.9.2.1: Interim Fix 30
Operational Decision Manager V8.9.2.1: Interim Fix 31
Operational Decision Manager V8.9.2.2: Interim Fix 14
Operational Decision Manager V8.9.2.1: Interim Fix 34
Operational Decision Manager V8.9.2.2: Interim Fix 15 for Decision Center
IBM Operational Decision Manager V8.10.4 Mod Pack
APAR status
Closed as program error.
Error description
2 instances of Decision Center working on the same database can lead to a deadlock, where it is not possible to edit rules anymore all operation appear blocked. On one server a thread is doing this blocking the global cache for all editors and waiting for DB2 to write on the other server similarly a thread is taking the globalcache lock and stuck trying to write.
Local fix
Problem summary
**************************************************************** * USERS AFFECTED: * * Users connecting to 2 instances of Decision Center working * * on the same database. * **************************************************************** * PROBLEM DESCRIPTION: * * 2 instances of Decision Center working on the same database * * can lead to a deadlock, where it is not possible to edit * * rules anymore all operation appear blocked. * * * * On one server a thread is doing this blocking the global * * cache for all editors and waiting for DB2 to write * * * * "Default Executor-thread-660339" * * J9VMThread:0x00000000018A5900, * * omrthread_t:0x00007F3B6C0BF320, * * java/lang/Thread:0x000000066050CCD8, state:R, prio=5 * * (java/lang/Thread getId:0x1F0DCB, isDaemon:true) * * (native thread ID:0x12165, native priority:0x5, native * * policy:UNKNOWN, vmstate:R, vm thread flags:0x000000a1) * * (native stack address range from:0x00007F3C1140D000, * * to:0x00007F3C1144E000, size:0x41000) * * CPU usage total: 2.910134575 secs, current * * category="Application" * * Heap bytes allocated since last GC cycle=0 (0x0) * * Java callstack: * * at java/net/SocketInputStream.socketRead0(Native Method) * * at * * java/net/SocketInputStream.socketRead(SocketInputStream.java * * :127(Compiled Code)) * * at * * java/net/SocketInputStream.read(SocketInputStream.java:182(C * * ompiled Code)) * * at * * java/net/SocketInputStream.read(SocketInputStream.java:152(C * * ompiled Code)) * * at com/ibm/jsse2/b.a(b.java:262(Compiled Code)) * * at com/ibm/jsse2/b.a(b.java:33(Compiled Code)) * * at com/ibm/jsse2/av.a(av.java:579(Compiled Code)) * * (entered lock: java/lang/Object@0x00000006736193E8, entry * * count: 1) * * at com/ibm/jsse2/av.a(av.java:637(Compiled Code)) * * at com/ibm/jsse2/f.read(f.java:21(Compiled Code)) * * (entered lock: com/ibm/jsse2/f@0x000000067360E018, entry * * count: 1) * * .... * * at com/ibm/db2/jcc/am/tp.executeUpdate(tp.java:791(Compiled * * Code)) * * (entered lock: com/ibm/db2/jcc/t4/b@0x00000006735B7100, * * entry count: 1) * * at * * com/ibm/ws/rsadapter/jdbc/WSJdbcPreparedStatement.executeUpd * * ate(WSJdbcPreparedStatement.java:521(Compiled Code)) * * at sun/reflect/GeneratedMethodAccessor160.invoke(Bytecode * * PC:42(Compiled Code)) * * at * * sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMe * * thodAccessorImpl.java:55(Compiled Code)) * * at java/lang/reflect/Method.invoke(Method.java:508(Compiled * * Code)) * * at * * ilog/rules/teamserver/transaction/logger/IlrStatementInvocat * * ionHandler.executeStatement(IlrStatementInvocationHandler.ja * * va:67(Compiled Code)) * * at * * ilog/rules/teamserver/transaction/logger/IlrPreparedStatemen * * tInvocationHandler.invoke(IlrPreparedStatementInvocationHand * * ler.java:38(Compiled Code)) * * at com/sun/proxy/$Proxy309.executeUpdate(Bytecode * * PC:11(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/dao/IlrElementDAOJDBC.upda * * teElement(IlrElementDAOJDBC.java:8284(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/dao/IlrElementDAOJDBC.upda * * teElement(IlrElementDAOJDBC.java:8095(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.updat * * eNonVersionableElementDetails(IlrSessionFacadeImpl.java:2950 * * (Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.commi * * tElementDetails(IlrSessionFacadeImpl.java:715(Compiled * * Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.commi * * t(IlrSessionFacadeImpl.java:1425(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.commi * * t(IlrSessionFacadeImpl.java:1039(Compiled Code)) * * at sun/reflect/GeneratedMethodAccessor170.invoke(Bytecode * * PC:42(Compiled Code)) * * at * * sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMe * * thodAccessorImpl.java:55(Compiled Code)) * * at java/lang/reflect/Method.invoke(Method.java:508(Compiled * * Code)) * * at * * ilog/rules/teamserver/transaction/IlrTransactionalInvocation * * Handler.invoke(IlrTransactionalInvocationHandler.java:107(Co * * mpiled Code)) * * at com/sun/proxy/$Proxy302.commit(Bytecode PC:18(Compiled * * Code)) * * at * * ilog/rules/teamserver/model/impl/IlrAbstractSession.commit(I * * lrAbstractSession.java:1259(Compiled Code)) * * at * * ilog/rules/teamserver/model/impl/IlrAbstractCachingSession.c * * ommit(IlrAbstractCachingSession.java:150(Compiled Code)) * * at * * ilog/rules/teamserver/model/IlrSettings.saveSetting(IlrSetti * * ngs.java:407(Compiled Code)) * * at * * ilog/rules/teamserver/model/IlrSettings.saveGlobalSetting(Il * * rSettings.java:373) * * at * * ilog/rules/teamserver/model/IlrGlobalCache.variablerOrParame * * terChanged(IlrGlobalCache.java:186) * * (entered lock: * * ilog/rules/teamserver/model/IlrGlobalCache$Lock@0x0000000650 * * 71CAB0, entry count: 1) * * * * * * on the other server similarly a thread is taking the * * globalcache lock and stuck trying to write. * * * * "Default Executor-thread-382926" * * J9VMThread:0x0000000001E8FA00, * * omrthread_t:0x00007F826400B140, * * java/lang/Thread:0x00000006AF549E40, state:R, prio=5 * * (java/lang/Thread getId:0x1BFAC6, isDaemon:true) * * (native thread ID:0xDCE, native priority:0x5, native * * policy:UNKNOWN, vmstate:R, vm thread flags:0x000000a1) * * (native stack address range from:0x00007F829F3A1000, * * to:0x00007F829F3E2000, size:0x41000) * * CPU usage total: 2.300567221 secs, current * * category="Application" * * Heap bytes allocated since last GC cycle=0 (0x0) * * Java callstack: * * at java/net/SocketInputStream.socketRead0(Native Method) * * at * * java/net/SocketInputStream.socketRead(SocketInputStream.java * * :127(Compiled Code)) * * at * * java/net/SocketInputStream.read(SocketInputStream.java:182(C * * ompiled Code)) * * at * * java/net/SocketInputStream.read(SocketInputStream.java:152(C * * ompiled Code)) * * at com/ibm/jsse2/b.a(b.java:262(Compiled Code)) * * at com/ibm/jsse2/b.a(b.java:33(Compiled Code)) * * at com/ibm/jsse2/av.a(av.java:579(Compiled Code)) * * (entered lock: java/lang/Object@0x000000067B770EB8, entry * * count: 1) * * at com/ibm/jsse2/av.a(av.java:637(Compiled Code)) * * at com/ibm/jsse2/f.read(f.java:21(Compiled Code)) * * (entered lock: com/ibm/jsse2/f@0x000000067B770F48, entry * * count: 1) * * .... * * at com/ibm/db2/jcc/t4/j.mc(j.java:194(Compiled Code)) * * at com/ibm/db2/jcc/am/tp.executeUpdate(tp.java:791(Compiled * * Code)) * * (entered lock: com/ibm/db2/jcc/t4/b@0x00000006AFB168D8, * * entry count: 1) * * at * * com/ibm/ws/rsadapter/jdbc/WSJdbcPreparedStatement.executeUpd * * ate(WSJdbcPreparedStatement.java:521(Compiled Code)) * * at sun/reflect/GeneratedMethodAccessor167.invoke(Bytecode * * PC:42(Compiled Code)) * * at * * sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMe * * thodAccessorImpl.java:55(Compiled Code)) * * at java/lang/reflect/Method.invoke(Method.java:508(Compiled * * Code)) * * at * * ilog/rules/teamserver/transaction/logger/IlrStatementInvocat * * ionHandler.executeStatement(IlrStatementInvocationHandler.ja * * va:67(Compiled Code)) * * at * * ilog/rules/teamserver/transaction/logger/IlrPreparedStatemen * * tInvocationHandler.invoke(IlrPreparedStatementInvocationHand * * ler.java:38(Compiled Code)) * * at com/sun/proxy/$Proxy309.executeUpdate(Bytecode * * PC:11(Compiled Code)) * * at sun/reflect/GeneratedMethodAccessor167.invoke(Bytecode * * PC:42(Compiled Code)) * * at * * sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMe * * thodAccessorImpl.java:55(Compiled Code)) * * at java/lang/reflect/Method.invoke(Method.java:508(Compiled * * Code)) * * at * * ilog/rules/teamserver/progress/IlrSQLProgressProxy$Statement * * InvocationHandler.invoke(IlrSQLProgressProxy.java:76(Compile * * d Code)) * * at com/sun/proxy/$Proxy309.executeUpdate(Bytecode * * PC:11(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/dao/IlrElementDAOJDBC.upda * * teElement(IlrElementDAOJDBC.java:8284(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/dao/IlrElementDAOJDBC.upda * * teElement(IlrElementDAOJDBC.java:8095(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.updat * * eNonVersionableElementDetails(IlrSessionFacadeImpl.java:2950 * * (Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.commi * * tElementDetails(IlrSessionFacadeImpl.java:715(Compiled * * Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.commi * * t(IlrSessionFacadeImpl.java:1425(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.commi * * t(IlrSessionFacadeImpl.java:1039(Compiled Code)) * * at sun/reflect/GeneratedMethodAccessor170.invoke(Bytecode * * PC:42(Compiled Code)) * * at * * sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMe * * thodAccessorImpl.java:55(Compiled Code)) * * at java/lang/reflect/Method.invoke(Method.java:508(Compiled * * Code)) * * at * * ilog/rules/teamserver/transaction/IlrTransactionalInvocation * * Handler.invoke(IlrTransactionalInvocationHandler.java:107(Co * * mpiled Code)) * * at com/sun/proxy/$Proxy300.commit(Bytecode PC:18(Compiled * * Code)) * * at * * ilog/rules/teamserver/model/impl/IlrAbstractSession.commit(I * * lrAbstractSession.java:1259(Compiled Code)) * * at * * ilog/rules/teamserver/model/impl/IlrAbstractCachingSession.c * * ommit(IlrAbstractCachingSession.java:150(Compiled Code)) * * at * * ilog/rules/teamserver/model/IlrSettings.saveSetting(IlrSetti * * ngs.java:407(Compiled Code)) * * at * * ilog/rules/teamserver/model/IlrSettings.saveGlobalSetting(Il * * rSettings.java:373) * * at * * ilog/rules/teamserver/model/IlrGlobalCache.variablerOrParame * * terChanged(IlrGlobalCache.java:186) * * (entered lock: * * ilog/rules/teamserver/model/IlrGlobalCache$Lock@0x000000064F * * 041030, entry count: 1) * * at * * ilog/rules/teamserver/model/IlrDefaultSessionController.vari * * ablesChanged(IlrDefaultSessionController.java:1327(Compiled * * Code)) * * at * * ilog/rules/teamserver/model/IlrDefaultSessionController.acce * * ss$300(IlrDefaultSessionController.java:114(Compiled Code)) * * at * * ilog/rules/teamserver/model/IlrDefaultSessionController$2.el * * ementCommitted(IlrDefaultSessionController.java:155(Compiled * * Code)) * * at * * ilog/rules/teamserver/model/IlrDefaultSessionController.elem * * entCommitted(IlrDefaultSessionController.java:2169(Compiled * * Code)) * * at * * ilog/rules/teamserver/model/IlrDefaultSessionController.elem * * entCommitted(IlrDefaultSessionController.java:2157(Compiled * * Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.commi * * t(IlrSessionFacadeImpl.java:1606(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.commi * * t(IlrSessionFacadeImpl.java:1039(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.creat * * eBaselineFrom(IlrSessionFacadeImpl.java:8526(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.creat * * eBaselineFrom(IlrSessionFacadeImpl.java:8515(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.creat * * eBaselineFrom(IlrSessionFacadeImpl.java:8515(Compiled Code)) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.creat * * eBaselineFromCurrentState(IlrSessionFacadeImpl.java:8471) * * at sun/reflect/NativeMethodAccessorImpl.invoke0(Native * * Method) * * at * * sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAcce * * ssorImpl.java:90(Compiled Code)) * * at * * sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMe * * thodAccessorImpl.java:55(Compiled Code)) * * at java/lang/reflect/Method.invoke(Method.java:508(Compiled * * Code)) * * at * * ilog/rules/teamserver/transaction/IlrTransactionalInvocation * * Handler.invoke(IlrTransactionalInvocationHandler.java:107(Co * * mpiled Code)) * * at * * com/sun/proxy/$Proxy300.createBaselineFromCurrentState(Bytec * * ode PC:45) * * at * * ilog/rules/teamserver/model/impl/IlrAbstractSession.createBa * * selineFromCurrentState(IlrAbstractSession.java:1964) * * at * * ilog/rules/teamserver/model/impl/IlrDecisionServiceHelper$6. * * run(IlrDecisionServiceHelper.java:1048) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.run(I * * lrSessionFacadeImpl.java:9550) * * at sun/reflect/GeneratedMethodAccessor648.invoke(Bytecode * * PC:42) * * at * * sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMe * * thodAccessorImpl.java:55(Compiled Code)) * * at java/lang/reflect/Method.invoke(Method.java:508(Compiled * * Code)) * * at * * ilog/rules/teamserver/transaction/IlrTransactionalInvocation * * Handler.invoke(IlrTransactionalInvocationHandler.java:107(Co * * mpiled Code)) * * at com/sun/proxy/$Proxy300.run(Bytecode PC:18) * * at * * ilog/rules/teamserver/model/impl/IlrAbstractSession.run(IlrA * * bstractSession.java:2594) * * at * * ilog/rules/teamserver/model/impl/IlrDecisionServiceHelper.cr * * eateDecisionServiceSnapshot(IlrDecisionServiceHelper.java:10 * * 62) * * at * * ilog/rules/teamserver/model/impl/IlrDecisionServiceHelper.cr * * eateDecisionServiceSnapshot(IlrDecisionServiceHelper.java:10 * * 26) * * at * * ilog/rules/teamserver/model/governance/commands/CompleteChan * * geActivity.apply(CompleteChangeActivity.java:109) * * at * * ilog/rules/teamserver/model/commands/ConcreteCommand.execute * * (ConcreteCommand.java:92) * * at * * ilog/rules/teamserver/model/governance/GovernanceCommand.exe * * cute(GovernanceCommand.java:72) * * at * * ilog/rules/teamserver/model/governance/AutomaticCommandEngin * * e.process(AutomaticCommandEngine.java:150) * * at * * ilog/rules/teamserver/model/governance/AutomaticCommandEngin * * e.processQueue(AutomaticCommandEngine.java:123) * * at * * ilog/rules/teamserver/ejb/service/IlrSessionFacadeImpl.trigg * * erAutomaticCommands(IlrSessionFacadeImpl.java:12691) * * at sun/reflect/GeneratedMethodAccessor1736.invoke(Bytecode * * PC:42) * * at * * sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMe * * thodAccessorImpl.java:55(Compiled Code)) * * at java/lang/reflect/Method.invoke(Method.java:508(Compiled * * Code)) * * at * * ilog/rules/teamserver/transaction/IlrTransactionalInvocation * * Handler.invoke(IlrTransactionalInvocationHandler.java:107(Co * * mpiled Code)) * * at com/sun/proxy/$Proxy300.triggerAutomaticCommands(Bytecode * * PC:18) * * at * * ilog/rules/teamserver/model/impl/IlrAbstractSession.triggerA * * utomaticCommands(IlrAbstractSession.java:418) * * at * * ilog/rules/teamserver/model/impl/IlrAbstractCachingSession.t * * riggerAutomaticCommands(IlrAbstractCachingSession.java:643) * * at * * ilog/rules/teamserver/model/impl/IlrAbstractSession.commit(I * * lrAbstractSession.java:1271(Compiled Code)) * * at * * ilog/rules/teamserver/model/impl/IlrAbstractCachingSession.c * * ommit(IlrAbstractCachingSession.java:150(Compiled Code)) * * at * * ilog/rules/teamserver/model/impl/IlrDecisionServiceHelper.co * * mmit(IlrDecisionServiceHelper.java:660) * * at * * ilog/rules/teamserver/model/changemanagement/IlrDefaultManag * * edBranchController$ApproverApprovesChanges.apply(IlrDefaultM * * anagedBranchController.java:614) * * at * * ilog/rules/teamserver/model/changemanagement/IlrAbstractOper * * ation.execute(IlrAbstractOperation.java:61) * * at * * com/ibm/rules/decisioncenter/web/core/changemanagement/Manag * * edBranchLifecycleController$1.run(ManagedBranchLifecycleCont * * roller.java:127) * * at * * com/ibm/rules/decisioncenter/web/core/changemanagement/Manag * * edBranchLifecycleController.applyOperation(ManagedBranchLife * * cycleController.java:131) * * at sun/reflect/NativeMethodAccessorImpl.invoke0(Native * * Method) * **************************************************************** * RECOMMENDATION: * ****************************************************************
Problem conclusion
The code is fixed.
Temporary fix
Comments
APAR Information
APAR number
RS03479
Reported component name
WS DECISION CTR
Reported component ID
5725B6900
Reported release
8A0
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2019-10-01
Closed date
2019-10-02
Last modified date
2019-10-02
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WS DECISION CTR
Fixed component ID
5725B6900
Applicable component levels
R8A0 PSY
UP
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSQP76","label":"IBM Operational Decision Manager"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8A0","Line of Business":{"code":"LOB45","label":"Automation"}}]
Document Information
Modified date:
24 November 2021