Fixes are available
8.5.5.15: WebSphere Application Server V8.5.5 Fix Pack 15
8.5.5.17: WebSphere Application Server V8.5.5 Fix Pack 17
8.5.5.20: WebSphere Application Server V8.5.5.20
8.5.5.18: WebSphere Application Server V8.5.5 Fix Pack 18
8.5.5.19: WebSphere Application Server V8.5.5 Fix Pack 19
8.5.5.16: WebSphere Application Server V8.5.5 Fix Pack 16
8.5.5.21: WebSphere Application Server V8.5.5.21
APAR status
Closed as program error.
Error description
WAS ND 8.5.5.12 on Linux Thread Name Thread-62 State Waiting on condition Monitor Owns Monitor Lock on com/ibm/ws/odc/nd/ODCTreeImpl@0x0000000080BC35F8 Java Stack at com/ibm/apc/xd/demand/predictors/APCXDDemandPredictorShim$1. cell Added(APCXDDemandPredictorShim.java:59) at com/ibm/ws/xd/container/state/odc/impl/XdCsODCHelper_Notific atio n.notifyListeners_Cell(XdCsODCHelper_Notification.java:309) at com/ibm/ws/xd/container/state/odc/impl/XdCsODCSensor$1.handl eCha nge_CellAddRemove(XdCsODCSensor.java:566) at com/ibm/ws/xd/container/state/odc/impl/XdCsODCDeMuxImpl.demu xHan dler_ODCEventNodeChange(XdCsODCDeMuxImpl.java:246(Compiled Code)) at com/ibm/ws/xd/container/state/odc/impl/XdCsODCDeMuxImpl.rout eOdc ConfigChangeEvent(XdCsODCDeMuxImpl.java:113(Compiled Code)) at com/ibm/ws/xd/container/state/odc/impl/XdCsODCSensor$2.handl eEve nt(XdCsODCSensor.java:680(Compiled Code)) at com/ibm/ws/odc/nd/ODCTreeImpl.commitTransaction(ODCTreeImpl. java :1182(Compiled Code)) at com/ibm/ws/odc/nd/ODCTreeImpl.commitTransaction(ODCTreeImpl. java :959) at com/ibm/ws/odc/nd/ODCTreeImpl.performEvents(ODCTreeImpl.java :275 ) at com/ibm/ws/wsgroup/nd/odc/ODCGroup.performEvents(ODCGroup.ja va:1 637) at com/ibm/ws/wsgroup/nd/odc/ODCGroup.handleGetTreeResponse(ODC Grou p.java:1385(Compiled Code)) at com/ibm/ws/wsgroup/nd/odc/ODCGroup.consumerOnMessage(ODCGrou p.ja va:550(Compiled Code)) at com/ibm/ws/wsgroup/nd/odc/ODCGroup.processMessage(ODCGroup.j ava: 725(Compiled Code)) at com/ibm/ws/wsgroup/nd/odc/ODCGroup.onMessage(ODCGroup.java:6 82(C ompiledCode)) at com/ibm/ws/wsgroup/WsGroup.deliverMessage(WsGroup.java:219(C ompi ledCode)) at com/ibm/ws/wsgroup/bb/CombinedGroup.onP2PMessage(CombinedGro up.j ava:294(Compiled Code)) at com/ibm/ws/wsgroup/bb/CombinedGroup$MyP2PGroupListener.onMes sage (CombinedGroup.java:449(Compiled Code)) at com/ibm/ws/wsgroup/WsGroup.deliverMessage(WsGroup.java:219(C ompi ledCode)) at com/ibm/ws/wsgroup/WsGroup.receiveMessage(WsGroup.java:212(C ompi ledCode)) at com/ibm/ws/wsgroup/WsGroup.receiveMessages(WsGroup.java:194( Comp iledCode)) at com/ibm/ws/wsgroup/p2p/P2PGroup.calledWhenMessageArrives(P2P Grou p.java:209(Compiled Code)) at com/ibm/ws/wsgroup/p2p/SonP2PShimImpl$Worker.oneStep(SonP2PS himI mpl.java:1347(Compiled Code)) at com/ibm/ws/wsgroup/p2p/SonP2PShimImpl$Worker.run(SonP2PShimI mpl. java:1213) at java/lang/Thread.run(Thread.java:785)
Local fix
Problem summary
**************************************************************** * USERS AFFECTED: All users of IBM WebSphere Application * * Server WAS ND edition- Virtual * * Enterprise/IM Component * **************************************************************** * PROBLEM DESCRIPTION: Hung thread attempting to initialize * * com/ibm/apc/xd/demand/predictors/APCXDD * * emandPredictorShim.<clinit>. * **************************************************************** * RECOMMENDATION: * **************************************************************** In dmgr or nodeagent processes a thread is hung attempting to initialize com/ibm/apc/xd/demand/predictors/APCXDDemandPredictorShim.<clini t>(APCXDDemandPredictorShim.java:69) and is blocked waiting on com/ibm/ws/odc/nd/ODCTreeImpl. The signature of the deadlock will reflect the following two thread snippets: 3XMTHREADINFO "Thread-62" J9VMThread:0x0000000002CE3F00, omrthread_t:0x00007FC45441BBB8, java/lang/Thread:0x00000000883CF120, state:B, prio=5 3XMJAVALTHREAD (java/lang/Thread getId:0x76, isDaemon:false) 3XMTHREADINFO1 (native thread ID:0x331B, native priority:0x5, native policy:UNKNOWN, vmstate:B, vm thread flags:0x00000201) 3XMTHREADINFO2 (native stack address range from:0x00007FC3EDE89000, to:0x00007FC3EDEC9000, size:0x40000) 3XMCPUTIME CPU usage total: 0.020904504 secs, current category="Application" 3XMTHREADBLOCK Blocked on: com/ibm/ws/odc/nd/ODCTreeImpl@0x00000000877B9E58 Owned by: "P=674747:O=0:CT" (J9VMThread:0x0000000000BC4700, java/lang/Thread:0x0000000085DB3CA0) 3XMHEAPALLOC Heap bytes allocated since last GC cycle=0 (0x0) 3XMTHREADINFO3 Java callstack: 4XESTACKTRACE at com/ibm/ws/xd/container/state/odc/impl/XdCsODCSensor.start(XdCsO DCSensor.java:804) 4XESTACKTRACE at com/ibm/ws/xd/container/state/odc/impl/XdCsODCSensor.getInstance (XdCsODCSensor.java:772) 5XESTACKTRACE (entered lock: com/ibm/ws/xd/container/state/odc/impl/XdCsODCSensor@0x000000008 7541798, entry count: 1) 4XESTACKTRACE at com/ibm/ws/xd/container/state/odc/impl/XdCsSubscriptionManagerFa ctory.resolve(XdCsSubscriptionManagerFactory.java:49) 4XESTACKTRACE at com/ibm/ws/xd/container/state/odc/impl/XdCsODCHelper_Subscriptio n.subscribe(XdCsODCHelper_Subscription.java:69) 5XESTACKTRACE (entered lock: com/ibm/apc/xd/demand/predictors/APCXDDemandPredictorShim$1@0x00 000000883D87D8, entry count: 1) 4XESTACKTRACE at com/ibm/ws/xd/container/state/odc/XdCsListener_Cell.subscribe(Xd CsListener_Cell.java:46) 4XESTACKTRACE at com/ibm/apc/xd/demand/predictors/APCXDDemandPredictorShim.<clini t>(APCXDDemandPredictorShim.java:69) 3XMTHREADINFO "P=674747:O=0:CT" J9VMThread:0x0000000000BC4700, omrthread_t:0x00007FC474007FD0, java/lang/Thread:0x0000000085DB3CA0, state:CW, prio=5 3XMJAVALTHREAD (java/lang/Thread getId:0x1, isDaemon:false) 3XMTHREADINFO1 (native thread ID:0x3259, native priority:0x5, native policy:UNKNOWN, vmstate:CW, vm thread flags:0x00000101) 3XMTHREADINFO2 (native stack address range from:0x00007FC47B885000, to:0x00007FC47C085000, size:0x800000) 3XMCPUTIME CPU usage total: 8.928669194 secs, current category="Application" 3XMTHREADBLOCK Waiting on: java/lang/J9VMInternals$ClassInitializationLock@0x000000008826EC 68 Owned by: <unowned> 3XMHEAPALLOC Heap bytes allocated since last GC cycle=0 (0x0) 3XMTHREADINFO3 Java callstack: 4XESTACKTRACE at sun/misc/Unsafe.ensureClassInitialized(Native Method) 4XESTACKTRACE at java/lang/J9VMInternals.initialize(J9VMInternals.java:87(Compile d Code))
Problem conclusion
The Application Placement Controller (APC) code was altered such that it no longer requires obtaining locks within global static blocks of com.ibm.apc.xd.demand.predictors.APCXDDemandPredictorShim which execute during class initialization, avoiding the possibility for deadlock in this area. The fix for this APAR is currently targeted for inclusion in fix packs 8.5.5.15 and 9.0.0.10. Please refer to the Recommended Updates page for delivery information: http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
Temporary fix
Comments
APAR Information
APAR number
PI88622
Reported component name
WEBS APP SERV N
Reported component ID
5724H8800
Reported release
850
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2017-10-10
Closed date
2018-09-10
Last modified date
2018-09-10
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WEBS APP SERV N
Fixed component ID
5724H8800
Applicable component levels
R850 PSY
UP
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.5","Line of Business":{"code":"LOB45","label":"Automation"}}]
Document Information
Modified date:
28 April 2022