Troubleshooting
Problem
While running WebSphere Application Server under a heavy load, it can throw OutOfMemoryError (OOM) when the heap is fully exhausted and produce a heapdump and javacore. If the heapdump shows a large number of com.ibm.jsse2.SSLSocketImpl objects as a leak suspect, this might be caused by an APAR with WebSphere MQ which blocks Finalizer thread from garbage collection.
Cause
WebSphere MQ (WMQ) Resource Adapter, shipped with WebSphere Application Server, had a defect which blocked the finalizer thread while closing a ServerSession object. This would then block the garbage collection of all objects with a finalize method, leading to OutOfMemory events. The largest heap user might be com.ibm.jsse2.SSLSocketImpl objects. This problem was fixed under WMQ APAR IV14093 which shipped with certain levels of WebSphere Application Server Fix Packs.
Environment
WebSphere Application Server V7.0, V8,0, V8.5
Diagnosing The Problem
Collect a heapdump and javacore from an OOM event. Use Memory Analyzer or IBM HeapAnalyzer for heapdump analysis. You might see "com.ibm.jsse2.SSLSocketImpl" as a leak suspect.
(For example)
16,985 instances of "com.ibm.jsse2.SSLSocketImpl" 594,807,720 (59.86%) byte
If corresponding javacore shows a Finalizer thread is waiting on "com/ibm/mq/connector/inbound/ServerSessionImpl$IsInUseLock", the root cause is most likely WMQ APAR IV14093.
(For example)
"Finalizer thread"
Waiting on: com/ibm/mq/connector/inbound/ServerSessionImpl$IsInUseLock@0x976F7DE0 Owned by: <unowned>
Java callstack:
at java/lang/Object.wait(Native Method)
at java/lang/Object.wait(Object.java:167(Compiled Code))
at com/ibm/mq/connector/inbound/ServerSessionImpl.close(ServerSessionImpl.java:285)
(entered lock:com/ibm/mq/connector/inbound/ServerSessionImpl$IsInUseLock
@0x976F7DE0, entry count: 1)
at com/ibm/mq/connector/inbound/ServerSessionImpl.finalize(ServerSessionImpl.java:397)
at java/lang/J9VMInternals.runFinalize(J9VMInternals.java:455(Compiled Code))
Resolving The Problem
Applying WebSphere Application Server Fix Pack 8.5.0.2, 8.0.0.5, 7.0.0.27 or a later Fix Pack.
Recommended fixes for WebSphere Application Server
http://www.ibm.com/support/docview.wss?uid=swg27004980
The APAR is included in WMQ Fix Pack 7.0.1.9 and 7.1.0.2.
IV14093: OUTOFMEMORYEXCEPTION IN WEBSPHERE APPLICATION SERVER DUE TO HUNG FINALIZER THREAD WHILE PROCESSING WMQ CONNECTION EVENT
http://www.ibm.com/support/docview.wss?uid=swg1IV14093
- WebSphere MQ Fix Pack 7.0.1.9
- WebSphere MQ Fix Pack 7.1.0.2
Those version of WMQ are shipped with WebSphere Application Server Fix Pack in 8.5.0.2, 8.0.0.5 and 7.0.0.27.
Which version of WebSphere MQ is shipped with WebSphere Application Server ?
http://www.ibm.com/support/docview.wss?uid=swg21248089
WebSphere Application Server Version 8.5 ships with the WebSphere MQ 7.1 Resource Adapter.
- WAS 8.5.0.2 - WMQ 7.1.0.2 - This level includes APAR IV14093
- WAS 8.5.0.1 - WMQ 7.1.0.1
- WAS 8.5.0.0 - WMQ 7.1.0.0
WebSphere Application Server Version 8 ships with the WebSphere MQ 7.0 Resource Adapter.
- WAS 8.0.0.5 - WMQ 7.0.1.9 - This level includes APAR IV14093
- WAS 8.0.0.4 - WMQ 7.0.1.7
- WAS 8.0.0.3 - WMQ 7.0.1.7
- WAS 8.0.0.2 - WMQ 7.0.1.6
- WAS 8.0.0.1 - WMQ 7.0.1.5
WebSphere Application Server Version 7 ships with the WebSphere MQ 7.0 Resource Adapter.
- WAS 7.0.0.27 - WMQ 7.0.1.9 - This level includes APAR IV14093
- WAS 7.0.0.25 - WMQ 7.0.1.7
- WAS 7.0.0.23 - WMQ 7.0.1.7
- WAS 7.0.0.21 - WMQ 7.0.1.6
- WAS 7.0.0.19 - WMQ 7.0.1.5
- WAS 7.0.0.17 - WMQ 7.0.1.4
Was this topic helpful?
Document Information
Modified date:
15 June 2018
UID
swg21647169