IBM Support

java.lang.OutOfMemoryError or Java process hang and a significant number of ORB threads in the thread dumps

Troubleshooting


Problem

This problem would likely manifest itself as a hung Application Server. In the javacore or thread dump of the process, a large number of ORB threads with thread ids beginning with RT=# or LT=# will be observed.

Cause

This problem is a fairly specific problem caused by an Enterprise JavaBeans having an ORB.init() call in it. When the ORB.init() call is made, it causes an ORB instance to be created. When an ORB instance is created it will create five threads: four listener threads and 1 reader thread. These threads would only get destroyed if the ORB instance is destroyed. If over time, more ORB instances are created, eventually the Application Server performance will be affected. It may also cause java.lang.OutOfMemoryErrors to be thrown.

The javacore file or thread dump shows that there are a large number of ORB listener threads and ORB reader threads.

This is what an ORB reader thread looks like.
RT=383:P=570960:O=123:WSSSLTransportConnection[addr=165.245.2.81,port=47765,local=46289]" (TID:0x57FD8938, sys_thread_t:0x9AB1F3B0, state:R, native ID:0x22EBFC) prio=5

The O= value is the ORB instance id. This says that there are 123 ORB instances on this Application Server. The address shows the address that the ORB connection is to; the port is the port number that is used.

This is what an ORB listener thread id looks like.
"LT=496:P=570960:O=123:port=46288" (TID:0x57FD8990, sys_thread_t:0x9AB1EF10, state:R, native ID:0x22E7FB) prio=5

Again the O=# is the ORB instance id. The port is the socket port that this is listening on.

Resolving The Problem

This is caused by an ORB.init() call in an Enterprise JavaBeans. The ORB.init() call is not needed for an Enterprise JavaBeans running under IBM® WebSphere® Application Server since there is an Enterprise JavaBeans Container that initiates the ORB; the Enterprise JavaBeans should not do so itself.

[{"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"Object Request Broker (ORB)","Platform":[{"code":"PF002","label":"AIX"}],"Version":"8.5.5;8.5;8.0;7.0","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}},{"Product":{"code":"SSNVBF","label":"Runtimes for Java Technology"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Java SDK","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
15 June 2018

UID

swg21224719