Fixes are available
7.0.0.19: WebSphere Application Server V7.0 Fix Pack 19
7.0.0.21: WebSphere Application Server V7.0 Fix Pack 21
7.0.0.23: WebSphere Application Server V7.0 Fix Pack 23
7.0.0.25: WebSphere Application Server V7.0 Fix Pack 25
7.0.0.27: WebSphere Application Server V7.0 Fix Pack 27
7.0.0.29: WebSphere Application Server V7.0 Fix Pack 29
6.1.0.47: WebSphere Application Server V6.1 Fix Pack 47
7.0.0.31: WebSphere Application Server V7.0 Fix Pack 31
7.0.0.27: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.33: WebSphere Application Server V7.0 Fix Pack 33
7.0.0.35: WebSphere Application Server V7.0 Fix Pack 35
7.0.0.37: WebSphere Application Server V7.0 Fix Pack 37
7.0.0.39: WebSphere Application Server V7.0 Fix Pack 39
7.0.0.41: WebSphere Application Server V7.0 Fix Pack 41
7.0.0.43: WebSphere Application Server V7.0 Fix Pack 43
7.0.0.45: WebSphere Application Server V7.0 Fix Pack 45
6.1.0.41: Java SDK 1.5 SR12 FP5 Cumulative Fix for WebSphere Application Server
6.1.0.43: Java SDK 1.5 SR13 Cumulative Fix for WebSphere Application Server
6.1.0.45: Java SDK 1.5 SR14 Cumulative Fix for WebSphere Application Server
6.1.0.47: Java SDK 1.5 SR16 Cumulative Fix for WebSphere Application Server
7.0.0.19: Java SDK 1.6 SR9 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.21: Java SDK 1.6 SR9 FP2 Cumulative Fix for WebSphere
7.0.0.23: Java SDK 1.6 SR10 FP1 Cumulative Fix for WebSphere
7.0.0.25: Java SDK 1.6 SR11 Cumulative Fix for WebSphere Application Server
7.0.0.27: Java SDK 1.6 SR12 Cumulative Fix for WebSphere Application Server
7.0.0.29: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.45: Java SDK 1.6 SR16 FP60 Cumulative Fix for WebSphere Application Server
7.0.0.31: Java SDK 1.6 SR15 Cumulative Fix for WebSphere Application Server
7.0.0.35: Java SDK 1.6 SR16 FP1 Cumulative Fix for WebSphere Application Server
7.0.0.37: Java SDK 1.6 SR16 FP3 Cumulative Fix for WebSphere Application Server
7.0.0.39: Java SDK 1.6 SR16 FP7 Cumulative Fix for WebSphere Application Server
7.0.0.41: Java SDK 1.6 SR16 FP20 Cumulative Fix for WebSphere Application Server
7.0.0.43: Java SDK 1.6 SR16 FP41 Cumulative Fix for WebSphere Application Server
Obtain the fix for this APAR.
APAR status
Closed as program error.
Error description
An app was updated/redeployed without quiescing the server. A deadlock was encountered which ended up causing the servant to timeout with ABENDSEC3 SEC3 RSN04130004. This can occur under the following circumstances A) the server has been configured to initialize all EJBs at application startup time or The server has a startup bean deployed in it or the server has an MDB deployed in it B) an app deployment is in progress C) inbound remote EJB requests are coming in to the server D) an EJB being invoked just so happens to overlap the EJB wrapper for an EJB just initializing Javacore from the failure contains: 1LKDEADLOCK Deadlock detected !!! NULL --------------------- NULL 2LKDEADLOCKTHR Thread "WebSphere:ORB.thread.pool t=007bd3e8" (0x5B3DB100) 3LKDEADLOCKWTR is waiting for: 4LKDEADLOCKMON sys_mon_t:0x572541C0 infl_mon_t: 0x57254200: 4LKDEADLOCKOBJ com/ibm/ejs/csi/J2EENameImpl@53711A30/53711A3C: 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "WebSphere t=007bb718" (0x5B414E00) 3LKDEADLOCKWTR which is waiting for: 4LKDEADLOCKMON sys_mon_t:0x57703C4C infl_mon_t: 0x57703C8C: 4LKDEADLOCKOBJ com/ibm/ejs/util/cache/WrapperBucket@36A5B778/36A5B784: 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "WebSphere:ORB.thread.pool t=007bd3e8" (0x5B3DB100) Call stack for "WebSphere:ORB.thread.pool t=007bd3e8" ====================================================== "WebSphere:ORB.thread.pool t=007bd3e8" (TID:0x5B3DB100, sys_thread_t:0x5B3DBAF0, state:B, native ID:0x3B5E9000) prio=5 at com/ibm/ejs/container/HomeRecord.getHomeAndInitialize( HomeRecord.java:467) at com/ibm/ejs/container/EJSContainer.getHomeWrapperCommon( EJSContainer.java:1273) at com/ibm/ejs/container/EJSContainer.getHomeInstance( EJSContainer.java:1182) at com/ibm/ejs/container/EJSContainer.startBean( EJSContainer.java:1168) at com/ibm/ws/runtime/component/EJBContainerImpl.startBean( EJBContainerImpl.java:3484) at com/ibm/ws/runtime/component/EJBContainerImpl. startAndBindHome(EJBContainerImpl.java:4761) at com/ibm/ws/runtime/component/EJBContainerImpl. initializeDeferredEJB(EJBContainerImpl.java:4546) at com/ibm/ejs/container/HomeOfHomes.getHome( HomeOfHomes.java:345) at com/ibm/ejs/container/HomeOfHomes.internalCreateWrapper( HomeOfHomes.java:481) at com/ibm/ejs/container/EJSContainer.createWrapper( EJSContainer.java:4396(Compiled Code)) at com/ibm/ejs/container/WrapperManager.faultOnKey( WrapperManager.java:528(Compiled Code)) at com/ibm/ejs/util/cache/Cache.findAndFault(Cache.java:497( Compiled Code)) at com/ibm/ejs/container/WrapperManager.keyToObject( WrapperManager.java:481(Compiled Code)) at com/ibm/ejs/oa/EJSOAImpl.keyToObject(EJSOAImpl.java:553( Compiled Code)) at com/ibm/ejs/oa/EJSRootOAImpl.keyToObject( EJSRootOAImpl.java:271(Compiled Code)) at com/ibm/ws390/orb/CommonBridge.invoke(Bytecode PC:777(Compiled Code)) at com/ibm/ws390/orb/ORBEJSBridge.invoke(Bytecode PC:6( Compiled Code)) at sun/reflect/GeneratedMethodAccessor234.invoke(Bytecode PC:24( Compiled Code)) at sun/reflect/DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43(Compiled Code)) at java/lang/reflect/Method.invoke(Method.java:618( Compiled Code)) at com/ibm/ws390/orb/parameters/ORBEJSBridgeInvoke. ORBEJSBridgeInvokeParmSetter( ORBEJSBridgeInvoke.java:159(Compiled Code)) at com/ibm/ws390/orb/CommonBridge.nativeRunApplicationThread( Native Method) at com/ibm/ws390/orb/CommonBridge.runApplicationThread( Bytecode PC:7) at com/ibm/ws/util/ThreadPool$ZOSWorker.run( ThreadPool.java:1697) Call stack for "WebSphere t=007bb718" ===================================== "WebSphere t=007bb718" (TID:0x5B414E00, sys_thread_t:0x5B412730, state:B, native ID:0x3B629D00) prio=5 at com/ibm/ejs/util/cache/Cache.findAndFault(Cache.java:463( Compiled Code)) at com/ibm/ejs/container/WrapperManager.getWrapper( WrapperManager.java:277(Compiled Code)) at com/ibm/ejs/container/EJSHome.getWrapper(EJSHome.java:776) at com/ibm/ejs/container/EJSContainer.loadBeanMetaData( EJSContainer.java:1697(Compiled Code)) at com/ibm/ejs/container/HomeOfHomes.loadBeanMetaData( HomeOfHomes.java:641) at com/ibm/ejs/container/HomeRecord.getHomeAndInitialize( HomeRecord.java:479) at com/ibm/ejs/container/EJSContainer.getHomeWrapperCommon( EJSContainer.java:1273) at com/ibm/ejs/container/EJSContainer.getHomeInstance( EJSContainer.java:1182) at com/ibm/ejs/container/EJSContainer.startBean( EJSContainer.java:1168) at com/ibm/ws/runtime/component/EJBContainerImpl.startBean( EJBContainerImpl.java:3484) at com/ibm/ws/runtime/component/EJBContainerImpl.install( EJBContainerImpl.java:2888(Compiled Code)) at com/ibm/ws/runtime/component/EJBContainerImpl.start( EJBContainerImpl.java:3730) at com/ibm/ws/runtime/component/ApplicationMgrImpl.start( ApplicationMgrImpl.java:1250(Compiled Code)) at com/ibm/ws/runtime/component/DeployedApplicationImpl. fireDeployedObjectStart(DeployedApplicationImpl.java:1140) at com/ibm/ws/runtime/component/DeployedModuleImpl.start( DeployedModuleImpl.java:572(Compiled Code)) at com/ibm/ws/runtime/component/DeployedApplicationImpl.start( DeployedApplicationImpl.java:819(Compiled Code)) at com/ibm/ws/runtime/component/ApplicationMgrImpl. startApplication( ApplicationMgrImpl.java:921(Compiled Code)) at com/ibm/ws/runtime/component/ApplicationMgrImpl$1.run( ApplicationMgrImpl.java:1437) at com/ibm/ws/security/auth/ContextManagerImpl.runAs( ContextManagerImpl.java:4158(Compiled Code)) at com/ibm/ws/security/auth/ContextManagerImpl.runAsSystem( ContextManagerImpl.java:4255(Compiled Code)) at com/ibm/ws/security/core/SecurityContext.runAsSystem( SecurityContext.java:245) at com/ibm/ws/runtime/component/ApplicationMgrImpl. startApplication( ApplicationMgrImpl.java:1442(Compiled Code)) at sun/reflect/NativeMethodAccessorImpl.invoke0(Native Method) at sun/reflect/NativeMethodAccessorImpl.invoke( NativeMethodAccessorImpl.java:79(Compiled Code)) at sun/reflect/DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43(Compiled Code)) at java/lang/reflect/Method.invoke(Method.java:618( Compiled Code)) at sun/reflect/misc/Trampoline.invoke(MethodUtil.java:68) at sun/reflect/GeneratedMethodAccessor238.invoke(Bytecode PC:40) at sun/reflect/DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43(Compiled Code)) at java/lang/reflect/Method.invoke(Method.java:618( Compiled Code)) at sun/reflect/misc/MethodUtil.invoke(MethodUtil.java:271) at javax/management/modelmbean/RequiredModelMBean.invokeMethod( RequiredModelMBean.java:1092) at javax/management/modelmbean/RequiredModelMBean.invoke( RequiredModelMBean.java:973) at com/sun/jmx/mbeanserver/DynamicMetaDataImpl.invoke( DynamicMetaDataImpl.java:231) at com/sun/jmx/mbeanserver/MetaDataImpl.invoke( MetaDataImpl.java:238) at com/sun/jmx/interceptor/DefaultMBeanServerInterceptor.invoke( DefaultMBeanServerInterceptor.java:833) at com/sun/jmx/mbeanserver/JmxMBeanServer.invoke( JmxMBeanServer.java:802) at com/ibm/ws390/management/connector/corba/CorbaConnectorImpl. invoke(CorbaConnectorImpl.java:705) at com/ibm/ws390/management/connector/corba/ _CorbaConnectorImplBase. _invoke(_CorbaConnectorImplBase.java:406) at com/ibm/ws390/orb/CommonBridge.invoke( Bytecode PC:1337(Compiled Code)) at com/ibm/ws390/orb/ORBEJSBridge.invoke(Bytecode PC:6( Compiled Code)) at sun/reflect/GeneratedMethodAccessor234.invoke( Bytecode PC:24(Compiled Code)) at sun/reflect/DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43(Compiled Code)) at java/lang/reflect/Method.invoke(Method.java:618( Compiled Code)) at com/ibm/ws390/orb/parameters/ORBEJSBridgeInvoke. ORBEJSBridgeInvokeParmSetter( ORBEJSBridgeInvoke.java:159(Compiled Code))
Local fix
quiesce workload before attempting to deploy a new version of the application.
Problem summary
**************************************************************** * USERS AFFECTED: All users of IBM WebSphere Application * * Server with Enterprise Java Beans (EJBs) * **************************************************************** * PROBLEM DESCRIPTION: Application start with EJBs can * * deadlock if the first request to an * * EJB is made simultaneously. * **************************************************************** * RECOMMENDATION: * **************************************************************** Remote EJB request processing acquires a hash table bucket lock; if the remote request is the first call to that EJB, then an application-wide lock is acquired. During application start of non-deferred EJBs, such as MDBs or startup beans, the application-wide lock is acquired and then the hash table bucket lock is acquired. If these two events happen concurrently, a deadlock occurs.
Problem conclusion
The application start code path was changed to acquire the hash table bucket lock before the application-wide lock, which prevents the deadlock. APAR PM37960 is currently targeted for inclusion in Service Level (Fix Pack) 6.1.0.41 of WebSphere Application Server V6.1. Please refer to URL: //www.ibm.com/support/docview.wss?rs=404&uid=swg27006970 for Fix Pack availability.
Temporary fix
Comments
APAR Information
APAR number
PM37960
Reported component name
WEBSPHERE FOR Z
Reported component ID
5655I3500
Reported release
610
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2011-04-28
Closed date
2011-05-10
Last modified date
2011-12-02
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WEBSPHERE FOR Z
Fixed component ID
5655I3500
Applicable component levels
R610 PSY UK73054
UP11/11/03 P F111
Fix is available
Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.
Document Information
Modified date:
27 October 2021