Fixes are available
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for IBM i
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for Windows
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for AIX
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for HP-UX
7.0.0.9: Java SDK 1.6 SR7 Cumulative Fix for WebSphere Application Server
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for Solaris
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for Linux
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for IBM i
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for Windows
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for HP-UX
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for AIX
7.0.0.11: Java SDK 1.6 SR7 Cumulative Fix for WebSphere Application Server
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for Solaris
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for Linux
6.1.0.31: Java SDK 1.5 SR11 FP1 Cumulative Fix for WebSphere Application Server
6.1.0.33: Java SDK 1.5 SR12 FP1 Cumulative Fix for WebSphere
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for AIX
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for HP-UX
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for IBM i
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for Linux
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for Solaris
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for Windows
7.0.0.13: Java SDK 1.6 SR8FP1 Cumulative Fix for WebSphere Application Server
6.1.0.35: Java SDK 1.5 SR12 FP2 Cumulative Fix for WebSphere
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for AIX
7.0.0.15: Java SDK 1.6 SR9 Cumulative Fix for WebSphere Application Server
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for HP-UX
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for IBM i
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for Linux
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for Solaris
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for Windows
6.1.0.37: Java SDK 1.5 SR12 FP3 Cumulative Fix for WebSphere
7.0.0.17: WebSphere Application Server V7.0 Fix Pack 17
7.0.0.17: Java SDK 1.6 SR9 FP1 Cumulative Fix for WebSphere Application Server
7.0.0.19: WebSphere Application Server V7.0 Fix Pack 19
7.0.0.21: WebSphere Application Server V7.0 Fix Pack 21
7.0.0.23: WebSphere Application Server V7.0 Fix Pack 23
7.0.0.25: WebSphere Application Server V7.0 Fix Pack 25
7.0.0.27: WebSphere Application Server V7.0 Fix Pack 27
7.0.0.29: WebSphere Application Server V7.0 Fix Pack 29
6.1.0.47: WebSphere Application Server V6.1 Fix Pack 47
7.0.0.31: WebSphere Application Server V7.0 Fix Pack 31
7.0.0.27: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.33: WebSphere Application Server V7.0 Fix Pack 33
7.0.0.35: WebSphere Application Server V7.0 Fix Pack 35
6.1.0.39: Java SDK 1.5 SR12 FP4 Cumulative Fix for WebSphere Application Server
6.1.0.41: Java SDK 1.5 SR12 FP5 Cumulative Fix for WebSphere Application Server
6.1.0.43: Java SDK 1.5 SR13 Cumulative Fix for WebSphere Application Server
6.1.0.45: Java SDK 1.5 SR14 Cumulative Fix for WebSphere Application Server
6.1.0.47: Java SDK 1.5 SR16 Cumulative Fix for WebSphere Application Server
7.0.0.19: Java SDK 1.6 SR9 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.21: Java SDK 1.6 SR9 FP2 Cumulative Fix for WebSphere
7.0.0.23: Java SDK 1.6 SR10 FP1 Cumulative Fix for WebSphere
7.0.0.25: Java SDK 1.6 SR11 Cumulative Fix for WebSphere Application Server
7.0.0.27: Java SDK 1.6 SR12 Cumulative Fix for WebSphere Application Server
7.0.0.29: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.45: Java SDK 1.6 SR16 FP60 Cumulative Fix for WebSphere Application Server
7.0.0.31: Java SDK 1.6 SR15 Cumulative Fix for WebSphere Application Server
7.0.0.35: Java SDK 1.6 SR16 FP1 Cumulative Fix for WebSphere Application Server
7.0.0.37: Java SDK 1.6 SR16 FP3 Cumulative Fix for WebSphere Application Server
7.0.0.39: Java SDK 1.6 SR16 FP7 Cumulative Fix for WebSphere Application Server
7.0.0.41: Java SDK 1.6 SR16 FP20 Cumulative Fix for WebSphere Application Server
7.0.0.43: Java SDK 1.6 SR16 FP41 Cumulative Fix for WebSphere Application Server
Obtain the fix for this APAR.
APAR status
Closed as program error.
Error description
When a high number of invalidation requests are being processed in a multi-Servant environment, a deadlock can occur which causes the server to become unresponsive and hang. . The customer was running a Portal server with 5 Servants. Portal pages were being accessed which also cause invalidations to flow . When an item is invalidated in the cache, the invalidation message is sent from the originating SR to the CR, which proxies it to ALL the SRs (excluding the origin SR) synchronously. If there are 5 servants, the flow is ... SR1 --> CR --> (SR2, SR3, SR4, SR5). If the call from one of the target SRs does not return, the original SR1 call is also hung. . ORB threads are used in the SR to send invalidations to the CR, as well as receive the proxied calls from other servants. If there is a lot of activity, the ORB thread used by the CR to proxy the invalidations to the other SRs can seem to be hung because the SR does not return. This can happen, if the DRSServantProxy call gets stuck in the SR waiting on locks owned by the other ORB threads sending invalidation requests to the other servants. . The CR might show the majority of its threads trying to proxy invalidation requests to the other servants: com/ibm/ws390/orb/ClientDelegate.jorbInvokeRequest Native Method com/ibm/ws390/orb/ClientDelegate.invoke (Compiled Code) org/omg/CORBA/portable/ObjectImpl._invoke ObjectImpl.java(Inlined Compiled Code) com/ibm/ws/drs/ws390/proxy/servant/_DRSServantProxyStub.updateEn tryProp (Compiled Code) com/ibm/ws/drs/ws390/DRSServantProxyWrapper.updateEntryProp DRSServantProxyWrapper.java(Inlined Compiled Code) com/ibm/ws/cache/drs/ws390/DynacacheDRSControllerDataXfer.update EntryProp DynacacheDRSControllerDataXfer.java(Compiled Code) com/ibm/ws/drs/ws390/proxy/controller/DRSControllerProxyHelper.u pdateEntryProp DRSControllerProxyHelper.java(Compiled Code) com/ibm/ws/drs/ws390/proxy/controller/DRSControllerProxyImpl.upd ateEntryProp DRSControllerProxyImpl.java(Inlined Compiled Code) com/ibm/ws/drs/ws390/proxy/controller/_DRSControllerProxyImpl_Ti e.updateEntryProp _DRSControllerProxyImpl_Tie.java(Compiled Code) com/ibm/ws/drs/ws390/proxy/controller/_DRSControllerProxyImpl_Ti e._invoke _DRSControllerProxyImpl_Tie.java(Compiled Code) sun/reflect/GeneratedMethodAccessor8.invoke (Compiled Code) sun/reflect/DelegatingMethodAccessorImpl.invoke DelegatingMethodAccessorImpl.java(Compiled Code) java/lang/reflect/Method.invoke Method.java(Compiled Code) com/ibm/ws390/orb/RIRProtectedProxyHandler.invoke (Compiled Code) $Proxy0._invoke (Compiled Code) com/ibm/ws390/orb/CommonBridge.invoke (Compiled Code) com/ibm/ws390/orb/ORBEJSBridge.invoke ORBEJSBridge.java(Inlined Compiled Code) com/ibm/ws390/orb/parameters/ORBEJSBridgeInvoke.ORBEJSBridgeInvo keParmSetter .
Local fix
Problem summary
**************************************************************** * USERS AFFECTED: IBM WebSphere Application Server V6.0.1 * * users of dynacache with multiple servant * * regions. * **************************************************************** * PROBLEM DESCRIPTION: A deadlock occurs and the server * * hangs when a dynacache user requests * * a high number of invalidations and * * there are multiple servant regions. * **************************************************************** * RECOMMENDATION: * **************************************************************** Data Replication Services (DRS) uses proxy calls to flow updates between the control region (CR) and servant regions (SR). These proxy calls run on Object Request Broker (ORB) threads. The SRs call back into the CR and a deadlock occurs because all the ORB threads are in use and a thread is not available to handle a proxied call from the SR.
Problem conclusion
Dynacache has been modified to batch invalidations on the SR and immediately return so the ORB thread is freed up. The invalidations are then processed on a daemon thread at periodic intervals. APAR PM02163 is currently targeted for inclusion in Service Level (Fix Pack) 6.0.2.41 of WebSphere Application Server V6.0.1. Please refer to URL: //www.ibm.com/support/docview.wss?rs=404&uid=swg27006970 for Fix Pack availability.
Temporary fix
Comments
APAR Information
APAR number
PM02163
Reported component name
WEBSPHERE FOR Z
Reported component ID
5655I3500
Reported release
601
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2009-11-25
Closed date
2010-01-20
Last modified date
2010-05-04
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WEBSPHERE FOR Z
Fixed component ID
5655I3500
Applicable component levels
R601 PSY UK55832
UP10/04/16 P F004
Fix is available
Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.
Document Information
Modified date:
28 December 2021