Fixes are available
APAR status
Closed as program error.
Error description
A Message Driven Bean(MDB) running on WebSphere Application Server(WAS) v7.0.x fails to process messages that are received from WebSphere MQ v7.0.x.This results in a hang situation. The following errors are written to WAS SystemOut.log: CWWMQ0007W: The message endpoint XXX has been paused by the system. Message delivery failed to the endpoint more than 2 times.The last attempted delivery failed with the following error: javax.jms.TransactionRolledBackException: at com.ibm.mq.connector.inbound.WorkImpl.call DeliveryFailed(WorkImpl.java:511) at com.ibm.mq.connector.inbound.WorkImpl.xa StateChanged(WorkImpl.java:460) at com.ibm.mq.connector.xa.XAObservable. update(XAObservable.java:120) at com.ibm.mq.connector.xa.XARWrapper. rollback(XARWrapper.java:444) at com.ibm.tx.jta.JTAXAResourceImpl. rollback(JTAXAResourceImpl.java:363) at com.ibm.tx.jta.RegisteredResources. deliverOutcome(RegisteredResources.java:1647) J2CA0140W: The Message Endpoint pause operation failed for ActivationSpec XXX and MDB Application XYZ because the endpoint is currently deactivated or stopped, .or an unexpected exception occurred deactivating the endpoint. J2CA0530I: The Message Endpoint for ActivationSpec XXX and MDB Application XYZ is partially activated but has been disabled from handling incoming messages. J2CA0520W: The Message Endpoint resume operation failed for ActivationSpec XXX and MDB Application XYZ because the endpoint is currently activated or stopped, or an unexpected exception occurred activating the endpoint. javacores show hung threads with the following stack: Blocking Thread:Thread:1: at java/lang/Object.wait(Native Method) at java/lang/Object.wait(Object.java:167(Compiled Code)) at com/ibm/mq/connector/inbound/ServerSessionImpl.close (ServerSessionImpl.java:298) at com/ibm/mq/connector/inbound/ServerSessionPoolImpl. closeInternal(ServerSessionPoolImpl.java:648) at com/ibm/mq/connector/inbound/ServerSessionPoolImpl. close(ServerSessionPoolImpl.java:557) at com/ibm/mq/connector/inbound/MessageEndpointDeployment. stop(MessageEndpointDeployment.java:455) at com/ibm/mq/connector/ResourceAdapterImpl.endpoint Deactivation(ResourceAdapterImpl.java:523) at com/ibm/ejs/j2c/ActivationSpecWrapperImpl.deactivate UnderRAClassLoaderContext(ActivationSpecWrapperImpl. java:508) at com/ibm/ejs/j2c/ActivationSpecWrapperImpl.deactivate EndPoint(ActivationSpecWrapperImpl.java:411) at com/ibm/ejs/j2c/mbeans/MessageEndpointMBeanImpl.pause (MessageEndpointMBeanImpl.java:162) Blocked thread:Thread:2 at com/ibm/mq/connector/ResourceAdapterImpl.endpoint Deactivation(ResourceAdapterImpl.java:516) at com/ibm/ejs/j2c/ActivationSpecWrapperImpl.deactivate UnderRAClassLoaderContext(ActivationSpecWrapperImpl. java:508) at com/ibm/ejs/j2c/ActivationSpecWrapperImpl.deactivate EndPoint(ActivationSpecWrapperImpl.java:411) at com/ibm/ejs/j2c/mbeans/MessageEndpointMBeanImpl.pause (MessageEndpointMBeanImpl.java:162)
Local fix
Problem summary
**************************************************************** USERS AFFECTED: This issue affects users of the WebSphere MQ Messaging Provider supplied with WebSphere Application Server, who have configured one or more Activation Specifications for the Messaging Provider, and are attempting to pause the activation specification(s) either manually using the application server administration console, or automatically as a result of a delivery failure. Platforms affected: All Distributed (iSeries, all Unix and Windows) +Java +Java zOS **************************************************************** PROBLEM SUMMARY: When delivery of a message to a message driven bean (MDB) fails, or the message driven bean reports an exception during message processing, the WebSphere MQ Messaging Provider notifies the application server of the failure event, in order to allow the application server to make the appropriate action. At this point, if the application server configuration permits, the application server may respond with the decision to pause the activation specification. When pausing an activation specification, the thread performing the pause must wait for all other threads associated with the Activation Specification to complete their current work before the pause operation can be completed. In the case where multiple threads are associated with the Activation Specification concurrently receive notification of a failed delivery, all threads receiving this notification will also receive the pause decision from the application server. This results in multiple threads simultaneously attempting to pause the Activation Specification. This potentially lead to a deadlock whereby multiple threads are waiting for each other to complete so that they can complete complete the pause operation. The same behaviour was also be observed if a user attempted to pause the Activation Specification from the WebSphere Application Server administration console at the same time as the application server was notifying an Activation Specification thread of a failed delivery.
Problem conclusion
The WebSphere MQ Messaging Provider code has been modified so that only one thread may concurrently attempt to pause a particular Activation Specification. This allows all other threads associated with the Activation Specification to complete as normal before the pause operation takes place, avoiding the deadlock scenario. --------------------------------------------------------------- The fix is targeted for delivery in the following PTFs: v7.0 Platform Fix Pack 7.0.1.10 -------- -------------------- Windows U200352 AIX U853055 HP-UX (PA-RISC) U853082 HP-UX (Itanium) U853087 Solaris (SPARC) U853083 Solaris (x86-64) U853089 iSeries 7.0.1.10 Linux (x86) U853084 Linux (x86-64) U853088 Linux (zSeries) U853085 Linux (Power) U853086 zOS 7.0.1.10 v7.1 Platform Fix Pack 7.1.0.3 -------- -------------------- Windows 7.1.0.3 AIX 7.1.0.3 HP-UX (Itanium) 7.1.0.3 Solaris (SPARC) 7.1.0.3 Solaris (x86-64) 7.1.0.3 iSeries 7.1.0.3 Linux (x86) 7.1.0.3 Linux (x86-64) 7.1.0.3 Linux (zSeries) 7.1.0.3 Linux (Power) 7.1.0.3 zOS 7.1.0.3 Platform v7.5 -------- -------------------- Multiplatforms 7.5.0.1 The latest available maintenance can be obtained from 'WebSphere MQ Recommended Fixes' http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037 If the maintenance level is not yet available information on its planned availability can be found in 'WebSphere MQ Planned Maintenance Release Dates' http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
Temporary fix
Comments
APAR Information
APAR number
IV26267
Reported component name
WMQ LIN X86-64
Reported component ID
5724H7230
Reported release
700
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2012-08-15
Closed date
2012-08-30
Last modified date
2014-04-02
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
PI15088
Fix information
Fixed component name
WMQ LIN X86-64
Fixed component ID
5724H7230
Applicable component levels
R700 PSY
UP
[{"Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSFKSJ","label":"WebSphere MQ"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.0"}]
Document Information
Modified date:
03 October 2021