APAR status
Closed as program error.
Error description
The IBM MQ resource adapter has been deployed inside a JEE application server, and an MQ JMS Connection Factory has been defined that uses a client channel definition table (CCDT) which contains entries to connect to two different queue managers: - QM1 - QM2 in a queue manager group. The connection pool associated with the Connection Factory has a maximum size of 1. An enterprise application is installed into the application server and performs the following steps: - Try - Look up the MQ Connection Factory from the application server's JNDI repository. - Create a JMS Connection from the JMS Connection Factory. - Set an Exception Listener on the JMS Connection. - Create a non-transacted JMS Session from the JMS Connection. - Create a Message Producer for a Queue Destination from the JMS Session. - Create a Text Message from the JMS Session. - Use the Message Producer to send the Text Message to the Queue Destination. - Catch Exception: - Write details of the exception to the standard output stream (System.out). - Finally: - If a JMS Connection was created, then: - Remove the Exception Listener from the JMS Connection, by setting it to null. - Close the JMS Connection. - End if - End try The first time the application runs, the MQ resource adapter uses the information in the CCDT to create a JMS Connection to QM1, which is then returned to the application for use. When the application has finished processing, this JMS Connection is stored in the Connection Factory's connection pool, where it is available for reuse. During a subsequent run of the application, QM1 is stopped. When this happens, the application's Exception Listener is invoked with a JMSException containing MQ reason code 2009 (MQRC_CONNECTION_BROKEN). The application then closes the JMS Connection. From this point onwards, every time the application is run its ExceptionListener is driven with a JMSException containing reason code 2009 (MQRC_CONNECTION_BROKEN) whenever it tries to create a JMS Session. The information in the error message indicates that the MQ resource adapter is trying to communicate with QM1, which is unexpected. Because the Connection Factory has been configured with a CCDT that contains entries for two queue managers in a queue manager group, the resource adapter should be communicating with QM2 (which is up) rather than QM1 (which is down).
Local fix
Problem summary
**************************************************************** USERS AFFECTED: This issue affects users of: - The IBM MQ 9.3 JMS 2.0 resource adapter. - The IBM MQ 9.3 JMS 3.0 resource adapter for Jakarta Messaging. who have enterprise applications that: - Use a Connection Factory that has been configured to use either a Connection Name List or Client Channel Definition Table (CCDT) to connect to queue managers. - And call the method Connection.setExceptionListener(ExceptionListener) to register an ExceptionListener on JMS Managed Connections obtained from the Connection Factory's connection pool. - And: - Either run outside of the EJB Container. - Or run inside the EJB Container, and have set the Java system property: -Dcom.ibm.mq.connector.performJavaEEContainerChecks=false on the application server hosting the application. Platforms affected: MultiPlatform **************************************************************** PROBLEM DESCRIPTION: When performing outbound messaging within a JEE application server, enterprise applications can make use of connection pooling. The application server can be configured to provides a pool of JMS Managed Connections, one for each Connection Factory that has been defined. Whenever an application uses the Connection Factory to create a connection, the application server's Connection Manager will check the appropriate connection pool and perform one of the following actions: - If there is a free Managed Connection in the pool, the Connection Manager will mark it as being "in use", create a connection handle for it and pass that handle back to the application to use. - If all of the Managed Connections in the pool are currently "in use", and the number of Managed Connections is less than the maximum size of the pool, then the Connection Manager will: - Create a new Managed Connection. - Store it in the connection pool. - Mark it as being "in use". - Create a connection handle for it. - And pass the connection handle back to the application. - If all of the Managed Connections in the pool are "in use", and the number of Managed Connections is equal to the maximum size of the pool (ie. the pool is full), then the Connection Manager will block waiting for "in-use" connection to become available. After an application has finished with a connection and called Connection.close(), the Connection Manager will: - Close the connection handle that the application was using. - Check if the Managed Connection has been marked as "Stale". - If it has, then: - Destroy the Managed Connection, and remove it from the connection pool. - Else: - Change the state of the Managed Connection from "in-use" to "free". This allows it to be resued later. - End if The IBM MQ JMS 2.0 resource adapter and JMS 3.0 resource adapter for Jakarta Messaging provide their own implementations of JMS Managed Connections and connection handles: "com.ibm.mq.connector.outbound.ManagedConnectionImpl": These objects represent Managed Connections. These objects contain an actual JMS Connection, along with some state information such as whether the Managed Connection is involved in a transaction. The Managed Connection also registers itself as an ExceptionListener on the underlying JMS Connection. "com.ibm.mq.connector.outbound.ConnectionWrapper": These objects are the connection handles. These contain references to the Managed Connection, and the underlying JMS Connection stored in the Managed Connection. Here is a diagram that shows this relationship: ConnectionWrapper: |->contains ManagedConnectionImpl | |-> contains JMS Connection | |-> ExceptionListener: ManagedConnectionImpl | |-> contains JMS Connection | |-> ExceptionListener: ManagedConnectionImpl If an connection error occurs on the JMS Connection the MQ resource adapter drives that Connection's ExceptionListener, which is the ManagedConnectionImpl. The ManagedConnectionImpl.onException() method then fires a CONNECTION_ERROR_OCCURRED event to the Connection Manager, informing it can no longer be used. When the Connection Manager receives this event, it marks the connection as "Stale". As mentioned above, this means that when the application closes its connection handle by calling Connection.close(), the Connection Manager takes the necessary steps to clean up and destroy the Managed Connection, rather than returning it to the connection pool - this ensures that the broken connection cannot be reused. It will also potentially remove all of the other free connections in the pool, depending on the Purge Policy that has been defined on the connection pool. Now, when the issue reported in this APAR occurred, an application had: - Called ConnectionFactory.createConnection() to get a ConnectionWrapper representing a handle to a Managed Connection from a Connection Factory's connection pool. - And had then called Connection.setExceptionListener(ExceptionListener) to register its own Exception Listener. This resulted in the ConnectionWrapper.setExceptionListener(ExceptionListener) method being called, which set the application's ExceptionListener on the underlying JMS Connection, like this: ConnectionWrapper: |->contains ManagedConnectionImpl | |-> contains JMS Connection | |-> ExceptionListener: Application's Exception Listener | |-> contains JMS Connection | |-> ExceptionListener: Application's Exception Listener This meant that if a connection error occurred on the JMS Exception, the application's ExceptionListener was called rather than the ManagedConnectionImpl.onException(JMSException) method. As a result, the Connection Manager did not receive a CONNECTION_ERROR_OCCURRED event for the Managed Connection, and so was unaware that it was broken and could no longer be used. This caused it to be left in the connection pool. The next time the application called Connection.createConnection(), it could get back a ConnectionWrapper representing a connection handle to the broken Managed Connection. If this happened, then when the application tried to use this to create a JMS Session, the attempt would fail with a JMSException containing reason code 2009 (MQRC_CONNECTION_BROKEN). If the application was using a Connection Factory that had been configured to use: - Either a Connection Name List - Or a Client Channel Definition Table (CCDT) then this would result in the application being unable to create a JMS Connection to a queue manager, even though other queue managers mentioned in either the Connection Name List or entries in the CCDT were available.
Problem conclusion
To resolve this issue, the IBM MQ JMS 2.0 resource adapter and IBM MQ JMS 3.0 resource adapter for Jakarta Messaging have been updated so that: - If an enterprise application has obtained a connection handle to a Managed Connection in a connection pool. - And registers its own ExceptionListener with the connection handle. then if a connection error occurs on the underlying JMS connection the "com.ibm.mq.connector.outbound.ManagedConnectionImpl" object will: - Send a CONNECTION_ERROR_OCCURRED event to the Connection Manager, to inform it that the JMS Connection has been broken. - And then drive the application's ExceptionListener. This ensures that the application is notified of the connection error (via its ExceptionListener), and that the Managed Connection is destroyed rather than being returned to the connection pool when the connection handle is closed. --------------------------------------------------------------- The fix is targeted for delivery in the following PTFs: Version Maintenance Level v9.3 LTS 9.3.0.5 v9.x CD 9.3.2 The latest available maintenance can be obtained from 'WebSphere MQ Recommended Fixes' http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037 If the maintenance level is not yet available information on its planned availability can be found in 'WebSphere MQ Planned Maintenance Release Dates' http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309 ---------------------------------------------------------------
Temporary fix
Comments
APAR Information
APAR number
IT42294
Reported component name
MQ BASE V9.2
Reported component ID
5724H7281
Reported release
923
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2022-10-19
Closed date
2022-12-01
Last modified date
2022-12-01
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
MQ BASE V9.2
Fixed component ID
5724H7281
Applicable component levels
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSYHRD","label":"IBM MQ"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"923","Line of Business":{"code":"LOB45","label":"Automation"}}]
Document Information
Modified date:
02 December 2022