IBM Support

JMSWMQ0018 due to RC=2397 AMQ9771 SSL handshake failed for channel connecting from WebSphere Application Server to IBM MQ

Troubleshooting


Problem

WebSphere Application Server (WSAS or WAS) tried to connect to IBM MQ by using a SVRCONN client channel.  The WSAS log has an error:

[Date Time] 000002b4 SendReceiveJM E ERROR : FAILED TO CREATE MQ QUEUE CONNECTION

com.ibm.msg.client.jms.DetailedJMSException: JMSWMQ0018: Failed to connect to queue manager 'MQW1' with connection mode 'Client' and host name 'ipaddr(port)'.

Check the queue manager is started and if running in client mode, check there is a listener running. Please see the linked exception for more information.

...

at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1892)

Caused by: com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2397' ('MQRC_JSSE_ERROR').

at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:204)

... 59 more

Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2397;AMQ9204: Connection to host 'ipaddr(port)' rejected. [1=com.ibm.mq.jmqi.JmqiException[CC=2;RC=2397;AMQ9771: SSL handshake failed. [1=javax.net.ssl.SSLException[Received fatal alert: bad_record_mac],3=ipinfo,4=SSLSocket.startHandshake,5=default]],3=ipinfo,5=RemoteTCPConnection.protocolConnect]

...

Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2397;AMQ9771: SSL handshake failed. [1=javax.net.ssl.SSLException[Received fatal alert: bad_record_mac],3=ipinfo),4=SSLSocket.startHandshake,5=default]

at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.protocolConnect(RemoteTCPConnection.java:1318)

...

Caused by: javax.net.ssl.SSLException: Received fatal alert: bad_record_mac

...

The queue manager was on the z/OS platform, where the CHIN job log had an error message CSQX620E with RC=456. 456 means that ICSF callable services returned an error. The comparable message for a distributed (multiplatform) queue manager is AMQ9620E.

Cause

Both https://www.ibm.com/support/pages/node/1084575 and PH29466 mention the CSQX620E RC456 in the context of elliptic curve cipher specs, which is what ECDHE_RSA_AES_256_GCM_SHA384 is. You don't intend to use an ECDHE cipher spec.

A TCP/IP packet trace showed that 009D - TLS_RSA_WITH_AES_256_GCM_SHA384 was negotiated in the handshake when the client sent a long list of ciphers. The SVRCONN channel was set with 003D - TLS_RSA_WITH_AES_256_CBC_SHA256, but it seemed to be ignored. The negotiation was handled by the first cipher agreed on. The mainframe alert packet returned x'0214' (Bad_Record_Mac).
009D was removed from the list of CIPHERs to send. A new packet trace showed that 45 2-byte cipher suites were passed in the Client Hello from the WSAS side:
            c0 2c c0 2b c0 30 c0 2f 00 9f 00 a3
00 9e 00 a2 c0 24 c0 28 c0 23 c0 27 00 6b 00 6a
00 67 00 40 c0 2e c0 32 c0 2d c0 31 c0 26 c0 2a
c0 25 c0 29 c0 0a c0 14 c0 09 c0 13 00 39 00 38
00 33 00 32 c0 05 c0 0f c0 04 c0 0e 00 9c 00 3d
00 3c 00 35 00 2f cc a9 cc a8 cc aa 00 ff
Table 1 at https://www.ibm.com/docs/en/ibm-mq/9.3?topic=cipherspecs-cipherspec-order-in-tls-handshake shows the default order of Cipher Specs used by IBM MQ. One of the points there is "The TLS server’s list order has a higher priority than the TLS client." The first match in that default order compared to the Client Hello list is C030 ECDHE_RSA_AES_256_GCM_SHA384. C030 and 009D, which was removed, are higher in the list than the intended 003D.

Resolving The Problem

Modify the "Selected Ciphers" list in the WSAS configuration as described at https://www.ibm.com/docs/en/was-zos/8.5.5?topic=configuration-quality-protection-qop-settings. https://www.ibm.com/docs/en/linux-on-systems?topic=setup-selecting-websphere-application-server-cipher-suites has a nice picture of that WSAS console page.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSYHRD","label":"IBM MQ"},"ARM Category":[{"code":"a8m0z00000008MzAAI","label":"Security"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSWN2J","label":"IBM MQ for z\/OS"},"ARM Category":[{"code":"a8m0z00000008MzAAI","label":"Security"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"ARM Category":[{"code":"a8m50000000CcyMAAS","label":"WebSphere Application Server traditional-All Platforms-\u003ESecurity-\u003ESSL"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
21 December 2023

UID

ibm17100976

Page Feedback