IBM Support

IBM MQ Troubleshooting Common TLS SSL Errors

Troubleshooting


Problem

This document is intended to help diagnose IBM MQ Server and Client TLS SSL setup errors.
It lists most of the common configuration errors that can cause an SSL connection to fail, and gives the course of action to resolve the problem.
In each case, the error can be diagnosed by a combination of the error seen in the IBM MQ system acting as the client's error logs and system acting as the server's error logs.
All cases here assume that 2-way authentication is being attempted (SSLCAUTH set to REQUIRED on the queue manager's SVRCONN channel). This is the default, and the errors are similar for 1-way authentication (SSLCAUTH set to OPTIONAL). 
.
For IBM MQ Java/JMS troubleshooting this document is applicable:
https://www.ibm.com/support/pages/node/709225
Troubleshooting IBM MQ Java/JMS SSL TLS Configurations

Symptom

Diagnosing The Problem

All of the following errors are in either the IBM MQ server and/or client AMQERR0*.log files.
They can be located in locations specified here:
These contain the AMQxxxx errors as seen below

Back to top




 

Resolving The Problem

AMQ9616 AMQ9616E: The CipherSpec '<CIPHERSPEC>’ proposed by remote host 'XXX.XXX.XXX.XXX' is not enabled.
QMGR ERROR LOG
EXPLANATION:
The SSL or TLS subsystem at the server end of a channel has been configured in such a way that it has rejected the CipherSpec proposed by an SSL or TLS client. This rejection occurred during the secure socket handshake (i.e. it happened before the proposed CipherSpec was compared with the CipherSpec in the server channel definition). This error most commonly occurs when the choice of acceptable CipherSpecs has been limited in one of the following ways:(a) The server queue manager SSLFipsRequired attribute is set to YES and the channel is using a CipherSpec which is not FIPS-certified on the server.
(b) The server queue manager EncryptionPolicySuiteB attribute has been set to a value other than NONE and the channel is using a CipherSpec which does not meet the server's configured Suite B security level.
(c) The protocol used by the channel has been deprecated. Note that IBM may need to deprecate a protocol via product maintenance in response to a security vulnerability, for example SSLv3 and TLSv1 have been deprecated. Continued use of SSLv3 and/or TLSv1 protocols is not recommended but may be enabled by setting environment variable AMQ_SSL_V3_ENABLE=TRUE or AMQ_TLS_V1_ENABLE=TRUE as required.
(d) The requested CipherSpec has been deprecated. Note that IBM may need to deprecate a CipherSpec via product maintenance in response to a security vulnerability, for example RC4_MD5_US has been deprecated. Continued use of deprecated CipherSpecs is not recommended but may be enabled by setting environment variable AMQ_SSL_WEAK_CIPHER_ENABLE=<CipherSpec>. For example,AMQ_SSL_WEAK_CIPHER_ENABLE=RC4_MD5_US
ACTION: Analyze why the proposed CipherSpec was not enabled on the server. Alter the client CipherSpec, or reconfigure the server to accept the original clientCipherSpec. Restart the channel.
SOLUTION
IBM MQ CipherSpec Table:
IBM MQ Deprecated CipherSpecs:
1) Ensure the qm.ini value for SSLFipsRequired matches your client’s cipher FIP’s value based on the IBM MQ Cipher table.
2) Ensure you that your QMGR’s SUITEB property matches your client’s SuiteB value based on the IBM MQ Cipher Table
3) Ensure the client’s cipher is not using a cipher listed in the deprecated cipher list.
4) If you want to allow the client to use a deprecated cipher ensure the following are set in the qm.ini:
SSL:
AllowTLSV1=Y
AllowWeakCipherSpec=<DESIRED CIPHER>
Or you have set the following environment variable:
export AMQ_SSL_WEAK_CIPHER_ENABLE = <DESIRED CIPHER>
5) Ensure your channel’s cipher spec matches the cipher that the client is proposing.

Back to top

 

AMQ9633 AMQ9633E: Bad SSL certificate for channel '????'.
QMGR ERROR LOG
EXPLANATION:
A certificate encountered during SSL handshaking is regarded as bad for one of the following reasons: (a) it was formatted incorrectly and could not be validated
(b) it was formatted correctly but failed validation against the Certification Authority (CA) root and other certificates held on the local system
(c) it was found in a Certification Revocation List (CRL) on an LDAP server
(d) a CRL was specified but the CRL could not be found on the LDAP server
(e) an OCSP responder has indicated that it is revoked The channel is '????'; in some cases its name cannot be determined and so is shown as '????'. The remote host is '<HOSTNAME>’. The channel did not start. The details of the certificate which could not be validated are '[Class=]GSKVALMethod::X509[Issuer=]CN=E<CN NAME>,OU=<SOME OU>C=US [#=]<CERT SERIAL NUMBER> [Subject=]CN=<SOMECN>,O'.
The certificate validation error was #######.
SOLUTION
The number provided by the certificate validation error will provide information and further troubleshooting steps. This document contains more information: Diagnostic improvements to IBM MQ SSL/TLS error AMQ9633

Back to top



AMQ9637 AMQ9637E: Channel is lacking a certificate.
QMGR ERROR LOG
EXPLANATION:
The channel is lacking a certificate to use for the SSL handshake. The channel name is '<insert one>' (if '????' it is unknown at this stage in the SSL processing).The remote host is '<insert three>'. The channel did not start.
ACTION:
Make sure the appropriate certificates are correctly configured in the key repositories for both ends of the channel.
SOLUTION
Potential Causes:
1. File permission issue on the keystore or truststore files.
2. Missing keystore truststore or keystore or truststore not found.
3. Missing personal certificate in the keystore.
4. Incorrect label name.
5. Incorrect signature algorithm.
6. Incomplete cert chain if using a CA signed certificate.
Solution 1:
Verify the file permissions on the keystore files, ensure that the user running the IBM MQ processes or application has read access to these files. On Unix variations, usually ensuring the mqm group has read access is adequate.
Solution2:
Confirm these properties are set correctly.
A queue manager and non-Java clients use the SSLKEYR property to locate its keystore.
Ensure that the SSLKEYR is the stem of the keystore location.
IE: If the keystore is located at /var/mqm/qmgrs/EXAMPLEQM/ssl/key.kdb
The stem is: /var/mqm/qmgrs/EXAMPLEQM/ssl/key
Java clients can use JVM system properties
-Djavax.net.ssl.trustStore=
-Djavax.net.ssl.keyStore=
Solution 3:
Confirm the personal certificate exists.
To list personal certificates:
runmqckm -cert -list personal -db (keystore filename)
versus the following, which lists signer certificates:
runmqckm -cert -list CA -db (keystore filename)
Solution 4:
The default certificate label name for a IBM MQ client is ibmwebspheremq followed by the userid, all in lowercase.
Solution 5:
Needs to be an RSA signature algorithm.
Solution 6:
Validate the certificate chain based on this article: IBM MQ Certificate Authority Chain Information

Back to top



AMQ9642 AMQ9642E: No SSL or TLS certificate for channel '<insert one>'.
QMGR ERROR LOG
Explanation
The channel <insert_3> did not supply a certificate to use during SSL or TLS handshaking, but a certificate is required by the remote queue manager.
The remote host is <insert_4>.
The channel did not start.
Response
Ensure that the key repository of the local queue manager or IBM MQ client contains a certificate which is associated with the queue manager or client. If you have configured a certificate label, check that the certificate exists. Alternatively, if appropriate, change the remote channel definition so that its SSLCAUTH attribute is set to OPTIONAL and it has no SSLPEER value set.
SOLUTION
1) Personal certificates are searched for by the label name so if a certificate label is configured, check to ensure that it is specified correctly. By default, the following certificate label names are used:
  • Queue manager: ibmwebspheremq followed by lowercase queue manager name.
  • IBM MQ client application: ibmwebspheremq followed by lowercase userid of user running the IBM MQ application.
On the queue manager side, check the CERTLABL attribute on the TLS channel. If the CERTLABL attribute for the TLS channel is blank, the certificate is determined by the queue manager’s CERTLABL attribute. If the CERTLABL attribute for the queue manager is blank, then the default certificate label name is used. If you have a CERTLABL on the QMGR and the Channel ensure that you can still validate the QMGR’s CERTLABL because it is still validated. Once that happens then the channel’s CERTLABL will be validated and passed to the other end of the connection.
You do not need to run the REFRESH SECURITY TYPE(SSL) command when you make any changes to CERTLABL on a channel. However, you must run a REFRESH SECURITY TYPE(SSL) command when you make any changes to CERTLABL on the queue manager.
For IBM MQ client application side, the certificate label can be specified in the following ways:
  • The CertificateLabel attribute for the SSL Stanza in mqclient.ini file.
  • The CertificateLabel field for the MQSCO structure on an MQCONNX call.
  • Environment variable MQCERTLABL.
  • CERTLABL in the CLNTCONN channel definition
For more information on certificate label, refer to the following documentation:
2) If the CertificateLabel attribute on the channel is correct, check to ensure that the personal certificate exists in the key repository for the queue manager.
If the certificate exists, but was added with '-cert -add' for the runmqakm or runmqckm command, this certificate is NOT added as personal certificate. This adds the PUBLIC part of the certificate as a signer certificate which does not include the private key. You can list the personal certificates in the keystore with the following command to verify that it is indeed a personal certificate:
runmqckm -cert -list personal -db KEYSTORE_FILENAME -pw KEYSTORE_PASSWORD
You can move personal certificates by using '-cert -export' and '-cert -import'. For more details on importing or exporting certificates:
3) If SSLCAUTH is set to REQUIRED on the channel for the SSL Server (RCVR or SVRCONN) then this means that the SSL Client (IBM MQ client application or remote sender queue manager) must have and present its personal certificate to the IBM MQ Server for validation. In this case, check that the SSL Client has a valid personal certificate.

Back to top



AMQ9660 AMQ9660E: SSL key repository: password stash file absent or unusable.
QMGR ERROR LOG
EXPLANATION:
The SSL key repository cannot be used because IBM MQ cannot obtain a password to access it. Reasons giving rise to this error include:
(a) the key database file and password stash file are not present in the location configured for the key repository,
(b) the key database file exists in the correct place but that no password stash file has been created for it,
(c) the files are present in the correct place but the userid under which IBM MQ is running does not have permission to read them,
(d) one or both of the files are corrupt.
SOLUTION
Common causes are the following:
1.Keystore file locations are defined incorrectly or the files are not present.
2.The userid running the program, does not have permission to read the keystore files
3..sth file missing or stash file format unreadable
Solutions
For 1) keystore location correct, and files exist
For a queue manager: SSLKEYR defines the location of the keystore files. SSLKEYR includes the base file name of the keystore files. It does not include the file extensions (.kdb, .sth, .rdb, etc)
Ensure the keystore files exist and are in the correct location.
For a IBM MQ client app, the keystore files locations can use the MQSSLKEYR environment variable, it needs to include the base filename, but not the extension. IE: MQSSLKEYR=E:\CLNTSSL\ssl\key (assuming key.kdb is your keystore db file)
For 2) keystore file permission
Ensure the userid running the IBM MQ processes (usually mqm on Unix) has permission to read these files. Many times the files are created with some other userid, and the mqm user might not have access to read the files. This can result in the AMQ9660 error.
For 3) .sth file missing or unreadable
Check the correct location for the keystore files and the .sth file. The .sth file should be created when the keystore was created using the –stash option. If the stash file was never created, you can use the runmqckm -keydb -stashpw command to create a stash file, see the documentation at:
If a stash file exists, and is corrupt, or you do not know the password and you are not able to access the keystore, you will need to create a new keystore and repopulate it. When creating the new keystore, make sure you use the -stash option, or if using the iKeyman, select stash the password to a file per:
One other reason the .sth file may be unreadable by IBM MQ, is if it was created with a newer version of IBM MQ. There has been change in the formats of password stash files. Depending on the version of GSKit where the keystore was created, see: In IBM MQ v 8.0.0.6 and higher and IBM MQ v 9.0.0.1 and higher the internal format of GSKit keystore stash files has changed.

Back to top



AMQ9663 AMQ9663E: An invalid SSL certificate was received from the remote system.
QMGR ERROR LOG
EXPLANATION:
An SSL certificate received from the remote system failed validation checks on its signature. The channel is <insert_3>; in some cases its name cannot be determined and so is shown as '????'. The channel did not start.
SOLUTION
List the certificates contained in the keystore and make sure all CA signer certs are present which are required for validating the remote certificate.
Command:
runmqakm –cert –list –db <keystoreFile> -stashed
If any CA cert is missing, added to the keystore:
-runmqakm –cert –add –db <keystoreFile> -pw <password> -label <certLabel> -file <caCertFile>
Reference Technotes:

Back to top



AMQ9665 AMQ9665: SSL connection closed by remote end of channel '<CHANNELNAME>'.
QMGR ERROR LOG
EXPLANATION:
The SSL or TLS connection was closed by the remote host 'host(<HOST><(PORT)>' during the secure socket handshake. The channel is' <CHANNELNAME> '; in some cases its name cannot be determined and so is shown as'????'.
The channel did not start.
ACTION:
Check the remote end of the channel for SSL and TLS errors. Fix them and restart the channel.
SOLUTION
This is a generic message from the opposite end of the connection that determined the SSL connection or handshake should have been stopped. Look for the error on the other end of the connection.

Back to top



AMQ9716 AMQ9716E: Remote SSL certificate revocation status check failed for channel
QMGR ERROR LOG
EXPLANATION:
WebSphere MQ failed to determine the revocation status of the remote SSL certificate for one of the following reasons: (a) The channel was unable to contact any of the CRL servers or OCSP responders for the certificate.
(b) None of the OCSP responders contacted knows the revocation status of the certificate.
(c) An OCSP response was received, but the digital signature of the response could not be verified. The details of the certificate in question are '[Class=]GSKVALMethod::X509[Issuer=]<ISSUER CN>[#=]<CERT SERIAL #>[Subject=]<SUBJECT CN>[Class=]GSKVALMethod::X5'.
The channel name is '<CHANNEL NAME>'.
In some cases the channel name cannot be determined and so is shown as '????'. The channel did not start.
SOLUTION
The QMGR can’t access the OCSP or OCSP has determined the certificate is revoked.
Solutions:
1) Ensure your IBM MQ server has access to any CRL or OCSP servers. IBM MQ allows you to configure a SSLHTTPProxyName, if needed, which can be used to configure access to these servers, depending on your network configuration.
2) You can disable the OCSP/CRL checking for your queue manager by adding an SSL stanza to your qm.ini, like the following:(for IBM MQ client, add this to the mqclient.ini):
SSL:
OCSPAuthentication=OPTIONAL
OCSPCheckExtensions=NO
CDPCheckExtensions=NO

Back to top

++ Additional Information:
tags: "MQ SSL"; "MQ TLS"; MQSSL; MQTLS
+++ end +++

Document Location

Worldwide

[{"Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSYHRD","label":"IBM MQ"},"ARM Category":[{"code":"a8m0z00000008JwAAI","label":"Security-\u003ETLS (SSL)"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)"}]

Document Information

Modified date:
26 October 2023

UID

ibm16359069

Page Feedback