Troubleshooting
Problem
A previously working instance of IBM Resilient Circuits fails to connect to IBM Resilient with an error such as "certificate verify failed."
Symptom
Actions and workflows may cease to work and customised code running in IBM Resilient Circuits may not be acting on incidents.
Checking the app.log of IBM Resilient Circuits shows the following errors
Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))
packages/requests/adapters.py", line 514, in send raise SSLError(e, request=request) SSLError: HTTPSConnectionPool(host='xx.xx.com', port=443): Max retries exceeded with url: /xx/xx/?key=xxx=xx.xx.xx.xx (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))
Cause
The SSL certificate used by IBM Resilient, or part of the chain such as the intermediary or root certificates have expired.
Diagnosing The Problem
Check the content of the IBM Resilient server's keystore where the SSL certificate is stored to see whether it has expired.
sudo keytool -list -v -keystore /crypt/certs/keystore -storepass "$(sudo resutil keyvaultget -name keystore)"
Also, run the following command to return and check for "certificate has expired" some where in the chain.
openssl s_client -connect {resilient_server_hostname}:65001 -showcerts
You can copy out the certificates from the last command, including the BEGIN and END lines, into a text file saved as .cer on your computer and inspect the certificate using the tools your operating system provides.
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
Resolving The Problem
If the IBM Resilient server's SSL certificate, or one of the certificates making up the chain, has expired, either renew the SSL certificate or if you are using self-signed certificates generate a new self-signed certificate.
For self-signed SSL certificates use How to generate a new self-signed SSL certificate for use with IBM Resilient Circuits.
To create a Certificate Signing Request which will be signed by a Certificate Authority, please see the Knowledge Center for further information.
Ensure you run sudo systemctl restart resilient-messaging after installing a new SSL certificate in IBM Resilient.
Once IBM Resilient is using a valid SSL certificate, download that SSL certificate so that IBM Resilient Circuits can use that SSL certificate to verify the connection.
If IBM Resilient has a single certificate in the chain then it is likely to be using a self-signed certificate.
openssl s_client -connect {resilient_server_hostname}:443 -showcerts < /dev/null 2> /dev/null | openssl x509 -outform PEM > cacerts.pem
If the SSL certificate uses a chain of SSL certificates.
keytool -printcert -rfc -sslserver {resilient_server_hostname}:443 > cacerts.pem
If the server where IBM Resilient Circuits is installed does not have the keytool application installed then output all the SSL certificates in the chain.
openssl s_client -connect {resilient_server_hostname}:443 -showcerts
Copy out all the BEGIN and END lines, including the base64 ASCII, in the order presented into a new file to make the chain.
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
Update app.config to point to the new file.
cafile=/path/to/cacerts.pem
Save the app.config
Restart the IBM Resilient Circuits.
sudo systemctl restart resilient_circuits
Related Information
Document Location
Worldwide
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSIP9Q","label":"IBM Security SOAR"},"ARM Category":[{"code":"a8m0z0000008ZtGAAU","label":"Integrations->Resilient Circuits"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)","Line of Business":{"code":"LOB24","label":"Security Software"}}]
Was this topic helpful?
Document Information
Modified date:
19 April 2021
UID
ibm16251657