Troubleshooting
Problem
HTTP request of Websphere application fails when SSL is enabled on the Websphere Application Server with error: Failed in r_gsk_secure-Soc_init: Peer not recognized or badly formatted message received.(gsk rc = 410)
Symptom
Failed in r_gsk_secure-Soc_init: Peer not recognized or badly formatted message received.(gsk rc = 410)
Cause
The version(s) of SSL/TLS that the Websphere Application Server is trying to negotiate is not supported on the IBM i operating system.
Environment
Websphere Application Server configured to use SSL
Diagnosing The Problem
Enable the JVM custom property javax.net.debug. Here are the instructions:
Follow these steps to set up the tracing and recreate the issue.
- In the Administrative Console set the javax.net.debug system property using one of the following options, depending on where the SSL issue is occurring:
- For tracing an Application server, select the following: Servers > Server Types > WebSphere Application Servers > server_name > Expand Java and Process Management (under Server Infrastructure) - >Process definition > Java Virtual Machine > Custom properties > New...
- For tracing a Deployment Manager, select the following: System Administration > Deployment manager > Expand Java and Process Management (under Server Infrastructure) >Process definition > Java Virtual Machine > Custom properties > New...
- For tracing a Nodeagent, select the following: System Administration > Node agents > (pick a nodeagent) > Expand Java and Process Management (under Server Infrastructure) >Process definition > Java Virtual Machine > Custom properties > New...
Type the following:
Name: javax.net.debug
Value: true
Note: Support may request this value be set to ssl:handshake to limit the volume of trace output.
Click Apply, and Save your changes to the master configuration.
Expand Troubleshooting > Logs and trace > server_name.
Select Diagnostic Trace. Set the Maximum Number of Historical Files to 20.
Click Apply, then select Change log detail levels.
Set the trace specification string to:
*=info : SSL=all
Click OK, then OK.
Select JVM Logs. Ensure under System.out under Installed Application Output that the
Show application print statements box is checked.
Click OK, and Save to the master configuration.
Stop the server(s) and backup/clear the logs directory for the server(s) you are tracing and the FFDC directory as well.
The trace.log will show similar to the following:
[11/10/16 11:12:48:636 NZDT] 00000067 O UOW= source=SystemOut org=IBM
prod=WebSphere component=Application Server thread=[WebContainer : 0]
WebContainer : 0, READ: TLSv1.1 Alert, length = 2
[11/10/16 11:12:48:636 NZDT] 00000017 O UOW= source=SystemOut org=IBM
prod=WebSphere component=Application Server thread=[Finalizer thread]
Finalizer thread, called closeInternal(true)
[11/10/16 11:12:48:637 NZDT] 00000067 O UOW= source=SystemOut org=IBM
prod=WebSphere component=Application Server thread=[WebContainer : 0]
WebContainer : 0, RECV TLSv1 ALERT: fatal, protocol_version
The http_plugin.log shows the following:
ERROR: lib_stream: openStream: Failed in r_gsk_secure_soc_init: Peer not
recognized or badly formatted message received.(gsk rc = 410)
In this example, the Websphere Application Server was trying to negotiate TLSV1 and the IBM i operating system was set to only support TLSV1.1 and TLSV1.2. The operating system is configured by the system value QSSLPCL. A value of *OPSYS means support whatever that version of OS supports. They had it set to the following:
*TLSV1.2
*TLSV1.1
This was on a V7R1 system so in this case *SSLV3 and *TLSV1 were excluded.
See the following links in regards to the SSLV3 security vulernabilities:
http://www-01.ibm.com/support/docview.wss?uid=swg21687173
http://www.ibm.com/support/docview.wss?uid=nas8N1020431
http://www.ibm.com/support/docview.wss?uid=nas8N1020451
Resolving The Problem
Add the requested SSL/TLS version to the system value QSSLPCL or change the list of supported versions on Websphere Application Server to include a SSL/TLS version on the IBM i OS.
Websphere Application Server can be changed by doing the following (example is WAS 8.5):
- In the Integrated Solutions Console, expand Security on the left. Now click on SSL certificates and key management.
- Click on SSL Configurations on the right.
- Choose a configuration
- Click on Quality of protection (QoP) settings
- Choose one from the Protocol drop down box
- Click Apply/Save.
Was this topic helpful?
Document Information
Modified date:
18 December 2019
UID
nas8N1021674