Troubleshooting
Problem
Since Planning Analytics 2.0.9.5, a custom SSL/TLS chain of certificates that was previously working, is not working anymore with TM1 Admin Server.
Despite "IBM Cognos TM1 Admin Server x64" service seems to be "Running", no TM1 Server is able to register against it.
In the tm1admsrv*.log files, the following error appears:
TM1.TM1AdmSvr Failed to start Admin server
After we set DEBUG in tm1admsrv-log.properties, we are getting more information:
TM1.Comm.SSL.Cert Unrecognized cert data type: 688, Value:OCSP unknown access method : PKIX_AD_CA_Issuer (1.3.6.1.5.5.7.48.2)
The following error is occurs when validating the server certificate:
Command (from \tm1_64\bin64):
gsk8capicmd_64 -cert -validate -db .\ssl\ibmtm1.kdb -stashed -label ibmtm1_server
Result:
GSKKM_ERR_BAD_ACCEPTABLE_POLICIES
gsk8capicmd_64 -cert -validate -db .\ssl\ibmtm1.kdb -stashed -label ibmtm1_server
Result:
GSKKM_ERR_BAD_ACCEPTABLE_POLICIES
Cause
Since Planning Analytics 2.0.9.5, the chain of certificates must conform to RFC 5280 standards.
Detailed explanation:
-TM1 data tier is using GSKit to handle certificates since Planning Analytics 2.0.
-For version 2.0.9.5, Product Management followed GSKit team recommendations in order to harden security.
The recommendation was to use the "vaccinate" option in GSKit. Product Management decided to comply with this recommendation in order to give extra protection to all our customers. This change is hardcoded in the product, so it cannot be changed. There will be no option to get back to the previous behavior as it would put all customers at risk.
-The use of "vaccinate" option forces the certificate chain of trust to conform to RFC-5280 standards (which was created in 2008, so it is an old recommendation already, and this is about time to start conforming to it). The following link is detailing all the requirements that a chain of certificates must conform to:
-The vaccination also sets GSK_STRICT_EE_KEYUSAGE_CHECK to True (GSK_TRUE). Here are the effects of setting this parameter:
-> Negotiated cipher suite TLS_RSA_XXXX check that we have keyEncipherment keyUsage value
-> Negotiated cipher suite TLS_ECDHE_XXXX check that we have digitalSignature keyUsage value
-> Negotiated cipher suite TLS_RSA_XXXX check that we have keyEncipherment keyUsage value
-> Negotiated cipher suite TLS_ECDHE_XXXX check that we have digitalSignature keyUsage value
Diagnosing The Problem
WARNING: solving this kind of problem might require some time. In case of urgency, work around the issue by applying the following document, while IBM Support and your IT security team are trying to solve your actual issue:
Resolving The Problem
Make sure that the certificate chain of trust conforms to RFC 5280 standards.
In particular (common issues):
-> Verify the server certificate does include at least keyEncipherment and digitalSignature in the keyUsage field
-> If "Certificate Policies" field exists, then verify it is the same for the server certificate and the intermediate certificate that signed it.
Once the chain of certificates does conform to the new requirements, and once custom SSL was correctly set up in Planning Analytics installation, we might still see some "Failed to start Admin server" is some tm1admsrv*.log files, whereas everything appears to work correctly.
Indeed, there is a side effect of this GSKit "vaccination" that we apply during TM1 Admin Server startup: there can be several more minutes between the moment "IBM Cognos TM1 Admin Server x64" service is started, and the moment it is available to receive registering requests from TM1 Servers. The behavior is normal and expected since the vaccination generates more work.
During that time, if a TM1 Server is started, it is not able to connect to the existing TM1 Admin Server service (yet). When that happens, the TM1 Server itself tries to run its own TM1 Admin Server (tm1admsrv.exe), which is not the same as the TM1 Admin Server service (tm1admsd.exe).
However, because there is already a running TM1 Admin Server service, this other TM1 Admin Server fails to start: it explains the "Failed to start Admin server" errors we might see in the tm1admsrv*.log files since version 2.0.9.5.
In brief: this error message in the TM1 Admin Server log files does not mean there is a problem with the TM1 Admin Server service: it means the service is not ready yet, so we need to wait few more minutes.
Indeed, there is a side effect of this GSKit "vaccination" that we apply during TM1 Admin Server startup: there can be several more minutes between the moment "IBM Cognos TM1 Admin Server x64" service is started, and the moment it is available to receive registering requests from TM1 Servers. The behavior is normal and expected since the vaccination generates more work.
During that time, if a TM1 Server is started, it is not able to connect to the existing TM1 Admin Server service (yet). When that happens, the TM1 Server itself tries to run its own TM1 Admin Server (tm1admsrv.exe), which is not the same as the TM1 Admin Server service (tm1admsd.exe).
However, because there is already a running TM1 Admin Server service, this other TM1 Admin Server fails to start: it explains the "Failed to start Admin server" errors we might see in the tm1admsrv*.log files since version 2.0.9.5.
In brief: this error message in the TM1 Admin Server log files does not mean there is a problem with the TM1 Admin Server service: it means the service is not ready yet, so we need to wait few more minutes.
If we are unable to determine what part of the chain of certificates does not comply with the RFC 5280 standards, then we can run the following command in order to generate a trace file:
gsk8capicmd_64.exe -cert -validate -label ibmtm1_server -db ".\ssl\ibmtm1.kdb" -stashed -trace output.trc
Send the output.trc file to IBM Support, then Support can transfer it to Development for analysis.
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB10","label":"Data and AI"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSCTEW","label":"IBM Planning Analytics Local"},"ARM Category":[{"code":"a8m50000000KzK7AAK","label":"Planning Analytics-\u003ESecurity-\u003ESSL"}],"ARM Case Number":"","Platform":[{"code":"PF033","label":"Windows"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
31 October 2022
UID
ibm16608188