Replacing the foundational services endpoint certificate

You can replace the certificate for the foundational services endpoint, which is the external endpoint that is used for access from outside the cluster to the console and API.

Replacing the foundational services endpoint certificate (version 3.6.4 and later)

Replacing the foundational services endpoint certificate (version 3.6.4 and later)

IBM Cloud Pak foundational services includes a self-signed TLS certificate that is used for enabling HTTPS connections of the foundational services endpoint, which is the external endpoint that is used for access from outside the cluster to the console and API. However, this certificate is not trusted by all HTTPS clients. Therefore, it is recommended that you replace the self-signed certificate with your own certificate.

Note: This task only includes updating the cp-console endpoint certificate. For updating {{site.data.keyword.cloud_pak_data_notm}} console with the zen-cpp installed, refer to the respective IBM Cloud Pak documentation to update the TLS certificate for enabling HTTPS connections. For more information on endpoint instructions, see Accessing your cluster by using the console. For more information on {{site.data.keyword.cloud_pak_data_notm}} console, see Using a custom TLS certificate for HTTPS connections to the platform.

Before you begin

To complete this task, you must be a Red Hat OpenShift Container Platform cluster administrator or the administrator on the project or namespace where foundational services is installed. The default namnespace is ibm-common-services.

Prepare and have your TLS certificate (tls, crt), TLS private key (tls.key), and CA certificate (ca.crt) ready. If needed, you can generate a new self-signed certificate by using OpenSSL.

ca.crt: You must have your full certificate chain if the CA certificate you bring is not self-signed. The full certificate chain includes all the CA certificates that signed each intermediate CA certificate that signed your CA certificate and your root CA. The files must be PEM-encoded files.
tls.crt: A Server Certificate (Leaf Certificate). The files must be PEM-encoded files.
tls.key: The private key corresponding to the signed certificate. The files must be PEM-encoded files.

You can review the full chain certificates in a cert file, using the following command:

  openssl crl2pkcs7 -nocrl -certfile <full-chain.crt> | openssl pkcs7 -print_certs -text -noout

Your certificate must include the external hostname for the foundational services endpoint in the Subject Alternative Name (SAN) list. You can identify this hostname by using the following command:

  oc get route cp-console -n ibm-common-services -o jsonpath="{.spec.host}"

The following shows an example of a CSR file:

  subjectAltName = @alt_names # Subject alternative names
  [ alt_names ]        # Subject alternative names
  DNS.1 = cp-console.apps.spmcsupgradetest.cp.fyre.ibm.com

The following example configuration file and OpenSSL commands provide an example for how to generate a TLS certificate by using OpenSSL.

OpenSSL commands for generating a certificate

The following OpenSSL commands are used with the preceding configuration file to generate the required TLS certificate.

Generate your certificate authority (CA) RSA private key:
```
openssl genrsa -out ca.key 2048
```

Generate a self-signed CA certificate by using your CA key:

openssl req -x509 -new -nodes -key ca.key -subj "/C=US/ST=New York/L=Armonk/O=IBM Cloud  Private" -days 100 -out ca.crt

Generate the RSA private key for your certificate:
```
openssl genrsa -out tls.key 2048
```
Generate the Certificate Signing request (CSR) by using the private key:
```
openssl req -new -key tls.key -out tls.csr -config csr.cnf
```

Generate a signed certificate by using your CA certificate and key, and CSR:

openssl x509 -req -in tls.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt -days 1000 -extensions v3_ext -extfile csr.cnf

Examine the certificate contents:
```
openssl x509  -noout -text -in ./tls.crt
```
Ensure that the common name on the certificate is the cluster CA domain for your product.

Example configuration file for generating a certificate

The following csr.cnf configuration file defines the configuration settings for generating certificates with OpenSSL.

  # cat csr.cnf
  [ req ]               # Main settings
  default_bits = 2048       # Default key size in bits.
  prompt = no               # Disables prompting for certificate values so the configuration file values are used.
  default_md = sha256       # Specifies the digest algorithm.
  req_extensions = req_ext  # Specifies the configuration file section that includes any extensions.
  distinguished_name = dn   # Specifies the section that includes the distinguished name information.

  [ dn ]               # Distinguished name settings
  C = US                    # Country
  ST = New York             # State or province
  L = Armonk                # Locality
  O = IBM Cloud Private     # Organization
  OU = IBM Cloud Pak        # Organizational unit
  CN = cp-console           # Common name.

  [ req_ext ]          # Extensions
  subjectAltName = @alt_names # Subject alternative names

  [ alt_names ]        # Subject alternative names
  DNS.1 = <foundational service endpoint hostname>

  [ v3_ext ]          # x509v3 extensions
  authorityKeyIdentifier=keyid,issuer:always  # Specifies the public key that corresponds to the private key that is used to sign a certificate.
  basicConstraints=CA:FALSE                   # Indicates whether the certificate is a CA certificate during the certificate chain verification process.
  keyUsage=keyEncipherment,dataEncipherment   # Defines the purpose of the key that is contained in the certificate.
  extendedKeyUsage=serverAuth,clientAuth      # Defines the purposes for which the public key can be used.
  subjectAltName=@alt_names                   # Identifies the subject alternative names for the identify that is bound to the public key by the CA.

Use your own certificate for the console and API

Update the managementingress custom resource so that ibm-management-ingress-operator does not re-create the route-cert certificate and the route-tls-secret secret after you replace them with your certificate.

 oc -n ibm-common-services patch managementingress default --type merge --patch '{"spec":{"ignoreRouteCert":true}}'

You can validate the changes by reviewing the ibm-management-ingress-operator pod logs, where you see the following message:

 oc get pods -l name=ibm-management-ingress-operator -o custom-columns=:metadata.name -n ibm-common-services | xargs oc logs | grep IgnoreRouteCert

I0504 00:03:05.824911 1 cert.go:107] Not watching certificate: route-cert, IgnoreRouteCert is true.

Get your TLS certificate as described in Before you begin. Remember to include the external hostname for the foundational services endpoint in the subjectAltName list of your certificate.
Save the cert, key, and ca-cert files from your certificate in a directory. For example,
```
 ls -l
 -rw-r--r-- 1 root root  2021 Oct 19 18:17 ca.crt
 -rw-r--r-- 1 root root  1777 Oct 19 18:18 tls.crt
 -rw-r--r-- 1 root root  1675 Oct 19 18:17 tls.key
```
You must have your full certificate chain if the CA certificate you bring is not self-signed. The full certificate chain includes all the CA certificates that signed each intermediate CA certificate that signed your CA certificate and your root CA. Save the full certificate chain in ca.crt.

Re-create the route-tls-secret with your certificate.

a. Backup the current certificate and secret:

oc -n ibm-common-services get certificates.v1alpha1.certmanager.k8s.io route-cert  -o yaml > route-cert-v1alpha1.yaml
oc -n ibm-common-services get secret route-tls-secret  -o yaml > route-tls-secret.yaml

b. Delete the related certificate resource so that cert manager does not re-create your updated secret.

 oc -n ibm-common-services delete certificates.v1alpha1.certmanager.k8s.io route-cert

c. Re-create the secret from your certificate.

 oc -n ibm-common-services delete secret route-tls-secret
 oc -n ibm-common-services create secret generic route-tls-secret --from-file=ca.crt=<your path>/ca.crt  --from-file=tls.crt=<your path>/tls.crt  --from-file=tls.key=<your path>/tls.key

d. Delete the ibmcloud-cluster-ca-cert secret. ibm-management-ingress-operator re-creates the ibmcloud-cluster-ca-cert secret and the cp-console route by using values from your new route-tls-secret.

 oc -n ibm-common-services delete secret ibmcloud-cluster-ca-cert

Restart auth-idp pods.

 oc -n ibm-common-services delete pod -l app=auth-idp

Restart operand-deployment-lifecycle-manager pods.

 oc -n ibm-common-services delete pod -l name=operand-deployment-lifecycle-manager

Access the console to verify that the certificate is used.

Revert back to the original self-signed certificate

If you have a problem accessing the cp-console after updating the endpoints with your own certificate, you can revert to the self-signed certificate by following these steps.

Delete the managementingress custom resource:

oc delete managementingress default -n ibm-common-services

Delete the cp-console route:

oc delete route cp-console  -n ibm-common-services

Restart the management-ingress pods by deleting the pods:

oc get pods | grep  management

ibm-common-services                                ibm-management-ingress-operator-844b45b8b9-nqqg7
ibm-common-services                                management-ingress-d5ccbf796-88rb6

Restart auth-idp pods.

oc -n ibm-common-services delete pod -l app=auth-idp

Restart the operand-deployment-lifecycle-manager pods.

oc -n ibm-common-services delete pod -l name=operand-deployment-lifecycle-manager