Replacing the foundational services endpoint certificate

You can replace the certificate for the foundational services endpoint, which is the external endpoint that is used for access from outside the cluster to the console and API.

Replacing the foundational services endpoint certificate (version 3.6.3 and earlier)

Replacing the foundational services endpoint certificate (version 3.6.3 and earlier)

Before you begin

To complete this task, you must be a Red Hat OpenShift Container Platform cluster administrator or the administrator on the project or namespace where foundational services is installed. The default namnespace is ibm-common-services.

Prepare and have your TLS certificate (tls, crt), TLS private key (tls.key), and CA certificate (ca.crt) ready. If needed, you can generate a new self-signed certificate by using OpenSSL.

ca.crt: You must have your full certificate chain if the CA certificate you bring is not self-signed. The full certificate chain includes all the CA certificates that signed each intermediate CA certificate that signed your CA certificate and your root CA. The files must be PEM-encoded files.
tls.crt: A Server Certificate (Leaf Certificate). The files must be PEM-encoded files.
tls.key: The private key corresponding to the signed certificate. The files must be PEM-encoded files.

You can review the full chain certificates in a cert file, using the following command:

  openssl crl2pkcs7 -nocrl -certfile <full-chain.crt> | openssl pkcs7 -print_certs -text -noout

Your certificate must include the external hostname for the foundational services endpoint in the Subject Alternative Name (SAN) list. You can identify this hostname by using the following command:

  oc get route cp-console -n ibm-common-services -o jsonpath="{.spec.host}"

The following shows an example of a CSR file:

  subjectAltName = @alt_names # Subject alternative names
  [ alt_names ]        # Subject alternative names
  DNS.1 = cp-console.apps.spmcsupgradetest.cp.fyre.ibm.com

The following example configuration file and OpenSSL commands provide an example for how to generate a TLS certificate by using OpenSSL.

OpenSSL commands for generating a certificate

The following OpenSSL commands are used with the preceding configuration file to generate the required TLS certificate.

Generate your certificate authority (CA) RSA private key:
```
openssl genrsa -out ca.key 2048
```

Generate a self-signed CA certificate by using your CA key:

openssl req -x509 -new -nodes -key ca.key -subj "/C=US/ST=New York/L=Armonk/O=IBM Cloud  Private" -days 100 -out ca.crt

Generate the RSA private key for your certificate:
```
openssl genrsa -out tls.key 2048
```
Generate the Certificate Signing request (CSR) by using the private key:
```
openssl req -new -key tls.key -out tls.csr -config csr.cnf
```

Generate a signed certificate by using your CA certificate and key, and CSR:

openssl x509 -req -in tls.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt -days 1000 -extensions v3_ext -extfile csr.cnf

Examine the certificate contents:
```
openssl x509  -noout -text -in ./tls.crt
```
Ensure that the common name on the certificate is the cluster CA domain for your product.

Use your own certificate for the console and API (version 3.6.3 and earlier)

Update the managementingress CR so that ibm-management-ingress-operator does not re-create the route-cert certificate and the route-tls-secret secret. “Unmanaged” is case-sensitive.
```
oc -n ibm-common-services patch managementingress default --type merge --patch '{"spec":{"managementState":"Unmanaged"}}'
```
After this change, the ibm-management-ingress-operator pod will continue to run but it will do nothing. In the log of the pod, you see this message:
```
do nothing for the managementingress: ibm-common-services/default because its state is unmanaged
```
Get your TLS certificate as described in Before you begin. Remember to include the external hostname for the foundational services endpoint in the subjectAltName list of your certificate.

Get the destinationCACertificate value from the current cp-console route.

 oc -n ibm-common-services get route cp-console -o jsonpath="{.spec.tls.destinationCACertificate}" > dest-ca.crt

Update the cp-console route.

a. Save the cert, key, and ca-cert of your certificate in the same directory as the destinationCACertificate. For example,
```
 ```cmd
 ls -l
 total 68
 -rw-r--r-- 1 root root  2021 Oct 19 18:17 ca.crt
 -rw-r--r-- 1 root root  1168 Oct 19 18:28 dest-ca.crt
 -rw-r--r-- 1 root root  1777 Oct 19 18:18 tls.crt
 -rw-r--r-- 1 root root  1675 Oct 19 18:17 tls.key
 ```
```
You must have your full certificate chain if the CA certificate you bring is not self-signed. The full certificate chain includes all the CA certificates that signed each intermediate CA certificate that signed your CA certificate and your root CA. Save the full certificate chain in ca.crt.

b. Get the hostname from the current cp-console route. For example, cp-console.apps.demo.cp.fyre.ibm.com.
```
   oc -n ibm-common-services get route cp-console -o jsonpath="{.spec.host}"
```
c. Regenerate the route spec. In the following command, replace HOSTNAME with the hostname you got in step b.
```
   oc -n ibm-common-services create route reencrypt cp-console --service=icp-management-ingress  --cert=./tls.crt  --key=./tls.key  --ca-cert=./ca.crt  --dest-ca-cert=./dest-ca.crt  --hostname=HOSTNAME  --insecure-policy='Redirect'  --dry-run=true -o yaml  > cp-console.yaml
```
d. Apply the change.
```
   oc -n ibm-common-services  apply -f cp-console.yaml
```

Re-create the route-tls-secret with your certificate.

a. Delete the related certificate resource so that cert manager does not re-create your updated secret.

   oc -n ibm-common-services delete certificates.v1alpha1.certmanager.k8s.io route-cert

b. Re-create the secret from your certificate.

   oc -n ibm-common-services delete secret route-tls-secret
   oc -n ibm-common-services create secret generic route-tls-secret --from-file=ca.crt=<your path>/ca.crt  --from-file=tls.crt=<your path>/tls.crt  --from-file=tls.key=<your path>/tls.key

Re-create the ibm-cloud-cluster-ca-cert secret with your certificate.

oc -n ibm-common-services delete secret ibmcloud-cluster-ca-cert
oc -n ibm-common-services create secret generic ibmcloud-cluster-ca-cert --from-file=ca.crt=<your path>/ca.crt

Restart auth-idp pods.

 oc -n ibm-common-services delete pod -l app=auth-idp

Restart operand-deployment-lifecycle-manager pods.

 oc -n ibm-common-services delete pod -l name=operand-deployment-lifecycle-manager

Access the console to verify that the certificate is used.
You can use your own certificate and refresh the internal CA certificate as described in Refreshing foundational services internal CA certificate. Be sure to update the cp-console route after the internal CA certificate is refreshed. Refreshing foundational services internal CA certificate. updates the destination CA certificate. The destination CA certificate comes from the icp-management-ingress-tls-secret that is generated by the management-ingress-cert certificate.

a. Get the updated destinationCACertificate.
```
  oc -n ibm-common-services get secret icp-management-ingress-tls-secret -o jsonpath={.data."ca\.crt"} | base64 -d > dest-ca.crt
```
b. Rerun step 4 to update the cp-console route by using the updated dest-ca.crt.

Revert back to the original self-signed certificate

If you have a problem accessing the cp-console after updating the endpoints with your own certificate, you can revert to the self-signed certificate by following these steps.

Delete the managementingress custom resource:

oc delete managementingress default -n ibm-common-services

Delete the cp-console route, certificate, and the secret:

oc -n ibm-common-services delete route cp-console
oc -n ibm-common-services delete certificates.v1alpha1.certmanager.k8s.io route-cert
oc -n ibm-common-services delete secret route-tls-secret

Restart the management-ingress pods by deleting the pods:

oc get pods | grep  management

ibm-common-services                                ibm-management-ingress-operator-844b45b8b9-nqqg7
ibm-common-services                                management-ingress-d5ccbf796-88rb6

Restart auth-idp pods.

oc -n ibm-common-services delete pod -l app=auth-idp

Restart the operand-deployment-lifecycle-manager pods.

oc -n ibm-common-services delete pod -l name=operand-deployment-lifecycle-manager