Replacing the foundational services endpoint certificate

You can replace the certificate for the foundational services endpoint, which is the external endpoint that is used for access from outside the cluster to the console and API.

Replacing the foundational services endpoint certificate (version 3.6.4 and later)

Before you begin

Prepare and have your TLS certificate, TLS private key, and CA certificate ready. If needed, you can generate a TLS certificate by using OpenSSL. Your certificate must include the external hostname for the foundational services endpoint in the Subject Alternative Name list. You can identify this hostname by using this command:

  oc get route cp-console -n ibm-common-services -o jsonpath="{.spec.host}"

The following example configuration file and OpenSSL commands provide an example for how to generate a TLS certificate by using OpenSSL.

Example configuration file for generating a certificate

The following csr.cnf configuration file defines the configuration settings for generating certificates with OpenSSL.

  # cat csr.cnf
  [ req ]               # Main settings
  default_bits = 2048       # Default key size in bits.
  prompt = no               # Disables prompting for certificate values so the configuration file values are used.
  default_md = sha256       # Specifies the digest algorithm.
  req_extensions = req_ext  # Specifies the configuration file section that includes any extensions.
  distinguished_name = dn   # Specifies the section that includes the distinguished name information.

  [ dn ]               # Distinguished name settings
  C = US                    # Country
  ST = New York             # State or province
  L = Armonk                # Locality
  O = IBM Cloud Private     # Organization
  OU = IBM Cloud Pak        # Organizational unit
  CN = cp-console           # Common name. 

  [ req_ext ]          # Extensions
  subjectAltName = @alt_names # Subject alternative names

  [ alt_names ]        # Subject alternative names
  DNS.1 = <foundational service endpoint hostname>

  [ v3_ext ]          # x509v3 extensions
  authorityKeyIdentifier=keyid,issuer:always  # Specifies the public key that corresponds to the private key that is used to sign a certificate.
  basicConstraints=CA:FALSE                   # Indicates whether the certificate is a CA certificate during the certificate chain verification process. 
  keyUsage=keyEncipherment,dataEncipherment   # Defines the purpose of the key that is contained in the certificate. 
  extendedKeyUsage=serverAuth,clientAuth      # Defines the purposes for which the public key can be used. 
  subjectAltName=@alt_names                   # Identifies the subject alternative names for the identify that is bound to the public key by the CA.

OpenSSL commands for generating a certificate

The following OpenSSL commands are used with the preceding configuration file to generate the required TLS certificate.

  1. Generate your certificate authority (CA) RSA private key:

    openssl genrsa -out ca.key 2048
    
  2. Generate a self-signed CA certificate by using your CA key:

    openssl req -x509 -new -nodes -key ca.key -subj "/C=US/ST=New York/L=Armonk/O=IBM Cloud  Private" -days 100 -out ca.crt
    
  3. Generate the RSA private key for your certificate:

    openssl genrsa -out tls.key 2048
    
  4. Generate the Certificate Signing request (CSR) by using the private key:

    openssl req -new -key tls.key -out tls.csr -config csr.cnf
    
  5. Generate a signed certificate by using your CA certificate and key, and CSR:

    openssl x509 -req -in tls.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt -days 1000 -extensions v3_ext -extfile csr.cnf
    
  6. Examine the certificate contents:

    openssl x509  -noout -text -in ./tls.crt
    

    Ensure that the common name on the certificate is the cluster CA domain for your product.

Use your own certificate for the console and API

  1. Update the managementingress CR so that ibm-management-ingress-operator does not re-create the route-cert certificate and the route-tls-secret secret.

    oc -n ibm-common-services patch managementingress default --type merge --patch '{"spec":{"ignoreRouteCert":true}}'
    

    In the log of the ibm-management-ingress-operator pod, you see this message:

    Not watching certificate: route-cert, IgnoreRouteCert is true.
    
  2. Get your TLS certificate as described in Before you begin. Remember to include the external hostname for the foundational services endpoint in the subjectAltName list of your certificate.

  3. Save the cert, key, and ca-cert files from your certificate in a directory. For example,

    # ls -l
    -rw-r--r-- 1 root root  2021 Oct 19 18:17 ca.crt
    -rw-r--r-- 1 root root  1777 Oct 19 18:18 tls.crt
    -rw-r--r-- 1 root root  1675 Oct 19 18:17 tls.key
    

    You must have your full certificate chain if the CA certificate you bring is not self-signed. The full certificate chain includes all the CA certificates that signed each intermediate CA certificate that signed your CA certificate and your root CA. Save the full certificate chain in ca.crt.

  4. Re-create the route-tls-secret with your certificate.

    a. Delete the related certificate resource so that cert manager does not re-create your updated secret.

     oc -n ibm-common-services delete certificates.v1alpha1.certmanager.k8s.io route-cert
    

    b. Re-create the secret from your certificate.

     oc -n ibm-common-services delete secret route-tls-secret
     oc -n ibm-common-services create secret generic route-tls-secret --from-file=ca.crt=<your path>/ca.crt  --from-file=tls.crt=<your path>/tls.crt  --from-file=tls.key=<your path>/tls.key
    

    c. Delete the ibmcloud-cluster-ca-cert secret. ibm-management-ingress-operator re-creates the ibmcloud-cluster-ca-cert secret and the cp-console route by using values from your new route-tls-secret.

     oc -n ibm-common-services delete secret ibmcloud-cluster-ca-cert
    
  5. Restart auth-idp pods.

     # oc -n ibm-common-services delete pod -l app=auth-idp
    
  6. Restart operand-deployment-lifecycle-manager pods.

     # oc -n ibm-common-services delete pod -l name=operand-deployment-lifecycle-manager
    
  7. Access the console to verify that the certificate is used.

Replacing the foundational services endpoint certificate (version 3.6.3 and earlier)

Before you begin

Prepare and have your TLS certificate, TLS private key, and CA certificate ready. If needed, you can generate a TLS certificate by using OpenSSL. Your certificate must include the external hostname for the foundational services endpoint in the Subject Alternative Name list. You can identify this hostname by using this command:

  oc get route cp-console -n ibm-common-services -o jsonpath="{.spec.host}"

The following example configuration file and OpenSSL commands provide an example for how to generate a TLS certificate by using OpenSSL.

Example configuration file for generating a certificate (version 3.6.3 and earlier)

The following csr.cnf configuration file defines the configuration settings for generating certificates with OpenSSL.

  # cat csr.cnf
  [ req ]               # Main settings
  default_bits = 2048       # Default key size in bits.
  prompt = no               # Disables prompting for certificate values so the configuration file values are used.
  default_md = sha256       # Specifies the digest algorithm.
  req_extensions = req_ext  # Specifies the configuration file section that includes any extensions.
  distinguished_name = dn   # Specifies the section that includes the distinguished name information.

  [ dn ]               # Distinguished name settings
  C = US                    # Country
  ST = New York             # State or province
  L = Armonk                # Locality
  O = IBM Cloud Private     # Organization
  OU = IBM Cloud Pak        # Organizational unit
  CN = cp-console           # Common name. 

  [ req_ext ]          # Extensions
  subjectAltName = @alt_names # Subject alternative names

  [ alt_names ]        # Subject alternative names
  DNS.1 = <foundational service endpoint hostname>

  [ v3_ext ]          # x509v3 extensions
  authorityKeyIdentifier=keyid,issuer:always  # Specifies the public key that corresponds to the private key that is used to sign a certificate.
  basicConstraints=CA:FALSE                   # Indicates whether the certificate is a CA certificate during the certificate chain verification process. 
  keyUsage=keyEncipherment,dataEncipherment   # Defines the purpose of the key that is contained in the certificate. 
  extendedKeyUsage=serverAuth,clientAuth      # Defines the purposes for which the public key can be used. 
  subjectAltName=@alt_names                   # Identifies the subject alternative names for the identify that is bound to the public key by the CA.

OpenSSL commands for generating a certificate (version 3.6.3 and earlier)

The following OpenSSL commands are used with the preceding configuration file to generate the required TLS certificate.

  1. Generate your certificate authority (CA) RSA private key:

    openssl genrsa -out ca.key 2048
    
  2. Generate a self-signed CA certificate by using your CA key:

    openssl req -x509 -new -nodes -key ca.key -subj "/C=US/ST=New York/L=Armonk/O=IBM Cloud  Private" -days 100 -out ca.crt
    
  3. Generate the RSA private key for your certificate:

    openssl genrsa -out tls.key 2048
    
  4. Generate the Certificate Signing request (CSR) by using the private key:

    openssl req -new -key tls.key -out tls.csr -config csr.cnf
    
  5. Generate a signed certificate by using your CA certificate and key, and CSR:

    openssl x509 -req -in tls.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt -days 1000 -extensions v3_ext -extfile csr.cnf
    
  6. Examine the certificate contents:

    openssl x509  -noout -text -in ./tls.crt
    

    Ensure that the foundational name on the certificate is the cluster CA domain for your product.

Use your own certificate for the console and API (version 3.6.3 and earlier)

  1. Update the managementingress CR so that ibm-management-ingress-operator does not re-create the route-cert certificate and the route-tls-secret secret. “Unmanaged” is case-sensitive.

    oc -n ibm-common-services patch managementingress default --type merge --patch '{"spec":{"managementState":"Unmanaged"}}'
    

    After this change, the ibm-management-ingress-operator pod will continue to run but it will do nothing. In the log of the pod, you see this message:

    do nothing for the managementingress: ibm-common-services/default because its state is unmanaged
    
  2. Get your TLS certificate as described in Before you begin. Remember to include the external hostname for the foundational services endpoint in the subjectAltName list of your certificate.

  3. Get the destinationCACertificate value from the current cp-console route.

     oc -n ibm-common-services get route cp-console -o jsonpath="{.spec.tls.destinationCACertificate}" > dest-ca.crt
    
  4. Update the cp-console route.

    a. Save the cert, key, and ca-cert of your certificate in the same directory as the destinationCACertificate. For example,

       # ls -l
       total 68
       -rw-r--r-- 1 root root  2021 Oct 19 18:17 ca.crt
       -rw-r--r-- 1 root root  1168 Oct 19 18:28 dest-ca.crt
       -rw-r--r-- 1 root root  1777 Oct 19 18:18 tls.crt
       -rw-r--r-- 1 root root  1675 Oct 19 18:17 tls.key
    

    You must have your full certificate chain if the CA certificate you bring is not self-signed. The full certificate chain includes all the CA certificates that signed each intermediate CA certificate that signed your CA certificate and your root CA. Save the full certificate chain in ca.crt.

    b. Get the hostname from the current cp-console route. For example, cp-console.apps.demo.cp.fyre.ibm.com.

       oc -n ibm-common-services get route cp-console -o jsonpath="{.spec.host}"
    

    c. Regenerate the route spec. In the following command, replace HOSTNAME with the hostname you got in step b.

       oc -n ibm-common-services create route reencrypt cp-console --service=icp-management-ingress  --cert=./tls.crt  --key=./tls.key  --ca-cert=./ca.crt  --dest-ca-cert=./dest-ca.crt  --hostname=HOSTNAME  --insecure-policy='Redirect'  --dry-run=true -o yaml  > cp-console.yaml
    

    d. Apply the change.

       oc -n ibm-common-services  apply -f cp-console.yaml
    
  5. Re-create the route-tls-secret with your certificate.

    a. Delete the related certificate resource so that cert manager does not re-create your updated secret.

       oc -n ibm-common-services delete certificates.v1alpha1.certmanager.k8s.io route-cert
    

    b. Re-create the secret from your certificate.

       oc -n ibm-common-services delete secret route-tls-secret
       oc -n ibm-common-services create secret generic route-tls-secret --from-file=ca.crt=<your path>/ca.crt  --from-file=tls.crt=<your path>/tls.crt  --from-file=tls.key=<your path>/tls.key
    
  6. Re-create the ibm-cloud-cluster-ca-cert secret with your certificate.

    oc -n ibm-common-services delete secret ibmcloud-cluster-ca-cert
    oc -n ibm-common-services create secret generic ibmcloud-cluster-ca-cert --from-file=ca.crt=<your path>/ca.crt
    
  7. Restart auth-idp pods.

     oc -n ibm-common-services delete pod -l app=auth-idp
    
  8. Restart operand-deployment-lifecycle-manager pods.

     # oc -n ibm-common-services delete pod -l name=operand-deployment-lifecycle-manager
    
  9. Access the console to verify that the certificate is used.

  10. You can use your own certificate and refresh the internal CA certificate as described in Refreshing foundational services internal CA certificate. Be sure to update the cp-console route after the internal CA certificate is refreshed. Refreshing foundational services internal CA certificate. updates the destination CA certificate. The destination CA certificate comes from the icp-management-ingress-tls-secret that is generated by the management-ingress-cert certificate.

    a. Get the updated destinationCACertificate.

      oc -n ibm-common-services get secret icp-management-ingress-tls-secret -o jsonpath={.data."ca\.crt"} | base64 -d > dest-ca.crt
    

    b. Rerun step 4 to update the cp-console route by using the updated dest-ca.crt.