Changing encryption keys and passwords

You can change encryption keys and passwords for on-prem and OCP systems.

About this task

OCP
In OCP systems, the key used to encrypt passwords is stored in the ${RELEASE}-custom-secrets secret.
Only observer job parameters are encrypted with this key.
Version 1.1.16: In previous versions, the encryption key was stored as an unreadable byte array, and attempts to view or edit it in the OCP console could corrupt it. From Agile Service Manager Version 1.1.16, the key is stored as a base64-encrypted human-readable version of the byte array. However, if an existing byte-array key exists, it will be used and will continue to be valid.
On-prem
In on-prem systems, the key used to encrypt passwords is stored in the $ASM_HOME/security/crypto.key file.
The key is used to encrypt:
  • Observer job parameters
  • Passwords stored on disk in $ASM_HOME/.*env files
Version 1.1.16: In previous versions, the encryption key was stored as an unreadable byte array. From Agile Service Manager Version 1.1.16, the $ASM_HOME/bin.print_key.sh script can be used to display a human-readable, base64-encoded version of the key. However, if an existing byte-array key exists, it will be used and will continue to be valid.

Procedure

Changing encryption key (on OCP)

You can modify an encryption key regularly, for example to satisfy customer security requirements for key or password rotation.
Tip: If a job fails due to a missing or incorrect key, this failure will be recorded against the observer's job history (Topology configuration > Observer jobs > View history).

  1. To make a copy of the old encryption key before updating it, use the following script:
    oc extract secret/${RELEASE}-custom-secrets
    The following example generates a file called `crypto.key` in the current directory:
    oc extract secret/noi-topology-custom-secrets
    • If the file size is 16 bytes, view the secret as base64 using cat crypto.key | base64
    • If the file size is greater, view the secret as base64 using cat crypto.key
  2. Generate a new human-readable, base64-encoded key:
    openssl rand -base64 16
    System output example: MpfvkUDvGHgOeGIO9XXLWQ==
  3. Using the Red Hat OpenShift Container Platform web console, edit the noi-topology-custom-secrets secret and provide the new crypto.key value. You can make a copy of the old key when you add the new key.
    Important:

    You will need the previous key to update existing observer jobs.

    The old key is displayed in the OpenShift web console as crypto.key, which you need to update. You can save the old key, for example as crypto.key.old.

  4. After updating the key, access the topology service Swagger pages and run the processEncryptionKeyChange crawler using the previous key, as in the following example:
    curl -X 'POST' \
      'https://netcool-noi.apps.panels.cp.fyre.ibm.com/1.0/topology/crawlers/processEncryptionKeyChange' \
      -H 'accept: application/json' \
      -H 'X-TransactionID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' \
      -H 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' \
      -H 'Content-Type: application/json' \
      -d '{
      "previousKey": "h1k3nK81/yCYeZWgSs8//Q==",
      "transactionLength": 1000
    }'
    
  5. Rerun the observer job from the Topology configuration > Observer jobs page. The Observer job history should indicate that the job completes successfully.

Sharing encryption keys in a geo-redundant deployment (on OCP)

In a geo-redundant system, the primary and backup must share the same encryption key.
Version 1.1.16: The key is auto-generated at startup if the ${RELEASE}-custom-secrets secret does not already exist.

  1. To ensure the primary and back-up servers sager the secret, use one of the following options.
    • Before installing on the back-up server, copy the auto-generated secret from the primary to the backup server.
    • Alternatively, create the secret manually:
      1. Generate a key:
        KEY=`openssl rand -base64 16`
      2. Use the same key for the secret on both the primary and backup clusters:
        oc create secret generic ${RELEASE}-custom-secrets --from-literal=crypto.key=${KEY}
        Where ${RELEASE} is the same prefix as is used for all other Kubernetes resources.

Changing encryption key (on-prem)

You can modify an encryption key regularly, for example to satisfy customer security requirements for key or password rotation.

  1. Run the following script:
    $ASM_HOME/bin/generate_key.sh
    The script performs the following actions:
    1. Prompts user input before making any changes.
    2. Updates the key.
    3. Updates locally stored passwords.
    4. Backs up the old key.
  2. Access the topology service Swagger pages, and using the previous key in base64-encoded format, run the processEncryptionKeyChange crawler.
    Tip: You can view the old key using the $ASM_HOME/bin.print_key.sh script.

Changing password (on-prem)

  • The Cassandra pod must be running for the Cassandra password to be updated.
  • The DASH password will not be auto-generated.
  • Any other passwords for which no value is given will be auto-generated.

  1. To change passwords, run the following script:
    $ASM_HOME/bin/change_passwords.sh
    Tip: For a usage example, run the script without any options.