Upgrading IBM Storage Scale container native

The following section describes how to upgrade the IBM Storage Scale container native cluster.

While an upgrade is in progress, do not perform the following:

• Do not make changes to the Cluster custom resource.
• Do not attempt to add a node to the cluster.

During an upgrade, the IBM Storage Scale operator orchestrates the upgrade procedure in a rolling node-by-node fashion. Each node will be:

• Cordoned (tainted unschedulable)
• Drained (pods are safely evicted and rescheduled to other available nodes)
• Rebooted, if necessary
• Uncordoned (returning it to normal service)

After the node is schedulable, IBM Storage Scale and IBM Storage Scale Container Storage Interface (CSI) pods will start. Applications may fail to attach storage until the system is started.

Prerequisites

• All the core pods need to be in running status.

  Use the following command to check the status of the core pods:

   oc get daemons ibm-spectrum-scale -n ibm-spectrum-scale -ojson | jq -r '.status.podsStatus'
  

  Ensure that there are no pods in any of the following states:

  • starting
  • terminating
  • unknown

  • waitingForDelete

    In the following example, the output shows 1 pod in "waitingForDelete", so the upgrade should not be done at this time.

    $ oc get daemons ibm-spectrum-scale -n ibm-spectrum-scale -ojson | jq -r '.status.podsStatus'
    {
    "running": "4",
    "starting": "0",
    "terminating": "0",
    "unknown": "0",
    "waitingForDelete": "1"
    }
    

Upgrade steps

Complete the following steps to upgrade:

1. Stop the running operator pod by setting the replicas in the deployment to 0.

   oc scale deployment ibm-spectrum-scale-controller-manager -n ibm-spectrum-scale-operator --replicas=0
   

2. Delete the old security context constraint.

   oc delete scc ibm-spectrum-scale-privileged
   

3. Delete the old role binding for privilege.

   oc delete rolebinding -n ibm-spectrum-scale ibm-spectrum-scale-privileged --ignore-not-found
   

4. Delete the MutatingWebhookConfiguration and ValidatingWebhookConfiguration. These will be created in later steps.

   oc delete MutatingWebhookConfiguration ibm-spectrum-scale-mutating-webhook-configuration
   oc delete ValidatingWebhookConfiguration ibm-spectrum-scale-validating-webhook-configuration
   

5. Apply the new manifests.

   oc apply -f https://raw.githubusercontent.com/IBM/ibm-spectrum-scale-container-native/v5.2.0.x/generated/scale/install.yaml
   

Verification

After the new IBM Storage Scale container native operator is deployed, the upgrade process will begin. It will take some time to complete as the new container images are rolled out into the cluster.

Validate the app.kubernetes.io/version on the operator deployment:

oc get deployment ibm-spectrum-scale-controller-manager \
-n ibm-spectrum-scale-operator  -ojson | jq -r .metadata.labels

To check the progress of the pod restarts and node reboots, query the daemon CR using the following command:

oc describe daemon ibm-spectrum-scale  -n ibm-spectrum-scale

Information will be available under the "Status Details" and "Events" sections.

Code version updated

The version details will be listed under .status.versions in the Daemon CR and will be updated as the pods roll. The following command will show the versions that core pods currently have on them. Wait until all the pods are reporting the same new version.

oc get daemon ibm-spectrum-scale -n ibm-spectrum-scale -ojson  | jq -r .status.versions