Updating multiple controllers in a cluster

How to split the cluster and update one node (and revert or commit)

With multiple controllers in a cluster, you can split the cluster, update one controller and test it. You can issue flash and switch commands from either controller. When the testing is finished, you can choose to commit or discard the changes.

The WRKCSE, WRKCSEDTA and WRKCSECRDL information is stored in PowerHA CRG's and DDD's. This information is propagated to all the nodes in the cluster. By removing a node from the cluster, that data can be updated independently of the data in other clusters.

Prior to making any changes, it is recommended to save the existing toolkit library (QZRDHASM) and save the WRKCSE/WRKCSEDTA/WRKCSECRDL information using the SAVDDD command, which was released in the March 2019 4.3 build (prior builds and versions will need to use DMPINF and manually recording the information).

In this example, we are using two controllers - A and B. We want to update A first.

Split the cluster

1. Ensure that the cluster is consistent. On either node, issue the command WRKCLU opt 6.
2. Use SAVDDD to save the toolkit information stored in the cluster to an IFS directory.
3. On node A, remove it from the cluster (This will remove all toolkit data on this node):
   RMVCLUNODE CLUSTER(OLDCLUSTER) NODE(A)
4. Update the toolkit (RSTLIB etc)
5. Create a new cluster with a different name than the old one. This cannot be changed later.
   CRTCLU CLUSTER(NEWCLUSTER) NODE((A ('1.2.3.4')))
6. Create a new Device Data Domain
   WRKCLU opt 7
7. Use RSTDDD to restore the data which was placed into an IFS directly using SAVDDD.
   Alternately, enter the WRKCSE, WRKCSEDTA and WRKCSECRDL information from DMPINF files or however it was recorded.
   Only specify one controller in the CSE Data.

Propagate to the other controllers (i.e. commit the changes)

1. On node B, delete the cluster (This will remove all toolkit data on this node):
   DLTCLU OLDCLUSTER
2. On node B, update the toolkit (RSTLIB etc)
3. On node A, add node B to the cluster:
   ADDCLUNODE CLUSTER(NEWCLUSTER) NODE(B ('5.6.7.8'))
4. Add node B to A's device data domain (WRKCLU opt 7)
5. Node B will now have all the toolkit information from node A

Discard the toolkit update (i.e. revert to the prior configuration):

1. On node A delete the cluster (This will remove all toolkit data on this node):
   DLTCLU NEWCLUSTER
2. Restore the older toolkit library
3. On node B add node A to the cluster:
   ADDCLUNODE CLUSTER(OLDCLUSTER) NODE(A ('1.2.3.4'))
4. Add node A to B's device data domain (WRKCLU opt 7)
5. Node A will now all the toolkit information from node A.

Source LPAR save/restore

Saving and restoring the Source LPAR toolkit information is much more straightforward. It's information remains in database files in the toolkit library (QZRDHASM). Simply save and restore that library.