Preventive Service Planning
Abstract
This document lists the available patches for the Analytics Engine Powered by Apache Spark service on IBM Cloud Pak for Data. Not all versions include patches. Only versions that include patches are shown.
Content
- Cloud Pak for Data 4.6.0 patches
- Cloud Pak for Data 4.5.0 patches
- Cloud Pak for Data 3.5.0 patches
- Cloud Pak for Data 3.0.1 patches
- Cloud Pak for Data 2.5.0 patches
Patch name | cpd-spark-4.6.3-patch-1 |
Released on | 21 February 2023 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 4.6.3 |
Applies to platform version | Cloud Pak for Data 4.6.3 |
Patch type |
Image patch (Defect fix)
|
Description |
The patch includes a fix for:
Validating the patch
You can check if the patch was applied correctly by running the following commands:
|
Applying the patch
To apply the patch in an airgapped environment: 1. Download the image by running the following command. You need to provide
the values for "<folder path>/auth.json" and "<local private registry>" .
skopeo copy --all --authfile "<folder path>/auth.json" --dest-tls-verify=false --src-tls-verify=false docker://cp.icr.io/cp/cpd/spark-hb-jkg@sha256:aea0ac214532225248f734e871667332c8d567cbe040050765519d0063d0b18a <local private registry>/cp/cpd/spark-hb-jkg@sha256:aea0ac214532225248f734e871667332c8d567cbe040050765519d0063d0b18a
Prepare the authentication credentials to access the IBM production repository. Use the same auth.json file used for CASE download and image mirroring. An example directory path:
Or create an auth.json file that contains credentials to access cp.icr.io and your local private registry. For example:
2. Run the following command to apply the hotfix image:
What to do nextMake sure to revert the image overrides before you install or upgrade to a newer refresh or a major release of IBM® Cloud Pak for Data. To revert the image overrides, run the following command to edit the Analytics Engine custom resource (ae):
After approximately 5-6 minutes, the spark-hb-control-plane pod should be up and running with the patched image.
|
Patch name | cpd-spark-4.6.3-patch-2 |
Released on | 15 March 2023 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 4.6.3 |
Applies to platform version | Cloud Pak for Data 4.6.3 |
Patch type |
Image patch (Defect fix)
|
Description |
The patch includes a fix for:
Validating the patch
You can check if the patch was applied correctly by running the following commands:
|
Applying the patch
To apply the patch in an airgapped environment: 1. Download the image by running the following command. You need to provide
the values for "<folder path>/auth.json" and "<local private registry>" .skopeo copy --all --authfile "<folder path>/auth.json" --dest-tls-verify=false --src-tls-verify=false docker://cp.icr.io/cp/cpd/spark-hb-control-plane@sha256:551bdb89f8bc2067bc1183d9e9f470aef064855def21851e6678845ded1a8d24 <local private registry>/cp/cpd/spark-hb-control-plane@sha256:551bdb89f8bc2067bc1183d9e9f470aef064855def21851e6678845ded1a8d24
Prepare the authentication credentials to access the IBM production repository. Use the same auth.json file used for CASE download and image mirroring. An example directory path:
Or create an auth.json file that contains credentials to access cp.icr.io and your local private registry. For example:
2. Run the following command to apply the hotfix image:
oc patch AnalyticsEngine analyticsengine-sample --namespace $PROJECT_CPD_INSTANCE --type merge --patch '{"spec": {"image_digests": {"spark-hb-control-plane":"sha256:551bdb89f8bc2067bc1183d9e9f470aef064855def21851e6678845ded1a8d24","spark-hb-jkg-v33": "sha256:1425e65ed867bc09acdec14f351e604dc2cadef363607a60c5220103f341cd43"}}}'
What to do nextMake sure to revert the image overrides before you install or upgrade to a newer refresh or a major release of IBM® Cloud Pak for Data. To revert the image overrides, run the following command to edit the Analytics Engine custom resource (ae):
After approximately 5-6 minutes, the spark-hb-control-plane pod should be up and running with the patched image.
|
Patch name | cpd-spark-4.6.3-patch-3 |
Released on | 16 March 2023 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 4.6.3 |
Applies to platform version | Cloud Pak for Data 4.6.3 |
Patch type |
Image patch (Defect fix)
|
Description |
The patch includes a fix for:
Validating the patch
You can check if the patch was applied correctly by running the following commands:
|
Applying the patch
To apply the patch in an airgapped environment: 1. Download the image by running the following command. You need to provide
the values for "<folder path>/auth.json" and "<local private registry>" .
or:
Or create an auth.json file that contains credentials to access cp.icr.io and your local private registry. For example:
2. Run the following command to apply the hotfix image:
What to do nextMake sure to revert the image overrides before you install or upgrade to a newer refresh or a major release of IBM® Cloud Pak for Data. To revert the image overrides, run the following command to edit the Analytics Engine custom resource (ae):
After approximately 5-6 minutes, the spark-hb-control-plane pod should be up and running with the patched image.
|
Patch name | cpd-spark-4.5.1-patch-1 |
Released on | 21 November 2022 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 4.5.1 |
Applies to platform version | Cloud Pak for Data 4.5.1 |
Patch type |
Image patch (Defect fix)
|
Description |
The patch includes a fix for:
Important
Validating the patch
You can check after the patch was applied to determine if the job state transition is correct by running the following commands:
1. Get the AnalyticsEngine CR:
2. After the patch is applied, the AnalyticsEngine CR will move to the "InProgress" state. Then wait for the CR to move to the "Completed" state.
3. When the CR moves to "Completed" state, verify the control-plane pod digest by running the following command:
|
Instructions |
Applying the patch
To apply the patch: 1. For airgapped environments only: Download the image by running the following commands:
2. For airgapped environments only: Prepare the authentication credentials to access the IBM production repository. Use the same auth.json file used for CASE download and image mirroring. An example directory path:
Or create an auth.json file that contains credentials to access cp.icr.io and your local private registry. For example:
3. Run the following command to apply the hotfix image:
What to do nextMake sure to revert the image overrides before you install or upgrade to a newer refresh or a major release of IBM® Cloud Pak for Data. To revert the image overrides, run the following command to edit the Analytics Engine custom resource (ae):
After approximately 5-6 minutes, the spark-hb-control-plane pod should be up and running with the patched image.
|
Patch name | cpd-spark-4.5.2-patch-1 |
Released on | 14 October 2022 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 4.5.2 |
Applies to platform version | Cloud Pak for Data 4.5.2 |
Patch type |
Image patch (Defect fix)
|
Description |
The patch includes a fix for:
Important
Validating the patch
You can check after the patch was applied to determine if the job state transition is correct by running the following command.
1. After the patch is applied, the AnalyticsEngine CR will move to the "InProgress" state. Then wait for the CR to move to the "Completed" state.
2. When the CR moves to "Completed" state, verify the control-plane pod digest by running the following command:
oc get deploy spark-hb-control-plane -n zen -o jsonpath='{.spec.template.spec.containers[0].image}' |
Instructions |
Applying the patch
To apply the patch in an airgapped environment: 1. Download the image by running the following command:
skopeo copy --all --authfile "<folder path>/auth.json" --dest-tls-verify=false --src-tls-verify=false docker://cp.icr.io/cp/cpd/spark-hb-control-plane@sha256:2d0d569aaa192e1345503d51560fbaaa00793a5a671cecddc77f79276e5ca209 <local private registry>/cp/cpd/spark-hb-control-plane@sha256:2d0d569aaa192e1345503d51560fbaaa00793a5a671cecddc77f79276e5ca209
Prepare the authentication credentials to access the IBM production repository. Use the same auth.json file used for CASE download and image mirroring. An example directory path:
Or create an auth.json file that contains credentials to access cp.icr.io and your local private registry. For example:
2. Run the following command to apply the hotfix image:
What to do nextMake sure to revert the image overrides before you install or upgrade to a newer refresh or a major release of IBM® Cloud Pak for Data. To revert the image overrides, run the following command to edit the Analytics Engine custom resource (ae):
After approximately 5-6 minutes, the spark-hb-control-plane pod should be up and running with the patched image.
|
Patch name | cpd-spark-4.5.3-patch-1 |
Released on | 1 February 2023 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 4.5.3 |
Applies to platform version | Cloud Pak for Data 4.5.3 |
Patch type |
Image patch (Defect fix)
|
Description |
The patch includes a fix for:
Important
If you are using the Spark kernel offering, you must define a storage volume named "spark-shuffle-storage". This storage volume must be available in the cluster to avoid issues when running kernels.Validating the patch
You can check if the patch was applied correctly by running the following commands:
|
Applying the patch
To apply the patch in an airgapped environment: 1. Download the image by running the following command. You need to provide
the values for "<folder path>/auth.json" and "<local private registry>" .
Prepare the authentication credentials to access the IBM production repository. Use the same auth.json file used for CASE download and image mirroring. An example directory path:
Or create an auth.json file that contains credentials to access cp.icr.io and your local private registry. For example:
2. Run the following command to apply the hotfix image:
What to do nextMake sure to revert the image overrides before you install or upgrade to a newer refresh or a major release of IBM® Cloud Pak for Data. To revert the image overrides, run the following command to edit the Analytics Engine custom resource (ae):
After approximately 5-6 minutes, the spark-hb-control-plane pod should be up and running with the patched image.
|
Patch name | cpd-spark-4.5.3-patch-2 |
Released on | 21 February 2023 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 4.5.3 |
Applies to platform version | Cloud Pak for Data 4.5.3 |
Patch type |
Image patch (Defect fix)
|
Description |
The patch includes a fix for:
Validating the patch
You can check if the patch was applied correctly by running the following commands:
|
Applying the patch
To apply the patch in an airgapped environment: 1. Download the image by running the following command. You need to provide
the values for "<folder path>/auth.json" and "<local private registry>" .
skopeo copy --all --authfile "<folder path>/auth.json" --dest-tls-verify=false --src-tls-verify=false docker://cp.icr.io/cp/cpd/spark-hb-jkg@sha256:
8ab9e3c4c6a19c9158ed030970a8ce051e0f3a972dca69c1fb9ef5d3d93a55e7 <local private registry>/cp/cpd/spark-hb-jkg@sha256:8ab9e3c4c6a19c9158ed030970a8ce051e0f3a972dca69c1fb9ef5d3d93a55e7 Prepare the authentication credentials to access the IBM production repository. Use the same auth.json file used for CASE download and image mirroring. An example directory path:
Or create an auth.json file that contains credentials to access cp.icr.io and your local private registry. For example:
2. Run the following command to apply the hotfix image:
What to do nextMake sure to revert the image overrides before you install or upgrade to a newer refresh or a major release of IBM® Cloud Pak for Data. To revert the image overrides, run the following command to edit the Analytics Engine custom resource (ae):
After approximately 5-6 minutes, the spark-hb-control-plane pod should be up and running with the patched image.
|
Patch name | cpd-spark-4.5.3-patch-3 |
Released on | 02 March 2023 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 4.5.3 |
Applies to platform version | Cloud Pak for Data 4.5.3 |
Patch type |
Image patch (Defect fix)
|
Description |
The patch includes a fix for:
Validating the patch
You can check if the patch was applied correctly by running the following commands:
|
Applying the patch
To apply the patch: 1. Set up MachineConfig for the SELinux handler, by saving the following contents to a file, for example machineconfig.yaml:
2. Run the following command to deploy this MachineConfig:
3. To deploy the SELinux runtime class, create a file called "runtimeclass.yaml" with the following contents:
4. Run the following command to deploy this runtime class:
oc apply -f runtimeclass.yaml
Note: After these steps, the operators and pods should restart in the cluster which will take some time. Wait for the cluster to become idle again before continuing with the next steps. 5. If the cluster is air-gapped: Download the image by running the following command. You need to provide
the values for "<folder path>/auth.json" .
Or create an auth.json file that contains credentials to access cp.icr.io and your local private registry. For example:
6. Run the following command to apply the hotfix image:
What to do nextMake sure to revert the image overrides before you install or upgrade to a newer refresh or a major release of IBM® Cloud Pak for Data. To revert the image overrides, run the following command to edit the Analytics Engine custom resource (ae):
After approximately 5-6 minutes, the spark-hb-control-plane pod should be up and running with the patched image.
|
|
Patch name | cpd-spark-4.5.3-patch-4 |
Released on | 22 March 2023 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 4.5.3 |
Applies to platform version | Cloud Pak for Data 4.5.3 |
Patch type |
Image patch (Defect fix)
|
Description |
The patch includes a fix for:
Validating the patch
You can check if the patch was applied correctly by running the following commands:
|
Applying the patch
To apply the patch in an airgapped environment: 1. Download the image by running the following command. You need to provide
the values for "<folder path>/auth.json" and "<local private registry>" .
Prepare the authentication credentials to access the IBM production repository. Use the same auth.json file used for CASE download and image mirroring. An example directory path:
Or create an auth.json file that contains credentials to access cp.icr.io and your local private registry. For example:
2. Run the following command to apply the hotfix image:
What to do nextMake sure to revert the image overrides before you install or upgrade to a newer refresh or a major release of IBM® Cloud Pak for Data. To revert the image overrides, run the following command to edit the Analytics Engine custom resource (ae):
After approximately 5-6 minutes, the spark-hb-control-plane pod should be up and running with the patched image.
|
Patch name | cpd-3.5.3-spark-patch-3 |
Released on | 14 June 2021 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 3.5.3 |
Applies to platform version | Cloud Pak for Data 3.5.3 |
Patch type |
Cumulative
|
Description |
The patch includes the following fix:
Important
Validating the patch
You can check after the patch application to determine whether the spark-hb-preload-jkg-image cronjob was triggered or not by running the following command and check for LAST SCHEDULE.
oc get cronjob | grep spark-hb-preload-jkg-image
If the cronjob was triggered, running the following command and confirm the output:
oc get pods | grep spark-hb-preload-jkg-image
You can describe one of the pods from the output and check the image tag by running the following command:
oc get pod <podname> -o jsonpath="{..image}" | tr -s '[[:space:]]' '\n' | uniq
The image tag should match with Patched image tag (2.4.7.v12.9-3.5.3 and 3.0.2.v1.9-3.5.3) that is included in the Patch logs in the List of images required section.
After validating the output, submit a Spark job or Kernel request and describe the jkg-deployment or spark-master pod output, and then check the image tag. The image tag must match the cronjob pulled image tag and Patched image tag.
|
Instructions |
See Applying patches.
|
Patch name | cpd-3.5.3-spark-patch-2 |
Released on | 4 June 2021 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 3.5.3 |
Applies to platform version | Cloud Pak for Data 3.5.3 |
Patch type |
Cumulative
|
Description |
The patch includes the following fix:
Important
Validating the patch
You can check after the patch application to determine whether the spark-hb-preload-jkg-image cronjob was triggered or not by running the following command and check for LAST SCHEDULE.
oc get cronjob | grep spark-hb-preload-jkg-image
If the cronjob was triggered, running the following command and confirm the output:
oc get pods | grep spark-hb-preload-jkg-image
You can describe one of the pods from the output and check the image tag by running the following command:
oc get pod <podname> -o jsonpath="{..image}" | tr -s '[[:space:]]' '\n' | uniq
The image tag should match with Patched image tag (2.4.7.v12.8-3.5.3 and 3.0.2.v1.8-3.5.3) that is included in the Patch logs in the List of images required section.
After validating the output, submit a Spark job or Kernel request and describe the jkg-deployment or spark-master pod output, and then check the image tag. The image tag must match the cronjob pulled image tag and Patched image tag.
|
Instructions |
See Applying patches.
|
Patch name | cpd-3.5.3-spark-patch-1 |
Released on | 21 May 2021 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 3.5.3 |
Applies to platform version | Cloud Pak for Data 3.5.3 |
Patch type |
Cumulative
|
Description |
The patch includes the following fix:
Important:
Validating the patch
You can check after the patch application to determine whether the spark-hb-preload-jkg-image cronjob was triggered or not by running the following command and check for LAST SCHEDULE.
oc get cronjob | grep spark-hb-preload-jkg-image
If the cronjob was triggered, running the following command and confirm the output:
oc get pods | grep spark-hb-preload-jkg-image
You can describe one of the pods from the output and check the image tag by running the following command:
oc get pod <podname> -o jsonpath="{..image}" | tr -s '[[:space:]]' '\n' | uniq
The image tag should match with Patched image tag (2.4.7.v12.7-3.5.3 and 3.0.2.v1.7-3.5.3) that is included in the Patch logs in the List of images required section.
After validating the output, submit a Spark job or Kernel request and describe the jkg-deployment or spark-master pod output, and then check the image tag. The image tag must match the cronjob pulled image tag and Patched image tag.
|
Instructions |
See Applying patches.
|
Patch name | cpd-3.5.8-spark-patch-1 |
Released on | 16 February 2023 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 3.5.8 |
Applies to platform version | Cloud Pak for Data 3.5.8 |
Patch type |
Cumulative
|
Description |
The patch includes the following fix:
Validating the patch
After you have applied the patch, check if the spark-hb-helm-repo pod was restarted by running the following command:
Then describe one of the pods from the output and check the image tag:
The image tag should match the patched image tag (3.5.8001.3-amd64) that is included in the patch logs in the list of images required section.
|
See Applying patches.
|
Patch name | cpd-3.0.1-spark-patch-5 |
Released on | 14 January 2020 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 3.0.1 |
Applies to platform version | Cloud Pak for Data 3.0.1 |
Patch type |
Cumulative
|
Description |
The patch includes the following fixes:
|
Instructions |
See Applying patches.
Important: After you apply the patch, it might take two hours for the newer Spark 2.4.7 images to be pulled on the worker nodes.
Once the images are pulled, stop and start the history server so that it can use the newer Spark 2.4.7 images.
|
Patch name | cpd-3.0.1-spark-patch-4 |
Released on | 14 December 2020 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 3.0.1 |
Applies to platform version | Cloud Pak for Data 3.0.1 |
Patch type |
Cumulative
|
Description |
The patch includes the following fixes:
|
Instructions |
See Applying patches.
Important: After you apply the patch, it might take two hours for the newer Spark 2.4.6 images to be pulled on the worker nodes.
Once the images are pulled, stop and start the history server so that it can use the newer Spark 2.4.6 images.
|
Patch name | cpd-3.0.1-spark-patch-3 |
Released on | 7 September 2020 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 3.0.1 |
Applies to platform version | Cloud Pak for Data 3.0.1 |
Patch type |
Cumulative
|
Description |
This patch fix supports OpenShift versions 3.11, 4.3 and 4.5.
The patch includes the following fixes:
|
Instructions |
See Applying patches.
Important: After you apply the patch, it might take two hours for the newer Spark 2.4.6 images to be pulled on the worker nodes.
Once the images are pulled, stop and start the history server so that it can use the newer Spark 2.4.6 images.
|
Patch name | cpd-3.0.1-spark-patch-2 |
Released on | 14 August 2020 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 3.0.1 |
Applies to platform version | Cloud Pak for Data 3.0.1 |
Patch type |
Additive
Prerequisite:
|
Description |
This patch supports both the x86 and Power platforms.
The patch includes the following enhancements:
The patch includes the following fixes to the Spark history server:
|
Instructions |
Before applying the patch, perform the following tasks:
|
Patch name | cpd-3.0.1-spark-patch-1 |
Released on | 26 June 2020 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 3.0.1 |
Applies to platform version | Cloud Pak for Data 3.0.1 |
Patch type |
Cumulative
|
Description |
The patch includes the following fixes:
|
Instructions |
See Applying patches.
Important: After you apply the patch, it might take two hours for the newer Spark 2.4.6 images to be pulled on the worker nodes.
Once the images are pulled, stop and start the history server so that it can use the newer Spark 2.4.6 images.
|
Patch name | cpd-2.5.0.0-spark-patch-3 |
Released on | nn February 2021 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 2.5.0 |
Applies to platform version | Cloud Pak for Data 2.5 |
Patch type |
Cumulative
Prerequisites:
|
Description |
The patch includes the following fixes:
|
Instructions | See Applying patches. |
Patch name | cpd-2.5.0.0-spark-patch-2 |
Released on | 23 June 2020 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 2.5.0 |
Applies to platform version | Cloud Pak for Data 2.5 |
Patch type |
Cumulative
Prerequisites:
|
Description |
The patch includes the following fixes:
|
Instructions | See Applying patches. |
Patch name | cpd-2.5.0.0-spark-patch-1 |
Released on | 10 June 2020 |
Service assembly | spark |
Applies to service version | Analytics Engine for Apache Spark 2.5.0 |
Applies to platform version | Cloud Pak for Data 2.5 |
Patch type |
Cumulative
Prerequisites:
|
Description |
The patch includes the following fixes:
|
Instructions | See Applying patches. |
Was this topic helpful?
Document Information
Modified date:
22 March 2023
UID
ibm15693756