Upgrading from IBM Cloud Pak for Data Version 4.0

A Red Hat® OpenShift® Container Platform cluster administrator and project administrator can work together to prepare the cluster and upgrade IBM® Cloud Pak for Data from Version 4.0 to Version 4.5.

Your Cloud Pak for Data deployment will be unavailable during the upgrade.

Before you begin

Before you upgrade Cloud Pak for Data:
  1. Review the information in the Planning section.

    Specifically, ensure that you review the System requirements. Your cluster must have sufficient resources.

  2. If your cluster pulls images from the IBM Entitled Registry, ensure that your cluster uses catalog sources that pull specific versions of images from the IBM Entitled Registry.
    Important: If your cluster uses the IBM Operator Catalog, you must migrate from the IBM Operator Catalog. You cannot upgrade to Cloud Pak for Data Version 4.5 if you want to continue using the IBM Operator Catalog.
  3. Determine which install plan the IBM Cloud Pak® foundational services operators and Cloud Pak for Data operators are using:
    oc get installplan
    • If the install plan approval is Automatic, you can proceed to the next step.
    • If the install plan approval is Manual, review the following options:
      Option Details
      Change the install plan to Automatic (recommended) It is strongly recommended that you change the install plan for the IBM Cloud Pak foundational services operators and Cloud Pak for Data operators to Automatic. This enables the cpd-cli manage commands to seamlessly update the operators.

      To update the install plan for the operators:

      1. For the IBM Cloud Pak foundational services operators, see the Changing approval strategy from Manual to Automatic in the IBM Cloud Pak foundational services documentation.
      2. For the Cloud Pak for Data operators, update the install plans for each operator through the Red Hat OpenShift Container Platform console. Open each subscription, view the subscription details, and edit the Update approval setting. For a list of the Cloud Pak for Data operators, see Creating operator subscriptions in the IBM Cloud Pak for Data Version 4.0 documentation.
      Important: Ensure that all of the operators in the ${PROJECT_CPFS_OPS} project and ${PROJECT_CPD_OPS} project are set to Automatic. If any of the install plans are set to Manual, Operator Lifecycle Manager (OLM) will automatically update the install plans to Manual when you run the cpd-cli manage apply-olm command.
      Leave the install plan as Manual You can optionally leave the install plan for the IBM Cloud Pak foundational services operators and Cloud Pak for Data operators Manual.
      Important: If you choose this option, you must watch the install plans and manually approve them during the upgrade to ensure that the cpd-cli manage apply-olm commands complete successfully.

      Additionally, you must repeat this process each time that you upgrade the operators to a newer release.

      Run the appropriate commands based on where your operators are installed:

      The IBM Cloud Pak foundational services operators and Cloud Pak for Data operators are installed in the same project (express installation)
      1. After you run the cpd-cli manage apply-olm command, open a new terminal window.
      2. Run the following command to watch the project where the operators are installed:
        watch oc get installplan -n ${PROJECT_CPFS_OPS}
      3. Manually approve each install plan as it is created.
      The IBM Cloud Pak foundational services operators and Cloud Pak for Data operators are installed in different projects (specialized installation)
      1. After you run the cpd-cli manage apply-olm command, open two new terminal windows.
      2. In the first terminal window, run the following command to watch the project where the IBM Cloud Pak foundational services operators are installed:
        watch oc get installplan -n ${PROJECT_CPFS_OPS}
      3. In the second terminal window, run the following command to watch the project where the Cloud Pak for Data operators are installed:
        watch oc get installplan -n ${PROJECT_CPD_OPS}
      4. Manually approve each install plan as it is created by setting spec.approved to true.
  4. Ensure that the Cloud Pak for Data control plane is pinned at a specific version:
    oc get ZenService lite-cr \
-n ${PROJECT_CPD_INSTANCE} \
-o jsonpath="{.spec.version}{'\n'}"
    • If the installation is pinned, the response includes the version at which the installation is pinned.
    • If the installation is not pinned, the response is empty.
      Run the following command to pin the Cloud Pak for Data control plane at a specific version:
      oc patch ZenService lite-cr \
--namespace ${PROJECT_CPD_INSTANCE} \
--type=merge \
--patch '{"spec": {"version":"N.N.N"}}'

      Replace N.N.N with the version to pin the control plane at based on the release of Cloud Pak for Data that you are running:

      Release Recommended version
      Refresh 2 4.2.0
      Refresh 3 4.3.1
      Refresh 4 4.3.2
      Refresh 5 4.3.2
      Refresh 6 4.4.0
      Refresh 7 4.4.2
      Refresh 8 4.4.3
      Refresh 9 4.4.4
  5. If the scheduling service is installed on your cluster, determine whether it's installed in the ibm-common-services project:
    oc get scheduling -A

    If the command returns a project other than ibm-common-services, uninstall the scheduling service, including the scheduling service operator.

    Remember: When you upgrade to Cloud Pak for Data Version 4.5, specify the scheduler component.

    The scheduling service will be installed in the project where IBM Cloud Pak foundational services is installed.

  6. Best practice Backup your Cloud Pak for Data installation before you upgrade.

    In the event of an unrecoverable failure, you can use the backup to recover your existing installation.

    Tip: If you decide to use the Cloud Pak for Data OADP backup and restore utility to perform the backup, see Creating an offline backup of a Cloud Pak for Data deployment, which includes guidance excluding services that do not support the OADP backup and restore utility from your backup.

Services that must be migrated from Version 4.0

The following services do not support upgrade from Cloud Pak for Data Version 4.0 to Version 4.5. However, you can migrate the data and objects from your existing installation on Version 4.0 to a new installation on Version 4.5:

Software Migration information
Cognos® Analytics To migrate to Cloud Pak for Data Version 4.5, you must:
  1. Back up your Cognos Analytics data store.
  2. Uninstall Cognos Analytics.

You can re-install Cognos Analytics when you upgrade to Cloud Pak for Data Version 4.5.

After you install Cognos Analytics, you can either:
  • Reuse your existing content store.
  • Create a new content store and use the backup that you created to restore the data.

Minimum required version

Some software must be at a specific version before you can upgrade from Cloud Pak for Data Version 4.0.

If any of the following software is installed on your cluster, ensure that it is at the minimum required version before you upgrade from Cloud Pak for Data Version 4.0 to Version 4.5. (It is recommended that you upgrade to the latest refresh, Version 4.5.3.)

Important:
There is a known issue for upgrades from Cloud Pak for Data Version 4.0.4.

If you have any of the following services installed, do not upgrade from Version 4.0.4 to Version 4.5.x:

  • Data Virtualization
  • Db2®
  • Db2 Big SQL
  • Db2 Warehouse
  • OpenPages® (with an embedded Db2 database)
  • Watson™ Knowledge Catalog

If you do not have any of these services, you can upgrade from Version 4.0.4 to Version 4.5.x.

If you do have one or more of these services, review the following guidance to determine the appropriate action for your environment:

Express installations
If all of your operators are installed in the same project, contact IBM Support.

An IBM Support representative must confirm that the Cloud Pak for Data operators and the IBM Cloud Pak foundational services are in the same project.

After the IBM Support representative confirms the location of the operators, they can apply the following workaround to your cluster prior to upgrade:
oc label sub ibm-db2u-operator -n ${PROJECT_CPD_OPS} operator.ibm.com/opreq-control-

If you attempt to run this workaround on specialized installation, the upgrade will fail.

Specialized installations
If your Cloud Pak for Data operators are in a different project from the IBM Cloud Pak foundational services services, you must upgrade to 4.0.9 before you upgrade to Cloud Pak for Data Version 4.5.x. For information on upgrading to Cloud Pak for Data Version 4.0.9, see Upgrading from IBM Cloud Pak for Data Version 4.0.x.
Minimum versions of required components
Software Minimum version
Cloud Pak for Data control plane Refresh 2 of Cloud Pak for Data, specifically:
  • IBM Cloud Pak for Data platform operator Version 2.0.4 or later
  • Zen operator Version 1.3.0 or later
  • Zen operand Version 4.2.0 or later
Scheduling service Version 1.2.3 or later.

Version 1.2.3 was released with Refresh 2 of Version 4.0
Minimum versions of services
Software Minimum version
Analytics Engine Powered by Apache Spark Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
Cognos Dashboards Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
Data Privacy Version 4.0.3 or later.

Version 4.0.3 was released with Refresh 3 of Version 4.0.
Data Refinery This service is upgraded when you upgrade one of the following services:
  • Watson Knowledge Catalog
  • Watson Studio
Data Virtualization Version 1.7.2 or later.

Version 1.7.2 was released with Refresh 2 of Version 4.0.
DataStage® Version 4.0.2 or later

Version 4.0.2 was released with Refresh 2 of Version 4.0.
Db2 Version 4.0.3 or later.

Version 4.0.3 was released with Refresh 2 of Version 4.0.

The service instance is running Version 11.5.6.0-cn3 or later.
Db2 Big SQL Version 7.2.2 or later.

Version 7.2.2 was released with Refresh 2 of Version 4.0.
Db2 Data Gate Version 2.0.8 or later.
Db2 Data Management Console Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
Db2 Warehouse Version 4.0.3 or later.

Version 4.0.3 was released with Refresh 2 of Version 4.0.

The service instance is running Version 11.5.6.0-cn3 or later.
Decision Optimization Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
EDB Postgres You must be running Refresh 2 or later of Cloud Pak for Data Version 4.0. (EDB Postgres operator Version 4.0.2 or later).
Execution Engine for Apache Hadoop Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
Guardium® External S-TAP® Version 4.0.8 or later.

Version 4.0.8 was released with Refresh 8 of Version 4.0.
IBM Match 360 with Watson Version 1.1.188 or later.

Version 1.1.188 was released with Refresh 6 of Version 4.0.

If you are running an older version of IBM Match 360 with Watson, contact IBM Support.
Informix® Version 4.0.0 or later.

Version 4.0.0 was released with Refresh 2 of Version 4.0.
MongoDB You must be running Refresh 2 or later of Cloud Pak for Data Version 4.0. (MongoDB operator Version 4.0.2 or later).
OpenPages Version 8.204.2 or later.

Version 8.204.2 was released with Refresh 5 of Version 4.0
Planning Analytics Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
Product Master Version 1.0.0 or later
RStudio® Server with R 3.6 Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
SPSS® Modeler Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
Voice Gateway Version 1.0.7 or later.

Version 1.0.7 was released with Refresh 2 of Version 4.0.
Watson Assistant Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
Watson Discovery You can upgrade from any 4.0.x version.

If you are running a Starter or Development deployment, you must specify the discovery_deployment_type: Starter configuration option. For details, see Specifying configuration options.

Important: If you are upgrading from Version 4.0.0 through 4.0.5, you must quiesce the service before you upgrade to Cloud Pak for Data Version 4.5. For details, see Enabling quiesce.
Watson Knowledge Catalog Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
Watson Machine Learning Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
Watson Machine Learning Accelerator Version 2.3.5 or later.

Version 2.3.5 was released with Refresh 5 of Version 4.0.
Watson OpenScale Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Version 4.0.
Watson Speech services Version 4.0.5 or later.

Version 4.0.5 was released with Refresh 5 of Version 4.0.
Watson Studio Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Data Version 4.0.
Watson Studio Runtimes Version 4.0.2 or later.

Version 4.0.2 was released with Refresh 2 of Data Version 4.0.

1. Setting up a client workstation

To upgrade to IBM Cloud Pak for Data Version 4.5, you must have a client workstation that can connect to the Red Hat OpenShift Container Platform cluster.

Tip: You can set up multiple client workstations if you want to enable multiple people to work on the upgrade.
The client workstation must be a Windows, Mac OS, or Linux® machine with the following software installed:
  • Cloud Pak for Data command-line interface (cpd-cli) Version 11.3.0 or later.
  • OpenShift command-line interface (oc) at a version that is compatible with your cluster.
Options What to do
You already have a client workstation set up
  1. Go to 2. Collecting required information
You don't have a client workstation set up
  1. Review the guidance in Setting up a client workstation (Upgrading from Version 3.5 to Version 4.5).
  2. Complete the following tasks to install the required software on the client workstation:
    1. Installing the IBM Cloud Pak for Data command-line interface (Upgrading from Version 4.0 to Version 4.5)
    2. Installing the OpenShift command-line interface (Upgrading from Version 4.0 to Version 4.5)
  3. Go to 2. Collecting required information

2. Collecting required information

a. Obtaining your IBM entitlement API key
All IBM Cloud Pak for Data images are accessible from the IBM Entitled Registry. The IBM entitlement API key enables you to pull software images from the IBM Entitled Registry, either for installation or for mirroring to a private container registry.
Options What to do
You already have your API key
  1. Go to b. Determining the list of components that you plan to install
You don't have your API key
  1. Complete Obtaining your IBM entitlement API key (Upgrading from Version 4.0 to Version 4.5)
  2. Go to b. Determining the list of components that you plan to install
b. Determining the list of components that you plan to install
Before you upgrade from Version 4.0, determine which components are installed on your cluster. In addition, determine whether you want to install any additional services on your cluster.
What to do
  1. You must review the list of components to ensure that you upgrade all the required components.

    Complete Determining which components to upgrade (Upgrading from Version 4.0 to Version 4.5).

  2. Go to c. Collecting information about your cluster that can be used to set up environment variables.
c. Collecting information about your cluster that can be used to set up environment variables
The commands for installing and upgrading IBM Cloud Pak for Data use variables with the format ${VARIABLE_NAME}. You can create a script to automatically export the appropriate values as environment variables before you run the installation commands. After you source the script, you will be able to copy most install and upgrade commands from the documentation and run them without making any changes.
What to do
  1. Complete Setting up installation environment variables (Upgrading from Version 4.0 to Version 4.5).
  2. Go to 3. Preparing your cluster.

3. Preparing your cluster

Before you upgrade Cloud Pak for Data, you must prepare your cluster.

a. Do you need to run the installation in a restricted environment?
If you need to run cpd-cli manage commands against a cluster in a restricted network, you must make the olm-utils image available inside the cluster network.
Options What to do
Your cluster is not in a restricted network
  1. Go to b. Do you have supported persistent storage on your cluster?
Your cluster is in a restricted network
  1. Review the guidance in Running cpd-cli manage commands in a restricted network (Upgrading from Version 4.0 to Version 4.5) to determine which method to use to make the required image available to one or more workstations in the cluster.
  2. Go to b. Do you have supported persistent storage on your cluster?
b. Do you have supported persistent storage on your cluster?
Supported storage for the Cloud Pak for Data platform

The Cloud Pak for Data platform supports the following storage:

Storage option Version Notes
OpenShift Data Foundation (formerly called OpenShift Container Storage) Version: 4.6 or later Available in the IBM Storage Suite for IBM Cloud® Paks

Ensure that you install a version of OpenShift Data Foundation that is compatible with the version of Red Hat OpenShift Container Platform that you are running. For details, see https://access.redhat.com/articles/4731161.
IBM Spectrum® Fusion Version 2.4.0 or later fixes Available in either:
  • IBM Spectrum Fusion
  • IBM Storage Suite for IBM Cloud Paks
IBM Spectrum Scale Container Native (with IBM Spectrum Scale Container Storage Interface) Version 5.1.3.x or later fixes

CSI Version 2.5.x or later fixes

 Available in either:
  • IBM Spectrum Fusion
  • IBM Storage Suite for IBM Cloud Paks
Portworx Version 2.9.1 or later fixes  
NFS Version 3 or 4

The latest version is recommend.

  
IBM Cloud Block Storage Not applicable Your environment must also include IBM Cloud File Storage.
IBM Cloud File Storage Not applicable It is recommended that you use both IBM Cloud Block Storage and IBM Cloud File Storage storage.
Options What to do
You have supported storage
  1. Go to c. Do you plan to upgrade any services that require custom SCCs?
You don't have supported storage
  1. Review Upgrading shared persistent storage (Upgrading from Version 3.5 to Version 4.5).
  2. Go to c. Do you plan to upgrade any services that require custom SCCs?
c. Do you plan to upgrade any services that require custom SCCs?
Services that require custom SCCs
If you had any of the following services installed on IBM Cloud Pak for Data Version 4.0, you can use the SCCs that you created as part of your Version 4.0 installation or upgrade:
  • Data Virtualization
  • Db2
  • Db2 Big SQL
  • Db2 Warehouse
  • OpenPages
  • Watson Knowledge Catalog
If you have any of the following services on IBM Cloud Pak for Data Version 4.0, you might need to create custom SCCs before you upgrade to IBM Cloud Pak for Data Version 4.5:
Options What to do
You plan to upgrade one or more of these services
  1. Create the appropriate SCCs for your environment. For details, see Creating custom security context constraints for services (Upgrading from Version 4.0 to Version 4.5).
  2. Go to d. Do you need to mirror the updated software images to a private container registry?
You don't plan to upgrade any of these services
  1. Go to d. Do you need to mirror the updated software images to a private container registry?
d. Do you need to mirror the updated software images to a private container registry?
If you are using a private container registry, you must mirror the updated images to the private container registry before you upgrade your environment.
Options What to do
You are pulling images from the IBM Entitled Registry
  1. Go to 4. Upgrading the Cloud Pak for Data platform and services
You are pulling images from a private container registry
  1. Complete Mirroring images to a private container registry (Upgrading from Version 4.0 to Version 4.5).
  2. Go to 4. Upgrading the Cloud Pak for Data platform and services

4. Upgrading the Cloud Pak for Data platform and services

After you prepare your cluster, you can upgrade the Cloud Pak for Data platform and services.

What to do
  1. Complete the appropriate tasks for your environment in Upgrading the IBM Cloud Pak for Data platform and services (Upgrading from Version 4.0 to Version 4.5).
  2. Go to 5. Completing post-upgrade tasks

5. Completing post-upgrade tasks

After you upgrade Cloud Pak for Data, determine whether there are any additional tasks that you should complete to configure your Cloud Pak for Data cluster.

What to do
Complete the appropriate tasks for your environment in Post-installation setup (Day 1 operations).

6. Upgrading services

Options What to do
You upgraded the services when you upgraded the platform Your environment is ready to use.
You didn't upgrade the services when you upgraded the platform Instructions for upgrading IBM services are available in Services.