Important:

IBM Cloud Pak® for Data Version 4.8 will reach end of support (EOS) on 31 July, 2025. For more information, see the Discontinuance of service announcement for IBM Cloud Pak for Data Version 4.X.
Upgrade to IBM Software Hub Version 5.1 before IBM Cloud Pak for Data Version 4.8 reaches end of support. For more information, see Upgrading from IBM Cloud Pak for Data Version 4.8 to IBM Software Hub Version 5.1.

Customizing deployment runtime images

Create a custom Docker image with libraries and packages that are required for your deployments, then use the custom image to deploy your Watson Machine Learning assets.

You can build custom images based on deployment runtime images available in IBM Watson Machine Learning. The images contain preselected open source libraries and selected IBM libraries. By building custom images, you can optimize the standard software configuration of a runtime for your application needs. You can also use custom images in air-gapped environments with requirements that forbid you to show any operations to the internet.

Notes for working with custom images

  • The way that you create a custom image depends on whether you are building an image for Python-based models and functions or an SPSS image. For more information, see Creating a custom image.
  • These custom image types are not compatible with 4.5.0 or later releases of Cloud Pak for Data:
    • Images that were created in Cloud Pak for Data version 3.5
    • Images that were created before the 4.0.6 release of Cloud Pak for Data 4.x

If your custom image belongs to one of these categories, you must rebuild your image. Use the new base images published in the current release.

Rebuilding an existing image

  1. Download the base image.
  2. Rebuild the image with required libraries.
  3. Upload the rebuilt image to the local Cloud Pak for Data registry.
  4. Update the existing runtime-definition file with the new image.
  5. Perform online or batch scoring on the existing deployment.
Note:

If your deployed model uses a custom image that was created in one of previous releases of Cloud Pak for Data, you don't need to create a fresh deployment.

Training the model

If you want to train your SPSS model that uses a custom image, you can start with Building custom images to install ODBC drivers.

If you want to train your Python models or functions that use a custom image, you can start with Creating and training with custom image for Watson Studio. The Watson Studio image cannot be used for deployment in Watson Machine Learning, though. For deployment, create a custom image for Watson Machine Learning, as described in Creating a Watson Machine Learning custom image.

Supported frameworks for custom images

This table lists frameworks that are supported by the wml-deployment-runtime-py39-server.json runtime image.

Note:

The wml-deployment-runtime-py39-server.json runtime is deprecated and will be removed in a future release.

Framework Versions Online Batch
Scikit-learn 1.0 Yes Yes
XGBoost 1.5 Yes Yes
TensorFlow 2.7 Yes Yes
PyTorch 1.10 Yes Yes
Python function 0.1 Yes Programmatic only
Inline payload only
Python scripts 1.0 No Yes

This table lists frameworks that are supported by the wml-deployment-runtime-py310-server.json runtime image.

Framework Versions Online Batch
Scikit-learn 1.1 Yes Yes
XGBoost 1.6 Yes Yes
TensorFlow 2.9 Yes Yes
PyTorch 1.12 Yes Yes
Python function 0.1 Yes Programmatic only
Inline payload only
Python scripts 1.0 No Yes

This table lists the supported frameworks for the SPSS base image under wml-deployment-runtime-spss-server.json:

Framework Versions Online Batch
SPSS 18.2 No Yes

Creating and registering a custom image

Required role: You must be a Cloud Pak for Data cluster administrator to create and register a custom image.

Follow these steps to create and register a custom image:

  1. Preparing to build a new image
  2. Adding customizations and building a new image
  3. Pushing the image to the container server
  4. Changing and uploading the configuration file

Using the image for deployments

To use the custom image, follow these steps:

  1. Save the trained model that you want to deploy with the custom software specification that is defined for the custom image to the Watson Machine Learning repository.
  2. Deploy the model

Sample notebook

For sample notebooks that demonstrate how to use custom images to deploy a Python function, see Watson Machine Learning Python client samples and examples.

More considerations

If you use custom images, you must consider the following aspects:

  • Custom images for deployment can be built for Python with CPU environments only.

  • Make sure all updates that are made to the available runtime images in Watson Machine Learning, including all security updates, are also made to your custom images. Watch out for new fix packs or any related information. When new versions of Cloud Pak for Data are released, consider rebuilding all of your custom images.

  • Watson Machine Learning custom images can’t be used in Watson Studio or Data Refinery. They can be used in Watson Machine Learning only.

  • Watson Machine Learning uses uwsgi to run the deployment server. Make sure any extra packages and libraries that you install do not break uwsgi.

  • To get the list of Watson Machine Learning custom images, enter:

    oc get pods -l type=wml

  • After you install any package for Watson Machine Learning, make it available to wmlfuser:condausers.

  • User Python code or packages that are copied to the container have read/write permissions for all users.

  • Pod spawning is taken care of by the deployment manager:

    oc get pods | grep wml-deployment-manager

    Therefore, pod spawning errors are more often captured in the deployment manager log.

    oc logs wml-deployment-manager-0 > rm.log

Parent topic: Customizing deployment runtimes