Supported machine learning tools, libraries, frameworks, and software specifications

In IBM Watson Machine Learning, you can use popular tools, libraries, and frameworks to train and deploy machine learning models and functions. The environment for these models and functions is made up of specific hardware and software specifications.

Software specifications define the language and version that you use for a model or function. They enable you to better configure the software that is used for running your models and functions. By using software specifications, you can precisely define the software version to be used and include your own extensions (for example, by using conda .yml files or custom libraries).

You can get a list of available software and hardware specifications and then use their names and IDs for use with your deployment. For details on how to do that, refer to the documentation for Python client or REST API.

Important: The tables in this topic document the supported frameworks and software specifications for the current release of Cloud Pak for Data. To see the list of supported frameworks and software specifications for a specific refresh version of Cloud Pak for Data, open the PDF file for "Deploying and managing models and functions" for that refresh version in Documentation for previous 4.6.x refreshes.

Predefined software specifications

you can use popular tools, libraries, and frameworks to train and deploy machine learning models and functions

This table lists the predefined (base) model types and software specifications.

List of predefined (base) model types and software specifications
Framework Versions Model Type Default
Software specification		 Supported platforms
AutoAI 0.1 NA hybrid_0.1
autoai-kb_rt22.1-py3.9 (deprecated)
Deprecated since release 4.6.4
autoai-kb_rt22.2-py3.10
autoai-ts_rt22.1-py3.9 (deprecated)
Deprecated since release 4.6.4
autoai-ts_rt22.2-py3.10		 x86, PPC
Decision Optimization 20.1 do-docplex_20.1
do-opl_20.1
do-cplex_20.1
do-cpo_20.1		 do_20.1 x86, PPC
Decision Optimization 22.1 do-docplex_22.1
do-opl_22.1
do-cplex_22.1
do-cpo_22.1		 do_22.1 x86, PPC
Hybrid/AutoML 0.1 wml-hybrid_0.1 hybrid_0.1 x86, PPC
PMML 3.0 to 4.3 pmml_. (or) pmml_..*3.0 - 4.3 pmml-3.0_4.3 x86, PPC
PyTorch 1.10 pytorch-onnx_1.10
pytorch-onnx_rt22.1		 runtime-22.1-py3.9 (deprecated)
Deprecated since release 4.6.4
pytorch-onnx_rt22.1-py3.9 (deprecated)
Deprecated since release 4.6.4
pytorch-onnx_rt22.1-py3.9-edt (deprecated)
Deprecated since release 4.6.4		 x86, PPC, s390x
PyTorch 1.10 pytorch-onnx_1.10
pytorch-onnx_rt22.1		 runtime_22.1-py3.9-nnpa s390x
PyTorch 1.12 pytorch-onnx_1.12
pytorch-onnx_rt22.2		 runtime-22.2-py3.10
pytorch-onnx_rt22.2-py3.10
pytorch-onnx_rt22.2-py3.10-edt		 x86, PPC
Added in release 4.6.5:
s390x
PyTorch 1.12 pytorch-onnx_1.12
pytorch-onnx_rt22.2
Added in release 4.6.1:
pytorch-onnx_rt22.2-py3.10-dist(x86)		 x86
Python Functions 0.1 NA runtime-22.1-py3.9 (deprecated)
Deprecated since release 4.6.4		 x86, PPC, s390x
Python Functions 0.1 NA runtime_22.1-py3.9-nnpa s390x
Python Functions 0.1 NA runtime-22.2-py3.10 x86, PPC
Python Functions 0.1 NA runtime-22.2-py3.10 s390x

Added in release 4.6.3
Python Scripts 1.0 NA runtime-22.1-py3.9 (deprecated)
Deprecated since release 4.6.4		 x86, PPC, s390x
Python Scripts 1.0 NA runtime_22.1-py3.9-nnpa s390x
Python Scripts 1.0 NA runtime-22.2-py3.10 x86, PPC
Python Scripts 1.0 NA runtime-22.2-py3.10 s390x

Added in release 4.6.3
R Scripts 1.0 NA default_r3.6 (deprecated) x86, PPC
R Scripts 1.0 NA runtime-22.1-r3.6 (deprecated)
runtime-22.2-r4.2		 x86
Scikit-learn 1.0 scikit-learn_1.0 runtime-22.1-py3.9 (deprecated)
Deprecated since release 4.6.4		 x86, PPC, s390x
Scikit-learn 1.0 scikit-learn_1.0 runtime_22.1-py3.9-nnpa s390x
Scikit-learn 1.1 scikit-learn_1.1 runtime-22.2-py3.10 x86, PPC
Scikit-learn 1.1 scikit-learn_1.1 runtime-22.2-py3.10 s390x

Added in release 4.6.3
Spark 3.2 mllib_3.2 spark-mllib_3.2 (deprecated) x86, PPC
Spark 3.3 mllib_3.3 spark-mllib_3.3 x86, PPC
SPSS 17.1 spss-modeler_17.1 spss-modeler_17.1 x86, PPC
SPSS 18.1 spss-modeler_18.1 spss-modeler_18.1 x86, PPC
SPSS 18.2 spss-modeler_18.2 spss-modeler_18.2 x86, PPC
Tensorflow 2.7 tensorflow_2.7
tensorflow_rt22.1		 runtime-22.1-py3.9 (deprecated)
Deprecated since release 4.6.4
tensorflow_rt22.1-py3.9 (deprecated)
Deprecated since release 4.6.4		 x86, PPC, s390x
Tensorflow 2.7 tensorflow_2.7
tensorflow_rt22.1		 tensorflow_rt22.1-py3.9-nnpa (deprecated)
Deprecated since release 4.6.4		 s390x
Tensorflow 2.9 tensorflow_2.9
tensorflow_rt22.2		 runtime-22.2-py3.10
tensorflow_rt22.2-py3.10		 x86, PPC
Tensorflow 2.9 tensorflow_2.9
tensorflow_rt22.2		 runtime-22.2-py3.10
tensorflow_rt22.2-py3.10		 s390x

Added in release 4.6.3
Tensorflow 2.9 tensorflow_2.9
tensorflow_rt22.2
Added in release 4.6.1:
tensorflow_rt22.2-py3.10-dist(x86)
tensorflow_rt22.2-py3.10-edt(x86)		 x86
XGBoost 1.5 xgboost_1.5 or scikit-learn_1.0 (see notes) runtime-22.1-py3.9 (deprecated)
Deprecated since release 4.6.4		 x86, PPC, s390x
XGBoost 1.5 xgboost_1.5 or scikit-learn_1.0 (see notes) runtime_22.1-py3.9-nnpa s390x
XGBoost 1.6 xgboost_1.6 or scikit-learn_1.1 (see notes) runtime-22.2-py3.10 x86, PPC
XGBoost 1.6 xgboost_1.6 or scikit-learn_1.1 (see notes) runtime-22.2-py3.10 s390x

Added in release 4.6.3

Important:

  • For XGBoost, if model is trained with sklearn wrapper (XGBClassifier or XGBRegressor):
    • in Python 3.9, use the scikit-learn_1.0 model type.
    • in Python 3.10, use the scikit-learn_1.1 model type.
  • You can also deploy R Shiny apps (version 0.1). Software specifications: rstudio_r4.2 (for x86) and shiny-r3.6 (deprecated, used for x86 and PPC).
  • If a framework version is marked as deprecated, then support for this framework will be removed in a future release.
  • Training a model based on Tensorflow and PyTorch requires using Watson Studio or Watson Machine Learning Accelerator.

Discontinued model types and software specifications

Support for the following model types was discontinued:

List of discontinued model types
Model types End of support
do-docplex_12.10
do-opl_12.10
do-cplex_12.10
do-cpo_12.10		 4.0.9
do-docplex_12.9
do-opl_12.9
do-cplex_12.19
do-cpo_12.9		 4.0.7
mllib_2.4 4.0.7
mllib_2.4
(for PMML deployments)		 4.0.8
mllib_3.0 4.6
pytorch-onnx_1.3 4.0.6
pytorch-onnx_1.7 4.0.8
scikit-learn_0.23 4.0.8
tensorflow_2.1 4.0.6
tensorflow_2.4 4.0.8
xgboost_0.90 4.0.6
xgboost_1.3 4.0.8

Support for the following software specifications was discontinued:

List of discontinued software specifications
Software specification End of support
autoai-kb_3.3-py3.7 4.0.8
autoai-kb_3.4-py3.8 4.0.8
autoai-obm_3.0 4.6
autoai-obm_3.2 4.6
autoai-ts_3.9-py3.8 4.0.8
default_py3.7 4.0.6
default_py3.7_opence 4.0.8
default_py3.8 4.0.8
do_12.10 4.0.9
do_12.9 4.0.7
pytorch-onnx_1.3-py3.7 4.0.6
pytorch-onnx_1.3-py3.7-edt 4.0.6
spark-mllib_3.0 4.5 (PMML model type only)
spark-mllib_2.4 4.0.7
spark-mllib_2.4
(for PMML deployments)		 4.0.8
spark-mllib_3.0 4.6
tensorflow_2.4-py3.7 4.0.8
tensorflow_2.4-py3.8 4.0.8

When you have assets that rely on discontinued software specifications or frameworks, in some cases the migration is seamless. In other cases, your action is required to retrain or redeploy assets.

  • Existing deployments of models that are built with discontinued framework versions or software specifications will be removed on the date of discontinuation.
  • No new deployments of models that are built with discontinued framework versions or software specifications will be allowed.
  • If you upgrade from a previous version of Cloud Pak for Data, deployments of models, functions, or scripts that are based on unsupported frameworks are removed. You must re-create the deployments with supported frameworks.
  • If you upgrade from a previous version of Cloud Pak for Data and you have models that use unsupported frameworks, you can still access the models. However, you cannot train or score them until you upgrade the model type and software specification, as described in Managing outdated software specifications or frameworks.

Runtime differences

For various reasons, package versions installed in Watson Machine Learning runtimes and Watson Studio Notebook runtimes can be different, even if they are based on the same software specification.

Differences in release 4.6.0:

List of differences for the deployment images based on the runtime-22.1-py3.9 software specification in release 4.6.0
Package Watson Machine Learning version Watson Studio Notebook version
cloudpickle 1.6.0 2.0.0
fsspec 2021.10.1 2022.2.0
werkzeug 2.1.1 2.0.3

Differences in release 4.6.1:

List of differences for the deployment images based on the runtime-22.1-py3.9 software specification in release 4.6.1
Package Watson Machine Learning version Watson Studio Notebook version
cloudpickle 1.6.0 2.0.0
fsspec 2021.10.1 2022.2.0
werkzeug 2.1.1 2.0.3
huggingface_hub Not available 0.2.1
importlib-metadata Not available 4.11.3
docplex 2.23.222 2.22.213
ibm-watson-openscale 3.0.26 3.0.27
dsx_core_utils 4.6.0.12 4.6.1-5
hadoop_lib_utils 4.6.0.12 4.6.1-5
List of differences for the deployment images based on the runtime-22.2-py3.10 software specification in release 4.6.1
Package Watson Machine Learning version Watson Studio Notebook version
libcurl 7.85.0 7.84.0
ibm-watson-openscale 3.0.26 3.0.27
docplex 2.23.222 2.22.213
dsx_core_utils 4.6.0.12 4.6.1-5
hadoop_lib_utils 4.6.0.12 4.6.1-5

Differences in release 4.6.3:

List of differences for the deployment images based on the runtime-22.1-py3.9 software specification in release 4.6.3
Package Watson Machine Learning version Watson Studio Notebook version
py4j 0.10.9.3 0.10.9.2
werkzeug 2.2.2 2.0.3
zlib 1.2.13 1.2.12
List of differences for the deployment images based on the runtime-22.2-py3.10 software specification in release 4.6.3
Package Watson Machine Learning version Watson Studio Notebook version
libcurl 7.85.0 7.84.0
werkzeug 2.2.2 2.0.3
mysql-connector-python 8.0.31 8.0.30

Differences in release 4.6.4:

List of differences for the deployment images based on the runtime-22.1-py3.9 software specification in release 4.6.4
Package Watson Machine Learning version Watson Studio Notebook version
pillow 9.3.0 9.0.1
werkzeug 2.1.1 2.0.3
zlib 1.2.13 1.2.12
py4j 0.10.9.3 0.10.9.2
ibm-watson-machine-learning 1.0.283 1.0.288
List of differences for the deployment images based on the runtime-22.2-py3.10 software specification in release 4.6.4
Package Watson Machine Learning version Watson Studio Notebook version
libcurl 7.87.0 7.84.0
mysql-connector-python 8.0.32 8.0.30
ibm-watson-machine-learning 1.0.283 1.0.288

Differences in release 4.6.5:

List of differences for the deployment images based on the runtime-22.1-py3.9 software specification in release 4.6.5
Package Watson Machine Learning version Watson Studio Notebook version
pillow 9.3.0 9.0.1
werkzeug 2.1.1 2.2.3
zlib 1.2.13 1.2.12

Note:
Deployment images based on the runtime-22.2-py3.10 software specification for Watson Machine Learning and Watson Studio do not differ.

Learn more

Parent topic: Managing frameworks and software specifications