Supported machine learning tools, libraries, frameworks, and software specifications
In IBM Watson Machine Learning, you can use popular tools, libraries, and frameworks to train and deploy machine learning models and functions. The environment for these models and functions is made up of specific hardware and software specifications.
Software specifications define the language and version that you use for a model or function. They enable you to better configure the software that is used for running your models and functions. By using software specifications, you can precisely define the software version to be used and include your own extensions (for example, by using conda .yml files or custom libraries).
You can get a list of available software and hardware specifications and then use their names and IDs for use with your deployment. For details on how to do that, refer to the documentation for Python client or REST API.
Important: The tables in this topic document the supported frameworks and software specifications for the current release of Cloud Pak for Data. To see the list of supported frameworks and software specifications for a specific refresh version of Cloud Pak for Data, open the PDF file for "Deploying and managing models and functions" for that refresh version in Documentation for previous 4.0.x refreshes.
Predefined software specifications
This table lists the predefined (base) model types and software specifications.
| Framework | Versions | Model Type | Default Software specification |
Supported platforms |
|---|---|---|---|---|
| AutoAI | 0.1 | NA | hybrid_0.1 autoai-obm_3.2 (deprecated from release 4.5.3) autoai-obm_3.0 (deprecated) autoai-kb_rt22.1-py3.9 autoai-ts_rt22.1-py3.9 |
x86, PPC |
| Decision Optimization | 20.1 | do-docplex_20.1 do-opl_20.1 do-cplex_20.1 do-cpo_20.1 |
do_20.1 | x86, PPC |
| Decision Optimization | 22.1 | do-docplex_22.1 do-opl_22.1 do-cplex_22.1 do-cpo_22.1 |
do_22.1 | x86, PPC |
| Hybrid/AutoML | 0.1 | wml-hybrid_0.1 | hybrid_0.1 | x86, PPC |
| PMML | 3.0 to 4.3 | pmml. (or) pmml..*3.0 - 4.3 | pmml-3.0_4.3 | x86, PPC |
| PyTorch | 1.10 | pytorch-onnx_1.10 | runtime-22.1-py3.9 | x86, PPC, s390x |
| PyTorch | 1.10 | pytorch-onnx_rt22.1 | pytorch-onnx_rt22.1-py3.9 pytorch-onnx_rt22.1-py3.9-edt |
x86, PPC |
| Python Functions | 0.1 | NA | runtime-22.1-py3.9 | x86, PPC, s390x |
| Python Scripts | 1.0 | NA | runtime-22.1-py3.9 | x86, PPC, s390x |
| R Scripts | 1.0 | NA | default_r3.6 (deprecated) | x86, PPC |
| R Scripts | 1.0 | NA | runtime-22.1-r3.6 | x86 |
| Scikit-learn | 1.0 | scikit-learn_1.0 | runtime-22.1-py3.9 | x86, PPC, s390x |
| Spark | 3.0 Cloud Pak for Data 4.5.0 only |
mllib_3.0 | spark-mllib_3.0 | x86, PPC |
| Spark | 3.2 | mllib_3.2 | spark-mllib_3.2 | x86, PPC |
| SPSS | 17.1 | spss-modeler_17.1 | spss-modeler_17.1 | x86, PPC |
| SPSS | 18.1 | spss-modeler_18.1 | spss-modeler_18.1 | x86, PPC |
| SPSS | 18.2 | spss-modeler_18.2 | spss-modeler_18.2 | x86, PPC |
| Tensorflow | 2.7 | tensorflow_2.7 | runtime-22.1-py3.9 | x86, PPC, s390x |
| Tensorflow | 2.7 | tensorflow_rt22.1 | tensorflow_rt22.1-py3.9 | x86, PPC |
| XGBoost | 1.5 | xgboost_1.5 | runtime-22.1-py3.9 | x86, PPC, s390x |
Important:
- You can also deploy R-shiny apps (version 0.1).
- If a framework version is marked as deprecated, then support for this framework will be removed in a future release.
- Training a model based on Tensorflow and PyTorch requires Watson Machine Learning Accelerator.
Action required: AutoAI experiments with joined data deprecated
The AutoAI experiment feature for joining multiple data sources to create a single training data set (software specifications: autoai-obm_3.0 and autoai-obm_3.2) is deprecated. Support for joining data in an AutoAI experiment will be removed in a future release. After support ends, AutoAI experiments with joined data and deployments of resulting models will no longer run.
To join multiple data sources, use a data preparation tool such as Data Refinery or DataStage to join and prepare data, then use the resulting data set for training an AutoAI experiment. Redeploy the resulting model.
Discontinued model types software specifications
Support for the following model types was discontinued:
| Model types | End of support |
|---|---|
| do-docplex_12.10 do-opl_12.10 do-cplex_12.10 do-cpo_12.10 |
4.0.9 |
| do-docplex_12.9 do-opl_12.9 do-cplex_12.19 do-cpo_12.9 |
4.0.7 |
| mllib_2.4 | 4.0.7 |
| mllib_2.4 (for PMML deployments) |
4.0.8 |
| pytorch-onnx_1.3 | 4.0.6 |
| pytorch-onnx_1.7 | 4.0.8 |
| scikit-learn_0.23 | 4.0.8 |
| tensorflow_2.1 | 4.0.6 |
| tensorflow_2.4 | 4.0.8 |
| xgboost_0.90 | 4.0.6 |
| xgboost_1.3 | 4.0.8 |
Support for the following software specifications was discontinued:
| Software specification | End of support |
|---|---|
| autoai-kb_3.3-py3.7 | 4.0.8 |
| autoai-kb_3.4-py3.8 | 4.0.8 |
| autoai-ts_3.9-py3.8 | 4.0.8 |
| default_py3.7 | 4.0.6 |
| default_py3.7_opence | 4.0.8 |
| default_py3.8 | 4.0.8 |
| do_12.10 | 4.0.9 |
| do_12.9 | 4.0.7 |
| pytorch-onnx_1.3-py3.7 | 4.0.6 |
| pytorch-onnx_1.3-py3.7-edt | 4.0.6 |
| spark-mllib_3.0 | 4.5 (PMML model type only) |
| spark-mllib_2.4 | 4.0.7 |
| spark-mllib_2.4 (for PMML deployments) |
4.0.8 |
| tensorflow_2.4-py3.7 | 4.0.8 |
| tensorflow_2.4-py3.8 | 4.0.8 |
When you have assets that rely on discontinued software specifications or frameworks, in some cases the migration is seamless. In other cases, your action is required to retrain or redeploy assets.
- Existing deployments of models that are built with discontinued framework versions or software specifications will be removed on the date of discontinuation.
- No new deployments of models that are built with discontinued framework versions or software specifications will be allowed.
- If you upgrade from a previous version of Cloud Pak for Data, deployments of models, functions, or scripts that are based on unsupported frameworks are removed. You must re-create the deployments with supported frameworks.
- If you upgrade from a previous version of Cloud Pak for Data and you have models that use unsupported frameworks, you can still access the models. However, you cannot train or score them until you upgrade the model type and software specification, as described in Managing outdated software specifications or frameworks.
Runtime changes not included in the software specification definition
When you check the software specification definition details, some last-minute changes may not be included in the output.
Here is the list of packages and their versions that are installed in the deployment images pertaining to the runtime-22.1-py3.9 software specification but different from the version specified in the software specification definition.
The list contains items for release 4.5.3.
| Package | Version |
|---|---|
_openmp_mutex |
5.1 |
autoai-libs |
1.13.6 |
ca-certificates |
2022.07.19 |
click |
8.0.4 |
ibm-watson-machine-learning |
1.0.237 |
jsonsubschema |
0.0.6 |
lale |
0.6.10 |
libgcc-ng |
9.3.0 |
libgomp |
9.3.0 |
libxml2 |
2.9.14 |
libxslt |
1.1.35 |
lxml |
4.9.1 |
pycryptodomex |
3.10.1 |
werkzeug |
2.1.1 (Watson Studio Notebook contains version 2.0.2) |
Learn more
- For details on customizing software specifications, refer to Creating a custom software specification in a project.
- For details on using and customizing environments, refer to Environments.
- For specific deployment examples, refer to sample Jupyter notebooks:
Parent topic: Managing frameworks and software specifications