What's new in IBM Cloud Pak for Data?

Version 4.8.5 of the Cloud Pak for Data platform includes the following features and updates:

Changes to the backup and restore process

The process to back up and restore Cloud Pak for Data changed. The changes depend on the backup and restore tool that you use.

If you use the OADP backup and restore utility:

• OADP 1.3.x is required.
• Version 4.8.5 of the backup and restore scripts must be used.
• The steps to create and restore online and offline backups are different.

If you use IBM® Storage Fusion, NetApp Astra Control Center, and Portworx, you must install the 4.8.5 version of the cpdbr service.

For more information, see Backing up and restoring Cloud Pak for Data.

Refresh 5 of Version 4.8 platform includes various fixes.

For details, see What's new and changed in the platform.

The 13.1.5 release of the Cloud Pak for Data command-line interface includes the following features and updates:

Collect information on the backup and restore process

Troubleshoot backup and restore problems when using the OADP backup and restore utility with the cpd-cli oadp gather info command. This command collects YAMLs of various Velero custom resources that can help IBM identify where problems occur during the backup and restore process.

Version 13.1.5 of theCloud Pak for Data command-line interface includes various fixes.

For details, see What's new and changed in the Cloud Pak for Data command-line interface.

The 8.5.0 release of the common core services includes changes to support features and updates in Watson™ Studio and IBM Knowledge Catalog.

Version 8.5.0 of the common core services includes various fixes.

For details, see What's new and changed in the common core services.

Version 1.22.0 of the scheduling service includes various fixes.

For details, see What's new and changed in the scheduling service.

Version 4.8.5 of the AI Factsheets service includes various fixes.

Version 4.8.5 of the Analytics Engine powered by Apache Spark service includes various fixes.

Version 4.8.5 of the Data Privacy service includes various fixes.

For details, see What's new and changed in Data Privacy.

Version 8.5.0 of the Data Refinery service includes various fixes.

For details, see What's new and changed in Data Refinery.

Version 4.8.5 of the Data Replication service includes various fixes.

For details, see What's new and changed in Data Replication.

The 4.8.5 release of DataStage includes the following features and updates:

Output rejected records from connectors on a designated reject link

You can now send records that meet specified reject criteria to a reject link on most connectors. For more information, see Reject links in DataStage.

Use new features with stored procedures

• For the Google BigQuery connector, you can now use stored procedures.
• For the Teradata connector, you can now use user-defined functions of types macro, scalar, and table in your stored procedures.
• For connectors with output links, you can now pass through all input columns to output columns.

For more information, see Using stored procedures in DataStage.

Use the Parquet file format with the Sequential file connector

You can now access data in the Parquet file format with the Sequential file connector. For more information, see Sequential file.

Authenticate to Google Cloud Pub/Sub with workload identity federation

You can now use workload identity federation to authenticate to Google Cloud Pub/Sub rather than using your Google service account key. Workflow identity federation provides increased security and centralized management. For more information, see Google Cloud Pub/Sub connection.

Use Delta Lake or Apache Iceberg table formats in the Generic S3 and the Microsoft Azure Data Lake Storage connectors

The Generic S3 and Microsoft Azure Data Lake Storage (ADLS) connectors now include interaction properties for the Delta Lake and Apache Iceberg table formats. These table formats are integral to data lakes, which provide a centralized repository for managing large volumes of data. You use data lakes to collect and analyze structured, semi-structured, or unstructured data in its original format.

Use Secure External Password Store to authenticate to an Oracle database with the Oracle (optimized) connection

The Oracle (optimized) connection now includes an option to authenticate with Secure External Password Store (SEPS). SEPS uses a client-side wallet to securely store the password credentials. For details about how to create the wallet files, see Connecting to Oracle data sources with SEPS authentication.

Specify alternate servers for the Oracle (optimized) connection to help ensure data access

You can now specify alternate servers for the Oracle (optimized) connection. Alternate servers provide increased availability in the event of a failure of the primary server. For more information, see Oracle (optimized) connection.

Version 4.8.5 of the DataStage service includes various fixes.

For details, see What's new and changed in DataStage.

The 4.8.5 release of Db2 includes the following features and updates:

Manage administrative tasks from a command line

You can now use the Db2 Administrative Task Scheduler (ATS) command-line interface to rotate SSL certificates and manage diagnostic log files. For more information, see Managing admin tasks from the command-line interface with the Administrative Task Scheduler (ATS) for the service.

Version 4.8.5 of the Db2 service includes various fixes.

For details, see What's new and changed in Db2.

Version 7.6.5 of the Db2 Big SQL service includes various fixes.

Version 4.8.5 of the Db2 Data Management Console service includes various fixes.

For details, see What's new and changed in Db2 Data Management Console.

The 4.8.5 release of Db2 Warehouse includes the following features and updates:

Manage administrative tasks from a command line

You can now use the Db2 Administrative Task Scheduler (ATS) command-line interface to rotate SSL certificates and manage diagnostic log files. For more information, see Running admin tasks from the command-line interface with ATS.

Version 4.8.5 of the Db2 Warehouse service includes various fixes.

For details, see What's new and changed in Db2 Warehouse.

Version 8.5.0 of the Decision Optimization service includes various fixes.

For details, see What's new and changed in Decision Optimization.

This release of the EDB Postgres service includes various fixes.

For details, see What's new and changed in EDB Postgres.

Version 4.8.5 of the Execution Engine for Apache Hadoop service includes various fixes.

For details, see What's new and changed in Execution Engine for Apache Hadoop.

Version 4.8.5 of the IBM Knowledge Catalog service includes various fixes.

For details, see What's new and changed in IBM Knowledge Catalog.

Version 7.3.0 of the Informix service includes various fixes.

Version 42.4.3 of the MANTA Automated Data Lineage service includes various fixes.

For details, see What's new and changed in MANTA Automated Data Lineage.

This release of the MongoDB service include various fixes.

For details, see What's new and changed in MongoDB.

The 9.002.0 release of OpenPages includes the following features and updates:

Create multiple report views

Administrators can now create multiple report views for object types, giving users more options for viewing and sharing information.

Create objects based on questionnaire responses

On a questionnaire template, a questionnaire author can now create response actions to create or copy an object and set field values on the object automatically. For example, in a risk assessment questionnaire, you might want to create issues based on the answers to the questions.

Configure a model to suggest relationships to create or copy

You can now configure a model to suggest relationships between objects for the user. A relationship can be new or copied from an existing relationship. For example, you can configure the model to make the following suggestions:

• When a user is creating a risk, the model can suggest a set of controls to copy from the library.
• When a user is creating an issue, the model can suggest a set of predefined action items from the library and a set of controls.

Associate objects in a workflow action

In a workflow, you can add a new Associate objects operation on an action. This operation creates a relationship between the current object and other objects based on conditions that you specify. For example, in the Questionnaire Assessment Workflow, you can automatically associate a questionnaire template with the current questionnaire assessment object.

Export and import questionnaire templates

You can now export and import questionnaire templates by using the Environment Migration tool. The items you can import and export include:

• Question Templates
• Questionnaire Templates
• Questionnaire Section Templates
• Questionnaire Subsection Templates

You can still use FastMap to import and export questionnaire assessment objects and the standard fields for questionnaire assessment instances. However, the FastMap export and import do not include the response actions and questionnaire dependency data.

Support for Db2 version 11.5.9

OpenPages 9.002.0 supports Db2 version 11.5.9.

For more information about these new features and other updates in OpenPages Version 9.002.0, see New features in Version 9.002.0 in the OpenPages documentation.

Version 9.002.0 of the OpenPages service includes various fixes.

Version 8.5.0 of the RStudio Server Runtimes service includes various fixes.

The 8.5.0 release of SPSS Modeler includes the following features and updates:

Connect to DataStax Enterprise

You can use the new DataStax Enterprise connector to connect SPSS Modeler to DataStax Enterprise, and your SPSS Modeler flows can have read and write access to the data source. For more information, see Supported data sources for SPSS Modeler.

Use timestamps with microseconds

For connectors that support SQL pushback, you can now use timestamps measured to an accuracy of six decimal places after the zero (0.000001). Your SPSS Modeler flows retain the more precise timestamp data. For information about using this new feature, see Setting properties for flows.

If you enable this new feature, you must manually open and save the Data Asset nodes that you want to use it on. For more information about this limitation, see Known issues and limitations for SPSS Modeler.

Version 8.5.0 of the SPSS Modeler service includes various fixes.

For details, see What's new and changed in SPSS Modeler.

The 8.5.0 release of Synthetic Data Generator includes the following features and updates:

Connect to DataStax Enterprise

You can use the new DataStax Enterprise connector to connect Synthetic Data Generator to DataStax Enterprise, and your flows can have read and write access to the data source. For more information, see Creating synthetic data from imported data.

Version 8.5.0 of the Synthetic Data Generator service includes various fixes.

Version 4.8.5 of the Watson Discovery service includes various fixes.

The 4.8.5 release of Watson Machine Learning includes the following features and updates:

Use the Cloud Pak for Data user interface for GPU inferencing

To accelerate the process of deploying machine learning models in a deployment space, you can now use the Cloud Pak for Data user interface to create deployments that use GPU inferencing for certain runtimes and software specifications for the x86 platform. For more information, see Managing hardware specifications for deployments.

Version 4.8.5 of the Watson Machine Learning service includes various fixes.

Version 4.8.5 of the Watson Machine Learning Accelerator service includes various fixes.

Version 4.8.5 of the Watson OpenScale service includes various fixes.

For details, see What's new and changed in Watson OpenScale.

The 4.8.5 release of Watson Pipelines includes the following features and updates:

Increased scalability of pipelines

All pipeline configuration sizes have been updated to support a greater number of nodes that can run in active pipelines simultaneously. For more details, download the guidance file for scaling components at Downloading the component scaling guidance PDF for entitled customers, or see Known issues and limitations for Watson Pipelines.

Version 4.8.5 of the Watson Pipelines service includes various fixes.

The 2.2.5 release of Watson Query includes the following features and updates:

Troubleshoot queries that fail during the fetch phase

You can now use the information in fetch phase errors to determine why queries fail. When a problem, such as a connection error, occurs during the fetch phase, the query stops and the error is sent back to you. You can check the SQL state that is linked to the error to find out why your query stopped.

Use fetch phase errors in addition to fetch phase warnings to gain insight into problems or potential problems with your queries.

For more information, see Fetch phase warnings and errors.

Query performance improvements

Query performance is improved for queries that join data between multiple tables by using a VARCHAR or character keys.

Version 2.2.5 of the Watson Query service includes various fixes.

For details, see What's new and changed in Watson Query.

Version 4.8.5 of the Watson Speech to Text service includes various fixes.

Version 8.5.0 of the Watson Studio service includes various fixes.

For details, see What's new and changed in Watson Studio.

The 8.5.0 release of Watson Studio Runtimes includes the following features and updates:

Notebook images based on runtime 22.2 are no longer mirrored by default to images in container registries

Starting with Cloud Pak for Data release 4.8.5, notebook images based on runtime 22.2 (deprecated in release 4.8.4) are not mirrored by default to images in your container registries. If you want to keep using notebook images based on runtime 22.2, you must mirror them, as described in Mirroring images using an intermediary container registry and Mirroring images directly to the private container registry.

Version 4.8.5 of the watsonx Assistant service includes various security fixes.

The 8.5.0 release of watsonx.ai includes the following features and updates:

Find targeted watsonx.ai and watsonx.governance documentation quickly

Your team can now navigate and search a smaller, targeted documentation set for their watsonx.ai and watsonx.governance tasks. See the IBM watsonx.ai and watsonx.governance 1.1.x documentation. This content is also available in the Cloud Pak for Data documentation.

Use Chat mode in Prompt Lab to inference models through conversational interactions

Chat mode in Prompt Lab is a simple chat interface that you can use to experiment with foundation models and simulate question-answering or conversational interactions for chatbot and virtual assistant use cases. The existing Structured and Freeform modes are useful when building few-shot or many-shot prompts for tasks such as extraction, summarization, and classification. Chat mode remembers the context of the conversation and each response builds on information from previous prompts.

For more information, see Prompt Lab.

New watsonx.ai end-to-end use case tutorial and video

Try the new tutorial to learn how to use watsonx.ai in an end-to-end use case starting from data preparation through to prompt engineering. For more information, see Try the watsonx.ai end-to-end use case

Prompt-tune the granite-13b-instruct-v2 model

You can tune the granite-13b-instruct-v2 foundation model by using the Tuning Studio. For more information, see Tuning Studio.

Work with new foundation models in Prompt Lab

You can now use the following foundation models for inferencing from the Prompt Lab in watsonx.ai:

jais-13b-chat

The jais-13b-chat foundation model is provided by Inception, Mohamed bin Zayed University of Artificial Intelligence, and Cerebras Systems. This foundation model specializes in conversational tasks in Arabic and English. You can also use the jais-13b-chat foundation model for general purpose tasks in the Arabic language, such as language translation between Arabic and English.

llama2-13b-dpo-v7

The llama2-13b-dpo-v7 foundation model from Meta is trained to understand and generate Korean text. You can use the llama2-13b-dpo-v7 foundation model for general-purpose tasks in the Korean language, such as classification, extraction, question-answering, and for language translation between Korean and English.

For more information, see Supported foundation models.

Version 8.5.0 of the watsonx.ai service includes various fixes.

For details, see What's new and changed in watsonx.ai.

The 1.1.4 release of watsonx.data includes the following features and updates:

Uploading description files for Apache Kafka data source

The Apache Kafka data source stores data as byte messages that producers and consumers must interpret. To query this data, consumers must first map it into columns. Now you can upload topic description files that convert raw data into a table format. Each file must be a JSON file that contains a definition for a table.

To upload these JSON files from the UI, go to the overview page of the Apache Kafka database that you registered and select the Add topic option.

New data sources

The following new data sources are now available:

• Oracle
• Amazon Redshift
• Informix
• Prometheus

New BINARY data type for data sources

In the Query workspace, you can now use the BINARY data type and its equivalents with the SELECT statement to build and run queries against your data for the following data sources:

• Db2 (BINARY data type)
• Snowflake (BINARY data type)
• PostgreSQL (BYTEA data type)
• Teradata (VARBYTE data type)

Test SSL connections

You can now test SSL connections for the MongoDB and SingleStore data sources.

Kerberos authentication for HDFS connections

You can now enable Kerberos authentication for secure Apache Hadoop Distributed File System (HDFS) connections.

Presto engine version upgrade

The Presto engine is now upgraded to version 0.285.1.

Version 1.1.4 of the watsonx.data service includes various fixes.

The 1.1.5 release of watsonx.governance includes the following features and updates:

Find targeted watsonx.ai and watsonx.governance documentation quickly

Your team can now navigate and search a smaller, targeted documentation set for their watsonx.ai and watsonx.governance tasks. See the IBM watsonx.ai and watsonx.governance 1.1.x documentation. This content is also available in the Cloud Pak for Data documentation.

Version 1.1.5 of the watsonx.governance service includes various fixes.

Version 1.1.0 of the watsonx Orchestrate service includes various fixes.

Refresh 4 of Version 4.8 platform includes various fixes.

The 13.1.4 release of the Cloud Pak for Data command-line interface includes the following features and updates:

Validate network performance and access

You can use the cpd-cli health network-performance command to run network health checks with iPerf on a Cloud Pak for Data deployment on a Red Hat® OpenShift® cluster. These checks include detecting issues that are related to network performance and network access failures.

Version 13.1.4 of the Cloud Pak for Data command-line interface includes various fixes.

For details, see What's new and changed in the Cloud Pak for Data command-line interface.

The 8.4.0 release of the common core services includes changes to support features and updates in Watson Studio and IBM Knowledge Catalog.

Version 8.4.0 of the common core services includes various fixes.

For details, see What's new and changed in the common core services.

Version 1.21.0 of the scheduling service includes various security fixes.

The 4.8.4 release of AI Factsheets includes the following features and updates:

Factsheets organization is now updated for machine learning models

Factsheets have a new design for presenting the facts for native or external machine learning models to make it easier for you to review the lifecycle activity for a tracked model.

Version 4.8.4 of the AI Factsheets service includes various fixes.

Version 4.8.4 of the Analytics Engine powered by Apache Spark service includes various fixes.

The 25.4.0 release of Cognos Analytics includes the following features and updates:

Online backup and restore

You can now create online backups of the Cognos Analytics operator and operand. This allows you to easily remove the instance record of instances that are not backed up. For more information, see Backing up and restoring your Cognos Analytics deployment.

Manage images

You can now upload, download, and delete images in your Cognos Analytics service instance. You can also create and delete soft links to these images. For more information, see Manage images in Cognos Analytics.

Updated software version for Cognos Analytics

This release of the service provides Version 12.0.2 + IF 1005 of the Cognos Analytics software. For more information, see Release 12.0.2 - New and changed features in the Cognos Analytics documentation.

Version 25.4.0 of the Cognos Analytics service includes various fixes.

The 4.8.4 release of Cognos Dashboards includes the following features and updates:

Updated software version

This release of the service provides Version 12.0.2 + IF 1005 of the Cognos Analytics dashboards software. For details, see Release 12.0.2 - Dashboards in the Cognos Analytics documentation.

Version 4.8.4 of the Cognos Dashboards service includes various fixes.

Version 4.8.4 of the Data Privacy service includes various fixes.

For details, see What's new and changed in Data Privacy.

Version 8.4.0 of the Data Refinery service includes various fixes.

For details, see What's new and changed in Data Refinery.

Version 4.8.4 of the Data Replication service includes various fixes.

For details, see What's new and changed in Data Replication.

The 4.8.4 release of DataStage includes the following features and updates:

Run your flows with pushdown to source

You can now run flows more efficiently by selecting a compile option that pushes transformation logic to the source database and performs work before extraction. Running in TETL mode executes the flow in Transform, Extract, Transform, Load order. For more information, see Compile options with SQL Pushdown.

Access data lake repositories with the Amazon S3 and the Apache HDFS connectors

DataStage now includes properties for the Iceberg and Delta Lake table formats. You can use these table formats to manage data in a data lake. Data lakes are built to contain large amounts of structured or unstructured data that can grow over time. See the Amazon S3 connector and the Apache HDFS connector for the setting information.

Version 4.8.4 of the DataStage service includes various fixes.

For details, see What's new and changed in DataStage.

Version 4.8.4 of the Db2 service includes various fixes.

For details, see What's new and changed in Db2.

Version 7.6.4 of the Db2 Big SQL service includes various fixes.

The 5.2.0 release of Db2 Data Gate includes the following features and updates:

Use the information in CSV files to synchronize data in Db2 Data Gate

You can now import a CSV file that contains the information from tables in the Db2 Data Gate web UI. You can import a CSV file that was exported from the same Db2 Data Gate instance or from an earlier instance, and you can synchronize data based on the information in the CSV file. For more information, see Importing tables into Db2 Data Gate.

Version 5.2.0 of the Db2 Data Gate service includes various fixes.

For details, see What's new and changed in Db2 Data Gate.

Version 4.8.4 of the Db2 Data Management Console service includes various fixes.

For details, see What's new and changed in Db2 Data Management Console.

Version 4.8.4 of the Db2 Warehouse service includes various fixes.

For details, see What's new and changed in Db2 Warehouse.

Version 8.4.0 of the Decision Optimization service includes various fixes.

For details, see What's new and changed in Decision Optimization.

This release of the EDB Postgres service includes various fixes.

For details, see What's new and changed in EDB Postgres.

Version 4.8.3 of the Execution Engine for Apache Hadoop service includes various fixes.

For details, see What's new and changed in Execution Engine for Apache Hadoop.

The 4.8.4 release of IBM Knowledge Catalog includes the following features and updates:

Review and manage data class and term assignments in a spreadsheet

If you prefer to work in a familiar spreadsheet program when you review and update metadata enrichment results, you can now install the new Review metadata add-in for Microsoft Excel. Use the spreadsheet template provided with the product in combination with the add-in:

• To download enriched data assets from a specific project and metadata enrichment.
• To review and update suggested and assigned data classes and terms for these data assets.
• To upload the updated data assets to the project.

For more information, see Reviewing and updating enrichment results in an external program.

Create dynamic views of connected data

A new type of connected data asset provides filtered access to data from data sources that support SQL queries so you can access only relevant data. In a project, provide an SQL query to create a view of specific columns or rows from one or more tables. You can use these data assets in metadata enrichment and data quality analysis just like any other connected data asset.

For more information, see Adding a dynamic view of connected data to a project.

Email notifications can now be sent when metadata is reimported

When a metadata import job is rerun, you can now receive an email notification with a brief summary about the data that changed with this reimport. For more information, see Rerunning the import.

Lineage metadata can now be imported from additional data sources

You can now import lineage metadata from IBM Db2 for z/OS® with the Get lineage import goal. For more information, see Supported data sources for metadata import, metadata enrichment, and data quality rules.

Custom relationships between governance artifacts are now viewable in lineage

You can now view custom relationships between governance artifacts in your lineage graph. By viewing custom relationships, you can get a more complete view of your data's lineage. For more information, see Lineage.

Import metadata from Apache Hive connections that use Kerberos SSO authentication

You can now import metadata from Apache Hive connections that are configured to use Kerberos SSO authentication. For more information, see Apache Hive connection.

Version 4.8.4 of the IBM Knowledge Catalog service includes various fixes.

For details, see What's new and changed in IBM Knowledge Catalog.

The 3.6.11 release of IBM Match 360 includes the following features and updates:

Improved record data export capabilities through the REST API

You can now use the IBM Match 360 REST API to export record data from a specified point in time instead of exporting the full data set. Your exported data can also now include information about deleted records within a particular time frame. This enhanced export capability for record data mirrors the Cloud Pak for Data 4.8.3 improvements for exporting entity data.

For more information about using the IBM Match 360 API to export record or entity data from a specified point in time, see Advanced master data export.

Version 3.6.11 of the IBM Match 360 service includes various fixes.

For details, see What's new and changed in IBM Match 360.

Version 7.2.0 of the Informix service includes various fixes.

Version 42.3.9 of the MANTA Automated Data Lineage service includes various fixes.

For details, see What's new and changed in MANTA Automated Data Lineage.

This release of the MongoDB service include various fixes.

For details, see What's new and changed in MongoDB.

The 4.8.4 release of Planning Analytics includes the following features and updates:

Updated versions of Planning Analytics software

The 4.8.4 release of the service provides the following software versions:

• Planning Analytics Workspace Version 2.0.94

  For information about this version of the software, see 2.0.94 - What's new in the Planning Analytics Workspace documentation.

• Planning Analytics Spreadsheet Services Version 2.0.94

  For information about this version of the software, see 2.0.94 - Feature updates in the TM1® Web documentation.

• Planning Analytics for Microsoft Excel Version 2.0.94

  For information about this version of the software, see 2.0.94 - Feature updates in the Planning Analytics for Microsoft Excel documentation.

• Planning Analytics Engine Version 12.3.9

  For information about this version of the software, see What's new in Planning Analytics Engine in the Planning Analytics Engine documentation.

Version 4.8.4 of the Planning Analytics service includes various fixes.

Version 5.4.0 of the Product Master service includes various fixes.

For details, see What's new and changed in Product Master.

The 8.4.0 release of RStudio Server Runtimes includes the following features and updates:

Deprecation of Runtime 22.2

Runtime 22.2 in RStudio Server Runtimes is now deprecated and will be removed in a future release. To avoid disruption when the removal is completed, update your RStudio environment to use Runtime 23.1.

For information about changing environments, see Changing notebook environments.

Version 8.4.0 of the RStudio Server Runtimes service includes various fixes.

The 8.4.0 release of SPSS Modeler includes the following features and updates:

You can now use timestamps with milliseconds

Your SPSS Modeler flows can now include timestamps with time measured in milliseconds. Previously, nodes in SPSS Modeler flows would truncate timestamps to seconds. The SPSS Modeler flow now retains and uses the more precise time data.

Version 8.4.0 of the SPSS Modeler service includes various fixes.

For details, see What's new and changed in SPSS Modeler.

Version 8.4.0 of the Synthetic Data Generator service includes various fixes.

Version 4.8.4 of the Voice Gateway service includes various fixes.

Version 4.8.4 of the Watson Discovery service includes various fixes.

The 4.8.4 release of Watson Machine Learning includes the following features and updates:

Exercise greater control over low-importance features in AutoAI experiments

When you configure a classification or regression experiment, you can now optionally specify how to handle features that have no impact on the model. The choices are to:

• Always remove features with no model impact
• Remove features only when it improves the model quality
• Do not remove features

For more information, see Configuring an experiment.

GPU inferencing for Watson Machine Learning deployments

To accelerate the process of deploying machine learning models in a deployment space to GPU, you can now create deployments that use GPU inferencing for certain runtimes and software specifications in Watson Machine Learning API. To learn more about CUDA software specifications that support GPU inferencing, see Supported machine learning tools, libraries, frameworks, and software specifications.

GPU inferencing requires MIG support in Red Hat OpenShift. To learn how to configure MIG support in OpenShift, see Configuring MIG support in Red Hat OpenShift.

Reusing Watson Studio base runtime image for Watson Machine Learning

You can now reuse a Watson Studio notebook runtime for Watson Machine Learning as the base image for an x86 platform to make sure that you use the same environment for creating and deploying the notebook.

For more information, see Customizing Python deployment images in Watson Machine Learning.

Deprecation of Runtime 22.2

IBM Runtime 22.2 is deprecated and will be discontinued in a future release. Update your deployed assets to use Runtime 23.1. For more information, see Supported machine learning tools, libraries, frameworks, and software specifications.

Version 4.8.4 of the Watson Machine Learning service includes various fixes.

Version 4.8.4 of the Watson Machine Learning Accelerator service includes various fixes.

The 4.8.4 release of Watson OpenScale includes the following features and updates:

Generate SHAP explanations for unstructured text models with Python SDK

View or run a sample notebook to learn how use the Watson OpenScale Python SDK to generate SHAP explanations for unstructured text models. For more information, see Tutorial on generating an explanation for a text-based model on Watson OpenScale.

Version 4.8.4 of the Watson OpenScale service includes various fixes.

For details, see What's new and changed in Watson OpenScale.

The 4.8.4 release of Watson Pipelines includes the following features and updates:

Error messages are now available as node output

You can now save standard error messages from the Run Bash node as output. Then you can use the error message as an input to create conditional functions and scripts. You can save and reuse error messages regardless of the error policy configured for your pipelines. For more information, see Run Bash script in Configuring pipeline nodes.

View additional parameters for the pipeline

When you create a new pipeline job, you can now view the number of parameters and parameter sets that are referenced in the pipeline for greater clarity regarding which parameters are impacting the pipeline. Parameters that are not created locally or not referenced in the pipeline will not be included in the count.

Open multiple pipelines with a single click

In your Watson Studio project, you can now select multiple pipelines and open them with a single click, making it faster to view a set of pipelines. Each pipeline displays in its own tab in the browser.

Version 4.8.4 of the Watson Pipelines service includes various fixes.

Version 2.2.4 of the Watson Query service includes various fixes.

For details, see What's new and changed in Watson Query.

Version 4.8.4 of the Watson Speech to Text service includes various fixes.

The 8.4.0 release of Watson Studio includes the following features and updates:

Load and access data from IBM watsonx.data

When you work with Juypter Notebooks in Watson Studio, you can now use code snippets to load and access data from watsonx.data.

For more information, see Loading and accessing data in a notebook.

Version 8.4.0 of the Watson Studio service includes various fixes.

For details, see What's new and changed in Watson Studio.

The 8.4.0 release of Watson Studio Runtimes includes the following features and updates:

Runtime 22.2 is deprecated

Runtime 22.2 in Watson Studio notebooks is now deprecated and will be removed in a future release. To avoid disruption when the removal is completed, update your notebook environment to use Runtime 23.1.

For information about changing environments, see Changing notebook environments.

The com.ibm.connect.spark data source is deprecated

The com.ibm.connect.spark data source is now deprecated and will be removed in a future release. If you used com.ibm.connect.spark in one of your notebooks based on Spark, you must update the code. Use the com.ibm.connect.spark.flight data source instead.

Version 8.4.0 of the Watson Studio Runtimes service includes various fixes.

The 4.8.4 release of watsonx Assistant includes the following features and updates:

Algorithm version Latest(15 Apr-2023) uses improved intent detection and matching

The Latest(15 Apr-2023) algorithm version uses a new foundation model to improve the intent detection and action matching in assistants with languages such as English, French, German, Portuguese (Brazilian), Spanish, Arabic, Chinese (Simplified), Chinese (Traditional), Czech, Dutch, Italian, Japanese, and Korean. The new foundation model is trained by using the transformer architecture. For more information, see Algorithm version and training in the watsonx Assistant documentation.

Change in backup PersistentVolumeClaim (PVC) name

The backup PersistentVolumeClaim (PVC) name is changed from store-pvc to store-db-backup-pvc. For more information, see Backing up and restoring data in the watsonx Assistant documentation.

Change in the default size of PersistentVolumeClaim (PVC)

The default size of PersistentVolumeClaim (PVC) is now 10Gi, which was 1Gi in the previous releases. For more information, see Backing up and restoring data in the watsonx Assistant documentation.

Version 4.8.4 release of watsonx Assistant service includes various security fixes.

The 8.4.0 release of watsonx.ai includes the following features and updates:

Bring your own foundation model to inference from watsonx.ai

In addition to working with foundation models that are curated by IBM, you can now upload and deploy your own foundational models. After the models are deployed and registered with watsonx.ai, create prompts that inference the custom models from the Prompt Lab. To learn more about uploading custom foundation models, see Deploying custom foundation models. Also described is a list of supported GPU configurations, model architectures, and suggested matching hardware configurations.

The watsonx.ai API is available for use

You can now programmatically work with foundation models that are hosted in watsonx.ai by calling the watsonx.ai API directly. For more information, see the watsonx.ai API reference documentation.

Prompt-tune the llama-2-13b-chat foundation model

The Tuning Studio now supports tuning the llama-2-13b-chat foundation model. Start by engineering prompts for the larger llama-2-70b-chat model in the Prompt Lab to find effective prompt inputs for your use case. Then tune the smaller version of the Llama 2 model to generate comparable or better outputs with zero-shot prompts. For more information, see Tuning Studio.

A modification to the granite-13b-chat-v2 foundation model is available

The latest version of the granite-13b-chat-v2 is 2.1.0. The modification includes improvements that were gained by applying a novel AI alignment technique to the version 2.0.0 model. AI alignment involves using fine-tuning and reinforcement learning techniques to guide the model to return outputs that are as helpful, truthful, and transparent as possible. For more information, see the What is AI alignment? blog post from IBM Research.

If you saved a prompt that inferences the granite-13b-chat-v2 foundation model as a template or notebook prior to this release, you might need to edit the asset to account for changes that were made to how the model handles white spaces.

Work with new foundation models in Prompt Lab

You can now use the following foundation models for inferencing from the Prompt Lab in watsonx.ai:

• granite-8b-japanese: A foundation model from the IBM Granite family that is trained to understand and generate Japanese text. You can use the granite-8b-japanese foundation model for general purpose tasks in the Japanese language, such as classification, extraction, question-answering, and for language translation between Japanese and English.
• granite-20b-multilingual: A foundation model from the IBM Granite family that is trained for translation tasks in English, German, Spanish, French, and Portuguese.
• codellama-34b-instruct: A programmatic code generation model from Code Llama that is based on Llama 2 from Meta. Use Code Llama to create prompts for generating code based on natural language inputs, and for completing and debugging code.
• mixtral-8x7b-instruct-v01-q: A version of the Mixtral 8x7B Instruct foundation model from Mistral AI that is quanitzed by IBM. You can use this new model for general-purpose tasks, including classification, summarization, code generation, language translation, and more.

For more information, see Supported foundation models.

Deprecated foundation models

The following models are now deprecated and will be withdrawn in a future release:

• granite-13b-chat-v1
• granite-13b-instruction-v1
• gpt-neox-20b
• mpt-7b-instruct2
• starcoder-15.5b

To avoid disruption when the withdrawal is completed, use a suitable alternative model as a replacement. For more information, see Foundation model lifecycle.

Version 8.4.0 of the watsonx.ai service includes various fixes.

For details, see What's new and changed in watsonx.ai.

The 1.1.3 release of watsonx.data includes the following features and updates:

New data types for some data sources

You can now use the BINARY data type with the SELECT statement in the Query workspace to build and run queries against data from the following data sources:

• Elasticsearch
• SAP HANA
• Microsoft SQL Server
• MySQL

You can now use the BLOB and CLOB data types with the SELECT statement in the Query workspace to build and run queries against data from the following data sources:

• MySQL
• PostgreSQL
• Snowflake
• Microsoft SQL Server
• Db2

Delete data by using the DELETE FROM feature for Apache Iceberg data sources

You can now delete data from tables from Apache Iceberg data sources by using the DELETE FROM feature. You can use copy-on-write mode or merge-on-read mode to delete data. For more information, see Supported SQL statements.

ALTER VIEW statement for Iceberg data source

You can now use the following SQL statement in the Query workspace to build and run queries against your data for ALTER VIEW:

ALTER VIEW name RENAME TO new_name

Upload SSL certificates for IBM Netezza® Performance Server data sources

You can now use the Infrastructure Manager in the watsonx.data console to browse for and upload SSL certificates for SSL connections to IBM Netezza Performance Server data sources. The valid file formats for SSL certificates are .pem, .crt, and .cer.

Query data from Db2 and Watson Query

You can now query nicknames that are created in Db2 and virtualized tables from Watson Query instances.

Use data from Apache Hudi data sources

You can now connect to and use data from Apache Hudi data sources.

Add Milvus as a service in watsonx.data

You can now provision Milvus as a service in watsonx.data. You can provision different storage variants such as starter, medium, and large nodes, and you can assign Administrator or User roles for Milvus users.

Use time-travel queries in Apache Iceberg tables

You can now run the following time-travel queries by using branches and tags in Apache Iceberg table snapshots:

• SELECT *FROM <table name> FOR VERSION AS OF 'historical-tag'
• SELECT *FROM <table name> FOR VERSION AS OF 'test-branch'

Load data in batch by using the ibm-lh ingestion tool

You can now use the ibm-lh ingestion tool to run batch ingestion procedures in non-interactive mode, from outside the ibm-lh-tools container, by using the ibm-lh-client package. For more information, see ibm-lh commands and usage.

Creating schemas by using bulk ingestion in web console

You can now create a schema by using the bulk ingestion process in the web console, if the schema is not previously created.

Data ingestion is possible only through Spark engine from web console

Iceberg copy loader method of ingesting data is no longer available on the web console. Now, data ingestion through Spark engine is the only available method of ingestion through web console. For more information, see Ingesting data by using Spark.

New storage type Hadoop Distributed File System (HDFS)

You can now use new storage type HDFS and attach a bucket and catalog to it. For more information, see For more information, see Hadoop Distributed File System (HDFS).

Add bucket feature is now available in Add storage option

You can now add a storage and attach a bucket to the respective storage type in the Infrastructure manager in the UI.

Version of the watsonx.data service includes various fixes.

1.1.3

The 1.1.4 release of watsonx.governance includes the following features and updates:

AI Factsheets are now consistent for machine learning models and prompt templates

The design of factsheets that display facts for native and external machine learning models is now updated to match the design of factsheets that display facts for prompt templates.

Version 1.1.4 of the watsonx.governance service includes various fixes.

Introducing watsonx Orchestrate

Use your time wisely. Streamline repetitive work so that you can focus on the work that matters.

Harness the power of generative AI (gen AI) to simplify complex processes, automate tasks, and increase productivity. IBM watsonx Orchestrate helps users complete daily tasks quickly and efficiently.

You can connect watsonx Orchestrate with other applications to enable automation and support user workflows.

Prebuilt skills and application connectors

IBM watsonx Orchestrate includes prebuilt connectors for common applications, such as Gmail and Microsoft Outlook. The software also includes a prebuilt catalog of skills. For example, users can use the chat interface to start skills that automate simple tasks, such as emailing coworkers, or complex tasks, such as creating cases in CRM platforms or managing talent acquisition processes.

Low-code tools for custom applications and skills

IBM watsonx Orchestrate includes low-code tools that you can use to create connections to custom applications and skills:

• Add custom applications and skills by connecting to prebuilt applications or by using an OpenAPI specification file.
• Use the assistant builder to build conversational AI-powered chatbots. You can use the included templates or start from scratch.

Version 4.8.3 includes security fixes for the platform operator.

The 13.1.3 release of the Cloud Pak for Data command-line interface includes the following features and updates:

Download CASE packages from a private Open Container Initiative registry

If you use the --from_oci option to download CASE packages, you can use the new --oci_location option to download the packages from a private Open Container Initiative registry rather than the IBM Cloud Pak Open Container Initiative registry (cp.icr.io/cpopen).

The --oci_location option is available for the following commands:

• apply-cluster-components
• apply-olm
• apply-scheduler
• case-download
• migrate-scheduler
• mirror-images
• setup-instance-topology
• upgrade-cpfs

Version 13.1.3 of the Cloud Pak for Data command-line interface includes various fixes.

For details, see What's new and changed in the Cloud Pak for Data command-line interface.

The 8.3.0 release of the common core services includes changes to support features and updates in Watson Studio and IBM Knowledge Catalog.

Version 8.3.0 of the common core services includes various fixes.

For details, see What's new and changed in the common core services.

Version 1.20.0 of the scheduling service includes various fixes.

For details, see What's new and changed in the scheduling service.

The 4.8.3 release of AI Factsheets includes the following features and updates:

No dependency on Watson Studio or IBM Knowledge Catalog

You no longer need to have Watson Studio or IBM Knowledge Catalog to install AI Factsheets. Inventories for AI use cases are now stored in the platform catalogs. For more information, see Installing AI Factsheets.

New UI for managing use cases and inventories

The UI and navigation for AI Factsheets are new in this release:

• Model use cases are now called AI use cases.
• You can now access use cases and external models from the AI Governance section of the main navigation menu.
• Manage inventory settings from the AI use case settings page.

For more information, see Governing assets in AI use cases.

Choose an inventory for external models

External models are no longer limited to the Platform assets catalog. You can now select an inventory for storing models that are created with a third-party provider.

For more information, see Governing assets in AI use cases.

Version 4.8.3 of the AI Factsheets service includes various fixes.

The 4.8.3 release of Analytics Engine powered by Apache Spark includes the following features and updates:

Select Red Hat OpenShift Container Platform cluster nodes for Spark runtimes

The administrators can now select the nodes in the Red Hat OpenShift Container Platform cluster to deploy the Spark runtime based on the Spark workload. For more information, see Running Spark applications interactively.

Configure Spark jobs and kernels

Administrators can now specify one of two configuration types for all the Spark kernels for the Analytics Engine service:

Mutable

This configuration can be overridden for a specific instance or kernel.

Immutable

This configuration cannot be overridden for a specific instance or kernel.

For more information, see Adding custom configurations.

Version 4.8.3 of the Analytics Engine powered by Apache Spark service includes various fixes.

For details, see What's new and changed in Analytics Engine powered by Apache Spark.

Version 25.3.0 of the Cognos Analytics service includes various fixes.

Version 4.8.3 of the Cognos Dashboards service includes various fixes.

Version 4.8.3 of the Data Privacy service includes various fixes.

For details, see What's new and changed in Data Privacy.

The 8.3.0 release of Data Refinery includes the following features and updates:

New Spark 3.4 environment for running Data Refinery flow jobs

When you select an environment for a Data Refinery flow job, you can now select Default Spark 3.4 & R 4.2, which includes enhancements from Spark.

The Default Spark 3.3 & R 4.2 environment is deprecated and will be removed in a future update.

Update your Data Refinery flow jobs to use the new Default Spark 3.4 & R 4.2 environment. For more information, see Data Refinery environments.

Version 8.3.0 of the Data Refinery service includes various fixes.

For details, see What's new and changed in Data Refinery.

Version 4.8.3 of the Data Replication service includes various fixes.

For details, see What's new and changed in Data Replication.

The 4.8.3 release of DataStage includes the following features and updates:

Run transactions that span multiple data sources with the new Distributed Transaction stage

You can use the Distributed Transaction stage to run transactions that are managed by IBM MQ or Apache Kafka. For more information, see Distributed Transaction stage.

Manage your assets with new features

You can use the following new features to build and compile your flows:

• Use the Replace action to replace subflows in a flow.
• Select and compile multiple flows with a click.
• View flow compile status under the Assets tab.

For more information, see Creating and managing DataStage assets.

Version 4.8.3 of the DataStage service includes various fixes.

For details, see What's new and changed in DataStage.

Version 8.3.0 of the Decision Optimization service includes various fixes.

For details, see What's new and changed in Decision Optimization.

This release of the EDB Postgres service includes various fixes.

Version 4.8.3 of the Execution Engine for Apache Hadoop service includes various fixes.

For details, see What's new and changed in Execution Engine for Apache Hadoop.

The 4.8.3 release of IBM Knowledge Catalog includes the following features and updates:

View native type information in the details panel for asset columns

Now, you can view both standardized and native data types directly in the column details panel. To view the native type information, click an asset column name from the Overview page of an asset.

Save your searches for catalog assets

Each user can now save up to 25 searches within each of their catalogs. The user who saves a search in a catalog is the only user who can view, run, edit, and remove the search. For more information, see Saving searches for catalog assets.

Select a database or connection as the scope of metadata import

When you import metadata with the Discover goal, you can select an entire database or connection as the scope of the import. As a result, when new schemas are added in the data source, the schemas are imported to IBM Knowledge Catalog next time the import job runs, or when you reimport assets manually. For more information, see Scope of import.

The code column in reference data sets is now case-sensitive

Reference data values consist of at least two columns: code and value. For all new reference data sets, the code column is now case-sensitive. When you add values to a new reference data set, the code is saved exactly as you type it. Reference data sets that were created before this change was introduced are case-insensitive, and any new values added there are saved in uppercase. These reference data sets are marked with a Case-insensitive tag in the UI. For more information, see Case-sensitive code.

New search, filter, and sort options for reference data sets

When you view a list of reference data values, you can use the following methods to find the required values faster:

• Use a search bar to type a query for a code, value, or a custom column value.
• Use one of the 6 advanced filter options.
• Use the sorting feature.

The search, filter, and sort options can be combined. For more information, see Viewing reference data sets.

Version 4.8.3 of the IBM Knowledge Catalog service includes various fixes.

For details, see What's new and changed in IBM Knowledge Catalog.

The 3.5.14 release of IBM Match 360 includes the following features and updates:

Gain finer control over matching by defining different matching thresholds that are based on the record source

Data Engineers can now define clerical review thresholds and autolink thresholds within the matching algorithm for each record source in your data. With source-level thresholds, your organization can weigh the data from trusted sources more heavily than data from less trusted sources, or even exclude some sources from matching altogether.

For example, you might want to configure a lower clerical review threshold and a higher autolink threshold for a source that is less trusted. The lower clerical review threshold means that comparisons that involve records from that source are more likely to require human review. The higher autolink threshold means that they must achieve a higher matching score to be automatically linked into an entity.

In this release, you can use the IBM Match 360 REST API to define the matching thresholds for each source.

For more information about defining source-level thresholds, see Advanced matching algorithm tuning. For information about the REST API, see the IBM Match 360 API documentation.

Improved data export capabilities through the REST API

You can now use the IBM Match 360 REST API to export entity data from a specified point in time instead of exporting the full data set. You can also now include information about deleted entities within a specific time frame.

The ability to refine exported data based on a timestamp provides your organization with a way to more efficiently synchronize entity information with downstream systems. This capability complements both configurations that use bulk data exports and configurations that use ongoing data streaming to downstream systems.

• In bulk export scenarios, instead of having to resynchronize the full downstream data set, you can use a smaller exported data set that captures the changes since the last synchronization.
• In streaming scenarios, you can now catch up your downstream entity data in a bulk manner so that changes made during a streaming downtime aren't lost.

For more information about using the IBM Match 360 REST API to export entity data from a specified time, see Advanced master data export. For information about the REST API, see the IBM Match 360 API documentation.

Version 3.5.14 of the IBM Match 360 service includes various fixes.

For details, see What's new and changed in IBM Match 360.

Version 42.3.0 of the MANTA Automated Data Lineage service includes various fixes.

For details, see What's new and changed in MANTA Automated Data Lineage.

This release of the MongoDB service include various fixes.

The 9.001.0 release of the OpenPages service includes features and updates from OpenPages Version 9.0.0.1.

Some of the highlights include:

Add a custom welcome message to the Home page

Administrators can now customize the welcome message on the OpenPages Home page to give users a more personalized experience.

Create Report views for object types

You can now create Report views to display specific information that users need. Report views are displayed as a tab in the Task view of an object, along with the Activity, and Admin tabs. Report views are read-only and can be printed to PDF for simplified governance and regulatory compliance (GRC) activity sharing.

Print or export views

You can now print Task, Admin, and Report views to PDF. The PDF file includes rich text, workflow status, tags, links to objects, and all tabular grid information. The PDF also captures the date and time of the export for auditability.

You can also export Grid and Activity views to Microsoft Excel. You can then use Microsoft Excel to analyze and share the data.

Automatically update fields based on responses to questionnaires

You can now define response actions in questionnaire assessments.

A response action sets the value of fields on objects based on the answers that a respondent provides in a questionnaire assessment. For example, you can use response actions to assign vendors to a tier, classify assets as high, medium, or low, and enrich inventory information based on the answers that users provide in questionnaires.

For more information about these new features and other updates in OpenPages Version 9.001.0, see New features in version 9.0.0.1 in the OpenPages documentation.

Version 9.001.0 of the OpenPages service includes various fixes.

The 4.8.3 release of Planning Analytics includes the following features and updates:

Updated versions of Planning Analytics software

The 4.8.3 release of the service provides the following software versions:

• Planning Analytics Workspace Version 2.0.93

  For information about this version of the software, see 2.0.93 - What's new in the Planning Analytics Workspace documentation.

• Planning Analytics Spreadsheet Services Version 2.0.93

  For information about this version of the software, see 2.0.93 - Feature updates in the TM1 Web documentation.

• Planning Analytics for Microsoft Excel Version 2.0.93

  For information about this version of the software, see 2.0.93 - Feature updates in the Planning Analytics for Microsoft Excel documentation.

• Planning Analytics Engine Version 12.3.8

  For information about this version of the software, see What's new in Planning Analytics Engine in the Planning Analytics Engine documentation.

Version 4.8.3 of the Planning Analytics service includes various fixes.

Version 5.3.0 of the Product Master service includes various fixes.

For details, see What's new and changed in Product Master.

The 8.3.0 release of RStudio Server Runtimes includes the following features and updates:

New security protocol for RStudio Server runtimes that run on a FIPS-enabled cluster

Starting with this release of RStudio Server Runtimes, R code that runs on a FIPS-enabled cluster enforces secure connections to remote servers by using TLS 1.3 or TLS 1.2 with Extended Master Secret. If you're running this service on a FIPS-enabled cluster, make sure that all of your RScripts and R Shiny Apps are using connections to remote servers that support those protocols before upgrading. See Known issues on FIPS-enabled clusters for more details.

Version 8.3.0 of the RStudio Server Runtimes service includes various fixes.

For details, see What's new and changed in RStudio Server Runtimes.

The 8.3.0 release of SPSS Modeler includes the following features and updates:

Save field formatting in chart node

When you use the chart builder to visualize data from columns, any chart settings that you specify in the chart builder are now saved. For example, you can set an Age column to have a number format. When you run the chart, the field formats that you set for the columns are used. For more information about chart builder, see Charts node.

Set data set precedence for a Merge node with the Anti-join setting

When you run a Merge node with the Anti-join setting, you receive a set of records from the data set that has precedence. By default, the first data set that you connect to the Merge node in a Modeler flow always takes precedence. Now you can edit the Tags and order of input data sets setting in the Merge node to specify which data set has precedence. For more information about the Merge node and the Anti-join setting, see Merge node.

Version 8.3.0 of the SPSS Modeler service includes various fixes.

For details, see What's new and changed in SPSS Modeler.

Version 8.3.0 of the Synthetic Data Generator service includes various fixes.

Version 4.8.3 of the Watson Discovery service includes various fixes.

The 4.8.3 release of Watson Machine Learning includes the following features and updates:

Use ordered data for all AutoAI experiments

You can now specify ordered data for all AutoAI experiments rather than just for time-series experiments. Specifying order data gives you greater control over how training data is processed. You can specify whether your training data is ordered sequentially according to a row index. When input data is sequential, model performance is evaluated on the newest records instead of on a random sampling, and holdout data uses the most recent records of the set rather than random records.. For more information, see Configuring a classification or regression experiment.

Deploy models that are trained with Spark 3.4

You can now deploy machine learning models that are trained with Spark 3.4. For more information, see Supported machine learning tools, libraries, frameworks, and software specifications.

Version 4.8.3 of the Watson Machine Learning service includes various fixes.

Version 4.8.3 of the Watson Machine Learning Accelerator service includes various fixes.

The 4.8.3 release of Watson OpenScale includes the following features and updates:

Configure model health monitor evaluations

When you configure model evaluations, you can now set thresholds for model health metrics to help you understand your model performance. For more information, see Configuring model health monitor evaluations.

Evaluate model robustness with the adversarial accuracy metric

You can now use the adversarial accuracy metric to evaluate the robustness of machine learning models and protect your deployments from security threats. For more information, see Watson OpenScale Python Client Samples.

Generate SHAP explanations with unstructured text data

You can now use the Watson OpenScale Python SDK to generate SHAP explanations for unstructured text models to understand how outcomes are predicted.

Generate LIME explanations with multimodal data

You can now use the Watson OpenScale Python SDK to generate LIME explanations for models that contain features with structured and unstructured text data. Structured data can contain numeric and categorical data.

Version 4.8.3 of the Watson OpenScale service includes various fixes.

The 4.8.3 release of Watson Pipelines includes the following features and updates:

View up-to-date parameter results

After you run multiple pipeline jobs, you can now view the pipeline results of parameters that were used for each run. You can view these results in the node panel and view additional details such as job attributes and associated jobs. For more information, see Running and saving a pipeline.

Easily organize your pipeline flow with drag and drop

If you need to restructure your pipeline, you can now quickly update your pipeline flow by dragging a connector that links two nodes to connect to a different source or target node. If the new connection results in configuration problems, such as missing parameters, error notifications indicate what to correct. For more information, see Creating a pipeline.

Version 4.8.3 of the Watson Pipelines service includes various fixes.

Version 4.8.3 of the Watson Speech to Text service includes various fixes.

For details, see What's new and changed in Watson Speech to Text.

The 8.3.0 release of Watson Studio includes the following features and updates:

View more catalogs and assets when you publish

When you prepare to publish assets, you can choose to view 10 to 100 catalogs and assets per page with the pagination view. For more information, see Publishing assets to a catalog.

Version 8.3.0 of the Watson Studio service includes various fixes.

For details, see What's new and changed in Watson Studio.

The 8.3.0 release of Watson Studio Runtimes includes the following features and updates:

New security protocol for notebook runtimes that run on a FIPS-enabled cluster

Starting with this release of Watson Studio Runtimes, notebook runtimes that run on a FIPS-enabled cluster enforce secure connections to remote servers by using TLS 1.3 or TLS 1.2 with Extended Master Secret. If you're running this service on a FIPS-enabled cluster, make sure that all of your notebooks and Python scripts are using connections to remote servers that support those protocols before you upgrade. See Known issues on FIPS-enabled clusters for more details.

Version 8.3.0 of the Watson Studio Runtimes service includes various fixes.

For details, see What's new and changed in Watson Studio Runtimes.

Version 4.8.3 of the watsonx Assistant service includes various security fixes.

For details, see What's new and changed in watsonx Assistant.

The 8.3.0 release of watsonx.ai includes the following features and updates:

New Japanese-language Llama 2 foundation model for general-purpose tasks in Japanese

The elyza-japanese-llama-2-7b-instruct foundation model provided by ELYZA, Inc. is available in watsonx.ai. The elyza-japanese-llama-2-7b-instruct model is a version of the Llama 2 model from Meta that was trained to understand and generate Japanese text.

You can use this new model for general-purpose tasks. This model works well for Japanese-language classification and extraction and for text translation between Japanese and English.

For more information, see Supported foundation models.

Deploy prompt templates for foundation models

You can now save a prompt template as a reusable asset and use it to generate output from a foundation model. Deploy the prompt template to a deployment space to test it or to retrieve the endpoint to use in an application. For more information, see Deploying a prompt template.

Version 8.3.0 of the watsonx.ai service includes various fixes.

The 1.1.2 release of watsonx.data includes the following features and updates:

SSL connections for data sources

You can now enable SSL connections for the following data sources by using the Add database window to secure and encrypt the database connection:

• Db2
• IBM Data Virtualization Manager for z/OS
• PostgreSQL

For IBM Data Virtualization Manager for z/OS and PostgreSQL, select Validate certificate to validate whether the SSL certificate that is returned by the host is trusted.

For the IBM Data Virtualization Manager for z/OS data source, you can choose to provide the hostname in the SSL certificate.

For more information, see Adding a database-catalog pair.

Secure ingestion job history

Now users can view only their own ingestion job history. Administrators can view the ingestion job history for all users.

Use more SQL statements

You can now use the following SQL statements in the Query workspace to build and run queries against your data:

Apache Iceberg data sources

• CREATE VIEW
• DROP VIEW

MongoDB data sources

• DELETE

New data types for SAP HANA and Teradata data sources

SAP HANA and Teradata data sources now support BLOB and CLOB data types. You can use these data types only with SELECT statements in the Query workspace to build and run queries against your data.

Create a table during data ingestion

Previously, you had to have a target table in watsonx.data to ingest data. Now, when you ingest data from the Data Manager, you can create a new table directly from your parquet or CSV source files. You can create the table by using the following methods of ingestion:

• Ingesting data by using the Apache Iceberg copy loader.

  For more information, see Ingesting data by using Iceberg copy loader.

• Ingesting data by using Spark.

  For more information, see Ingesting data by using Spark.

Perform ALTER TABLE operations on a column

With an Apache Iceberg data source, you can now perform ALTER TABLE operations on a column for the following data type conversions:

• int to bigint
• float to double
• decimal to decimal, where the source decimal type has fewer digits than the converted decimal type.

Grant user access to Spark instance directly from watsonx.data

If you have the watsonx.data admin permission and you have access to an instance of Analytics Engine powered by Apache Spark in Cloud Pak for Data, you can provide user access directly from the watsonx.data user interface without switching to the Cloud Pak for Data user interface.

Better query performance by using sorted files

With an Apache Iceberg data source, you can generate sorted files, which reduce the query result latency and improve the performance of Presto.

Data in the Apache Iceberg table is sorted during the writing process within each file. You can configure the order to sort the data by using the sorted_by table property. When you create the table, specify the array of columns involved in sorting.

Version 1.1.2 of the watsonx.data service includes various fixes.

For details, see What's new and changed in watsonx.data.

Introducing watsonx.governance

Accelerate responsible, transparent, and explainable AI workflows for generative AI models and machine learning models with the integrated tools provided by watsonx.governance.

When you install watsonx.governance, you get the capabilities of Watson OpenScale and AI Factsheets, and you get the Model Risk Governance features of OpenPages in a single service. watsonx.governance also provides governance features for generative AI assets. For installation and administration information, see IBM watsonx.governance.

Evaluate foundation model prompts and machine learning models

Make sure that your governed assets are meeting the goals that you establish for performance as well as dimensions that relate to fairness, quality, and drift. For more information, see Evaluate your foundation model and machine learning assets.

Create AI use cases to track solutions to business problems

Track a business solution from request to production in a collaborative use case. Factsheets capture details about the associated assets for each phase of the AI lifecycle.

Enhancements to factsheets include:

• Capture facts about generative AI models, generative AI prompt templates, and machine learning models.
• Create AI use cases to track and govern AI assets. (AI use cases were previously called Model use cases.)
• Access AI use cases and External models from the AI Governance section of the main navigation menu.
• Manage inventory settings from the AI use case settings page.
• Select an inventory for storing models that are created with a third-party provider. (External models are no longer limited to the Platform assets catalog.)

For more information, see Governing assets in AI use cases.

Design a workflow and monitor AI lifecycle activity

Use the Governance Console to use the capabilities of OpenPages to design workflows and view AI lifecycle activity from a dashboard to aid in meeting compliance and regulatory goals. See Managing risk and compliance with Governance Console on IBM watsonx.

Plan your AI governance strategy with watsonx.governance.

Version 1.1.3 of the watsonx.governance service includes various fixes.

For details, see What's new and changed in watsonx.governance.

Version 13.1.2 of the Cloud Pak for Data command-line interface includes various fixes.

For details, see What's new and changed in the Cloud Pak for Data command-line interface.

Version 8.2.0 of the common core services includes various fixes.

For details, see What's new and changed in the common core services.

Version 4.8.2 of the AI Factsheets service includes various fixes.

The 25.2.0 release of Cognos Analytics includes the following features and updates:

Service health monitor

You can now use the Cognos Analytics health monitor to monitor the health of your Cognos Analytics service instances. For more information, see Installing service monitors.

Updated software version for Cognos Analytics

This release of the Cognos Analytics service provides Version 12.0.2 of the Cognos Analytics software. For more information, see Release 12.0.2 - New and changed features in the Cognos Analytics documentation.

Version 25.2.0 of the Cognos Analytics service includes various fixes.

The 4.8.2 release of Data Privacy includes the following features and updates:

New option for rule action precedence

Rule action precedence enables you to specify how rules are applied when there are multiple rules with different actions on a data set. You can use the new Hierarchical enforcement option to configure a two-layer evaluation of data protection rules.

• The first layer evaluates the rules for an Allow or Deny action without considering any masking actions. The decision from this first layer must be to allow access to move to the second layer.
• The second layer evaluates the rules for a Transform action.

You can set this option from the user interface or from the access_decision_precedence API.

For more information, see Managing rule settings.

Enforce rules based on a tenant modality setting for aliases

You can now evaluate assets based on a tenant modality setting by calling the update tenant settings API and defining the asset_dealiasing_item_selection option. For more information, see Data protection rules enforcement.

Version 4.8.2 of the Data Privacy service includes various fixes.

For details, see What's new and changed in Data Privacy.

Version 4.8.2 of the Data Replication service includes various fixes.

For details, see What's new and changed in Data Replication.

The 4.8.2 release of DataStage includes the following features and updates:

Use the Teradata (optimized) connector in ELT run mode with SQL Pushdown

You can now run the Teradata (optimized) connector in ELT mode. For more information, see ELT run mode in DataStage.

Version 4.8.2 of the DataStage service includes various fixes.

For details, see What's new and changed in DataStage.

Version 4.8.2 of the Db2 service includes various fixes.

For details, see What's new and changed in Db2.

The 7.6.2 release of Db2 Big SQL includes the following features and updates:

Query data in Hadoop clusters on Cloudera Data Platform 7.1.9

Db2 Big SQL can now connect to remote Hadoop clusters on Cloudera Data Platform Version 7.1.9. For more information, see Setting up a connection from Db2 Big SQL to a remote data source.

Version 7.6.2 of the Db2 Big SQL service includes various fixes.

Version 4.8.2 of the Db2 Data Management Console service includes various fixes.

For details, see What's new and changed in Db2 Data Management Console.

Version 4.8.2 of the Db2 Warehouse service includes various fixes.

For details, see What's new and changed in Db2 Warehouse.

This release of the EDB Postgres service includes various fixes.

The 4.8.2 release of IBM Knowledge Catalog includes the following features and updates:

View data source information in the details panel for catalogs

If you click on an asset in the related items grid, you can view data source information directly in the asset details panel.

Deprecation of Egeria synchronization to external repositories

Egeria synchronization to external repositories is deprecated and will be removed in a future release. If you have questions or concerns related to the deprecation, you can open a support ticket.

Import metadata from additional data sources

You can now import metadata from Google Cloud Storage. For more information, see Supported data sources for metadata import, metadata enrichment, and data quality rules.

Work with sample data quality definitions

IBM Knowledge Catalog now includes a set of sample data quality definitions that you can use to build your data quality rules. The predefined rule expressions cover a wide range of standard checks, for example, for validity or completeness of certain values.

For more information, see Sample data quality definitions.

Searchable custom property values

Now, when you create a custom property, you can enable that custom property to be searchable through global search.

For more information, see Creating custom properties.

Version 4.8.2 of the IBM Knowledge Catalog service includes various fixes.

For details, see What's new and changed in IBM Knowledge Catalog.

Version 7.1.0 of the Informix service includes various fixes.

Version 42.2.3 of the MANTA Automated Data Lineage service includes various fixes.

For details, see What's new and changed in MANTA Automated Data Lineage.

Version 5.2.0 of the Product Master service includes various fixes.

For details, see What's new and changed in Product Master.

Version 4.8.2 of the Voice Gateway service includes various fixes.

Version 4.8.2 of the Watson Discovery service includes various fixes.

The 4.8.2 release of Watson Machine Learning Accelerator includes the following features and updates:

New NVIDIA GPU Operator version

You can now use the following versions of the NVIDIA GPU Operator with Watson Machine Learning Accelerator:

Red Hat OpenShift Container Platform Version 4.12

You can use the following versions of the NVIDIA GPU Operator:

• Version 22.9.2
• Version 23.3.2
• Version 23.6.1
• Version 23.9.0
• Version 23.9.1

Red Hat OpenShift Container Platform Version 4.14

You can use the following versions of the NVIDIA GPU Operator:

• Version 23.9.0
• Version 23.9.1

Version 4.8.2 of the Watson Machine Learning Accelerator service includes various fixes.

The 2.2.2 release of Watson Query includes the following features and updates:

New warnings during the fetch phase of queries

You can now gain additional insight into the problems that might occur when you retrieve results after a query first runs. New warnings alert you to a range of potential issues such as network disruptions, resource depletion problems (such as thread and memory constraints), SQL exceptions, and warnings that originate from the remote data source itself. For more information, see Enabling and disabling fetch phase warnings in Watson Query.

Enhanced monitoring of user activity with auditing

Events for the CHECKING category are now included in the default audit policy in Watson Query. The audit log now includes any authorization checks that occur when users attempt to access or manipulate database objects or functions.

To streamline the log, events for the CONTEXT and EXECUTE categories are no longer part of the default audit policy.

For a list of all audit events, see Db2 audit events for Watson Query.

Max Pushdown mode is automatically enabled to improve query performance

To improve the performance of your queries, Max Pushdown mode is enabled by default for new installations.

If you upgrade to IBM Cloud Pak for Data Version 4.8.2, queries use Max Consistency mode. However, a user with the Watson Query Admin role can change the query mode. For more information, see Setting the query mode in Watson Query.

Version 2.2.2 of the Watson Query service includes various fixes.

For details, see What's new and changed in Watson Query.

Version 4.8.2 of the Watson Speech to Text service includes various fixes.

For details, see What's new and changed in Watson Speech to Text.

The 4.8.2 release of watsonx Assistant includes the following features and updates:

Mask confidential customer information

You can now mask confidential customer information in an action by marking variables as private. For more information, see the following topics in the watsonx Assistant documentation:

• Using variables to manage conversation information
• Protecting the privacy of the customer information

Activity log

You can use the activity log to track changes that you made to an assistant. For more information, see Activity log in the watsonx Assistant documentation.

Message logs for a dialog without action

You can now use the Message logs tab on the Analyze page to see a log of all messages that are sent between the user and the assistant by turns. This feature is available for all messages, including the messages that are sent when dialog is activated. For more information, see Review customer conversations in the watsonx Assistant documentation.

More control over pause responses

Use a pause response to have your assistant wait for a specified interval before the next response. Previously, pauses were 1 to 10 seconds. You can now pause a response for 0 to 60 seconds. You can also specify milliseconds by specifying a decimal. For more information, see Pause response in the watsonx Assistant documentation.

Visualize an action

After you create an action, you can optionally switch from the step edit view to a visualization that displays a canvas with a flowchart of the action. For more information, see Visualizing the flow of the action in the watsonx Assistant documentation.

Invalid date entry not accepted

Starting from this release, the assistant does not recognize invalid date entries such as Feb 31, 31/11/2022, and Feb 29 2023. In addition, the assistant does not automatically parse the invalid dates to the first day of the month. For more information, see @sys-date in the watsonx Assistant documentation.

OpenAPI document file size limit for integrating custom extension

When you integrate a custom extension by using REST API, the maximum file size of the OpenAPI document that you can import is limited to 4 MB. For more information, see Preparing the API definition in the watsonx Assistant documentation.

Version 4.8.2 of the watsonx Assistant service includes various security fixes.

For details, see What's new and changed in watsonx Assistant.

Version 4.8.1 of the Cloud Pak for Data platform includes the following features and updates:

Red Hat OpenShift Container Platform Version 4.14

You can install Cloud Pak for Data Version 4.8.1 on the following versions of Red Hat OpenShift Container Platform:

• Version 4.12
• Version 4.14

Version 4.8.1 includes security fixes for the platform operator.

The 13.1.1 release of the Cloud Pak for Data command-line interface includes the following features and updates:

Download CASE packages with the new cpd-cli manage command

You can use the new cpd-cli manage case-download command to download CASE packages to client workstations before you run cpd-cli manage commands in a restricted network.

For more information, see:

• Preparing to run IBM Cloud Pak for Data installation commands in a restricted network
• 4.6.x upgrades: Preparing to run IBM Cloud Pak for Data upgrade commands in a restricted network
• 4.7.x upgrades: Preparing to run IBM Cloud Pak for Data upgrade commands in a restricted network
• 4.8.x upgrades: Preparing to run IBM Cloud Pak for Data upgrade commands in a restricted network

Version 13.1.1 of the Cloud Pak for Data command-line interface includes various fixes.

For details, see What's new and changed in the Cloud Pak for Data command-line interface.

The 8.1.0 release of the common core services includes changes to support features and updates in Watson Studio and IBM Knowledge Catalog.

Version 8.1.0 of the common core services includes various fixes.

For details, see What's new and changed in the common core services.

Version 1.19.0 of the scheduling service includes various fixes.

For details, see What's new and changed in the scheduling service.

Version 4.8.1 of the AI Factsheets service includes various fixes.

The 4.8.1 release of Analytics Engine powered by Apache Spark includes the following features and updates:

Configure the level of log information for Spark

You can now change the log levels for Spark drivers and executors to display relevant and concise messages. For more information, see Configuring Spark log level information.

Version 4.8.1 of the Analytics Engine powered by Apache Spark service includes various fixes.

For details, see What's new and changed in Analytics Engine powered by Apache Spark.

The 4.8.1 release of Cognos Dashboards includes the following features and updates:

Service health monitor

You can now use the Cognos Dashboards health monitor to monitor the health of your Cognos Dashboards service. For more information, see Installing service monitors.

Version 4.8.1 of the Cognos Dashboards service includes various fixes.

Version 4.8.1 of the Data Privacy service includes various fixes.

For details, see What's new and changed in Data Privacy.

Version 8.1.0 of the Data Refinery service includes various fixes.

For details, see What's new and changed in Data Refinery.

Version 4.8.1 of the Data Replication service includes various fixes.

For details, see What's new and changed in Data Replication.

The 4.8.1 release of DataStage includes the following features and updates:

Connect to a new data source

You can now include data from Looker as source data in your DataStage flows.

For the full list of DataStage connectors, see Supported data sources in DataStage.

Use the Teradata connector in ELT run mode with SQL Pushdown

You can now run the Teradata connector in ELT mode. For more information, see ELT run mode in DataStage.

Select multiple DataStage flows at the same time

You can now select multiple DataStage flows from your assets and open each of them in a new tab with one click.

Version 4.8.1 of the DataStage service includes various fixes.

For details, see What's new and changed in DataStage.

The 5.1.0 release of Db2 Data Gate includes the following features and updates:

See the status of the target and source databases from the web client

Previously, you saw only the status of the Db2 Data Gate service instance from the web client. Now, you can also see the status of the target database and the status of the source database that are associated with the service instance.

For more information, see Monitoring a Db2 Data Gate instance.

Version 5.1.0 of the Db2 Data Gate service includes various fixes.

For details, see What's new and changed in Db2 Data Gate.

Version 8.1.0 of the Decision Optimization service includes various fixes.

For details, see What's new and changed in Decision Optimization.

This release of the EDB Postgres service includes various fixes.

The 4.8.1 release of Execution Engine for Apache Hadoop includes the following features and updates:

Cloudera Data Platform 7.1.9

Execution Engine for Apache Hadoop can now work with data on Cloudera Data Platform 7.1.9.

Version 4.8.1 of the Execution Engine for Apache Hadoop service includes various fixes.

For details, see What's new and changed in Execution Engine for Apache Hadoop.

Version 4.8.1 of the IBM Knowledge Catalog service includes various fixes.

For details, see What's new and changed in IBM Knowledge Catalog.

The 3.4.36 release of IBM Match 360 includes the following features and updates:

Configuration snapshots now include attribute composition rules

Data engineers can now capture attribute composition rules within their IBM Match 360 configuration snapshots. With configuration snapshots, data engineers can save point-in-time configuration settings and matching configurations within their IBM Match 360 instance, and then share configurations across instances. Now when a data engineer updates or imports a snapshot, they can view, compare, and verify any attribute composition rule differences between the incoming snapshot and the current configuration.

Data engineers can use IBM Match 360 attribute composition rules to define how matched records get formed into master data entities. The rules determine how an entity's attribute values get selected from its multiple component records.

For more information about configuration snapshots, see Saving and loading master data configuration snapshots.

For more information about attribute composition rules, see Defining attribute composition rules.

Real-time data event streaming improvements

The IBM Match 360 real-time data event streaming capability includes multiple improvements in this release:

• IBM Match 360 can now stream more kinds of data events, including changes to persisted entities and supplementary attributes.
• The data streaming process can now automatically reconcile itself to help prevent messages from being lost due to any kind of failure. When the Apache Kafka server is not reachable for any reason, including intermittent network failures, messages are staged in an internal queue. When the system returns to normal, IBM Match 360 automatically reprocesses the staged messages.
• Data event streaming messages are now more traceable.

For more information about real-time data streaming, see Streaming record and entity data changes.

Version 3.4.36 of the IBM Match 360 service includes various fixes.

For details, see What's new and changed in IBM Match 360.

Version 42.1.0 of the MANTA Automated Data Lineage service includes various fixes.

This release of the MongoDB service include various fixes.

The 4.8.1 release of Planning Analytics includes the following features and updates:

Service health monitor

You can now use the Planning Analytics health monitor to monitor the health of your Planning Analytics service instances. For more information, see Installing service monitors.

Updated versions of Planning Analytics software

The 4.8.1 release of the service provides the following software versions:

• Planning Analytics Workspace Version 2.0.92

  For details about this version of the software, see 2.0.92 - What's new in the Planning Analytics Workspace documentation.

• Planning Analytics Spreadsheet Services Version 2.0.92

  For details about this version of the software, see 2.0.92 - Feature updates in the TM1 Web documentation.

• Planning Analytics for Microsoft Excel Version 2.0.92

  For details about this version of the software, see 2.0.92 - Feature updates in the Planning Analytics for Microsoft Excel documentation.

• Planning Analytics Engine Version 12.3.5

  For details about this version of the software, see What's new in Planning Analytics Engine in the Planning Analytics Engine documentation.

Version 4.8.1 of the Planning Analytics service includes various fixes.

Version 5.1.0 of the Product Master service includes various fixes.

For details, see What's new and changed in Product Master.

Version 8.1.0 of the RStudio Server Runtimes service includes various fixes.

Version 8.1.0 of the SPSS Modeler service includes various fixes.

For details, see What's new and changed in SPSS Modeler.

Version 8.1.0 of the Synthetic Data Generator service includes various fixes.

For details, see What's new and changed in Synthetic Data Generator.

The 4.8.0 release of Watson Discovery includes features and updates.

For a list of new features in Watson Discovery, see the Watson Discovery release notes.

Version 4.8.0 of the Watson Discovery service includes various fixes.

Version 4.8.1 of the Watson Machine Learning service includes various fixes.

The 4.8.1 release of Watson Machine Learning Accelerator includes the following features and updates:

New NVIDIA GPU Operator version

You can now use the following versions of the NVIDIA GPU Operator with Watson Machine Learning Accelerator:

Red Hat OpenShift Container Platform Version 4.12

You can use the following versions of the NVIDIA GPU Operator:

• Version 22.9.2
• Version 23.3.2
• Version 23.6.1
• Version 23.9.0

Red Hat OpenShift Container Platform Version 4.14

You can use Version 23.9.0 of the NVIDIA GPU Operator.

Version 4.8.1 of the Watson Machine Learning Accelerator service includes various fixes.

The 4.8.1 release of Watson Pipelines includes the following features and updates:

Assign job and job run names for simplicity

You can now assign names for pipeline jobs and job runs so that you can use the name, rather than a job ID, to run the job from the dashboard or the Run Pipeline job node. For more information, see Running and saving Pipelines.

Version 4.8.1 of the Watson Pipelines service includes various fixes.

Version 8.1.0 of the Watson Studio service includes various fixes.

For details, see What's new and changed in Watson Studio.

Version 8.1.0 of the Watson Studio Runtimes service includes various fixes.

The 8.1.0 release of watsonx.ai includes the following features and updates:

Explore the risks of working with AI

You can now explore some of the risks of working with generative AI, foundation models, and machine learning models. Read about risks for privacy, fairness, explainability, value alignment, and other areas. For more information, see AI risk atlas.

Use variables to build reusable prompts

Add flexibility to your prompts with prompt variables. Prompt variables function as placeholders in the static text of your prompt input. You can replace this placeholder with text dynamically at inference time. You can save prompt variable names and default values in a prompt template asset to reuse yourself or share with collaborators in your project. For more information, see Building reusable prompts.

Apply AI guardrails

You can now enable AI guardrails in the Prompt Lab. AI guardrails remove potentially harmful text from the input to and output from foundation models. Harmful text can include hate speech, abusive language, and profanity. For more information, see Prompt Lab.

Introducing the Tuning Studio

Use the Tuning Studio to tailor the output that a foundation model returns to better meet your needs. When you tune a model, you influence the content that is included in the model’s output and how the content is formatted. For example, if you plan to add the step of inferencing a foundation model into a business workflow, then you need to:

• Know what to expect from the foundation model.
• Be able to retrieve the output in a reliable format.

With the Tuning Studio, you can apply the prompt tuning method to the Google flan-t5-xl-3b model. For more information, see Tuning Studio.

Work with new foundation models in Prompt Lab

You can now use the following foundation models for inferencing from the Prompt Lab in watsonx.ai:

• flan-t5-xl-3b
• granite-13b-chat-v2
• granite-13b-instruct-v2
• llama-2-13b-chat

For more information about how to add models, see Supported foundation models.

Add sample prompts to notebooks

You can now add various sample prompts for specific models into your notebooks. To add a sample prompt, click the Code snippets icon (), select Prompt Engineering, and browse the various categories to find a sample prompt. When you select a prompt, click Insert code to cell to insert the prompt into your notebook.

Version 8.1.0 of the watsonx.ai service includes various fixes.

The 1.1.1 release of watsonx.data includes the following features and updates:

Audit logging

IBM watsonx.data now integrates with the Cloud Pak for Data audit logging service. Auditable events for watsonx.data are forwarded to the security information and event management (SIEM) solution that you integrate with.

For more information, see Audit events for watsonx.data and Exporting Cloud Pak for Data audit records to a security information and event management solution.

Use self-signed certificates and CA certificates to connect to object stores

Previously, watsonx.data could connect to HTTPS endpoints that used certificates signed by well-known certificate authorities, such as IBM Cloud Object Storage and Amazon S3. Now, you can connect to object stores that use self-signed certificates or certificates signed by other certificate authorities.

For more information, see Connecting to external object stores over https.

IBM Data Virtualization Manager for z/OS connector

You can use the new IBM Data Virtualization Manager for z/OS connector to read and write IBM Z®, without having to move, replicate or transform the data.

For more information, see Connecting to an IBM Data Virtualization Manager (DVM) data source.

Version 1.1.1 of the watsonx.data service includes various fixes.

For details, see What's new and changed in watsonx.data.

Need more insight into what users are doing? You can configure Cloud Pak for Data to send user activity logs to your rsyslog server. Cloud Pak for Data tracks activities such as a user:

• Pausing or resuming a Data Replication job.
• Accessing the Watson Speech services speech-to-text synthesis interface.
• Reviewing a Watson OpenScale model risk evaluation.

For more information, see Monitoring Cloud Pak for Data user activity.

If you have a custom authentication service, you can configure Cloud Pak for Data to authenticate users against the custom authentication service. This method does not replace the default Cloud Pak for Data method. Instead, this method enables you to provide an alternative method to authenticate users who provide a token that is associated with the custom authentication service. For more information, see Enabling Cloud Pak for Data to authenticate users against a custom authentication service.

Cloud Pak for Data Version 4.8 includes additional monitors to help you manage your environment:

• The node status check monitor, which reports the health and status of all of the nodes on the cluster.
• The volume usage status check monitor that reports the volume use of persistent volume claims.

The monitors provide additional information about the cluster where Cloud Pak for Data is installed so that you can keep your deployment running smoothly. To use these monitors, a cluster administrator must install the privileged monitoring service. For more information, see Installing privileged monitors.

Version 4.8 also includes service-specific monitors that an instance administrator can install. The service-specific monitors are designed to help you keep the services on your environment healthy. For more information, see Installing service monitors.

Starting in Cloud Pak for Data Version 4.8, a Cloud Pak for Data administrator can create announcements that are displayed in the web client. You can use announcements to let users know about upcoming maintenance and planned outages. The announcements are displayed below the toolbar in the web client. For more information, see Creating announcements that are displayed in the web client.

Previously, the ZenApiKey worked for a subset of Cloud Pak for Data services. In some situations, developers needed different keys and tokens to access all of the services that they had access to. Now, developers can use the ZenApiKey to access all Cloud Pak for Data services and APIs that they have the appropriate permissions to access. For more information, see Generating an API authorization token.

If you deployed Cloud Pak for Data on Portworx storage, you can now use Portworx as your disaster recovery solution.

For more information, see Data replication with Portworx.

By default, Cloud Pak for Data can connect to HashiCorp and CyberArk vaults. If you want to use another external vault, you can use the new vault bridge SDK at https://github.com/IBM/zen-secrets-vaults to connect to other types of external vaults.

The vault bridge SDK includes examples for the following enterprise vaults:

• IBM Cloud Secrets Manager
• Microsoft Azure Key Vault
• AWS Secrets Manager

For more information on creating a bridge using the vault bridge SDK, see Bridges to enterprise vaults.

You can use the new cpd-cli health commands to perform health checks on your Cloud Pak for Data environment. You can use the commands to collect detailed diagnostic information about:

• Storage health and performance
• Cluster health
• Operator health
• Operand health

You can also run multiple health checks at the same time.

For more information, see the cpd-cli health command reference.

Starting in Cloud Pak for Data Version 4.8, you can now install the following services on Power (ppc64le) hardware:

• Analytics Engine powered by Apache Spark
• Data Refinery
• Decision Optimization
• Watson Machine Learning
• Watson Studio
• Watson Studio Runtimes

For more information, see Hardware requirements for services.

The 8.0.0 release of the common core services includes changes to support features and updates in Watson Studio and IBM Knowledge Catalog.

Version 8.0.0 of the common core services includes the following features and updates:

Create your own connector

If Cloud Pak for Data does not have a predefined connector for a particular data source, an administrator can create a custom JDBC connector for the data source. Users can then use that connector to connect to multiple instances of the data source in a consistent way.

For more information, see Creating a custom JDBC connector .

Access data from SAP BusinessObjects

You can use a new connector to access SAP BusinessObjects data. SAP BusinessObjects is a platform that is used for reporting and visualization.

For more information, see SAP BusinessObjects connection.

Use Azure Active Directory authentication (Azure AD) for Microsoft Azure SQL Database connection permissions

You can now use Active Directory authentication for Microsoft Azure SQL Database connections. Active Directory authentication is an alternative to Microsoft SQL Server authentication. With this enhancement, administrators can centrally manage user permissions to Azure.

For more information, see Microsoft Azure SQL Database connection.

Version 8.0.0 of the common core services includes various fixes.

For more information, see What's new and changed in the common core services.

The 1.18.0 release of the scheduling service includes the following features and updates:

Monitor the health of the scheduling service

If you install the privileged monitoring service, you can optionally give Cloud Pak for Data administrators information about the health of the scheduling service For more information, see Installing privileged monitors.

Version 1.18.0 of the scheduling service includes various fixes.

For more information, see What's new and changed in the scheduling service.

The 4.8.0 release of AI Factsheets includes the following features and updates:

View model health and Drift v2 details in AI Factsheets

When you use Watson OpenScale to evaluate a model deployment, details for the model health and for the new Drift v2 metrics are recorded in the associated factsheets for the model deployment. For more information, see Viewing model and deployment details with AI Factsheets.

Version 4.8.0 of the AI Factsheets service includes various fixes.

The 4.8.0 release of Analytics Engine powered by Apache Spark includes the following features and updates:

Run applications on Spark 3.4

You can now use Spark 3.4 to run your applications on Analytics Engine powered by Apache Spark.

For more information, see Submitting Spark jobs via API.

View and edit more service instance details from the web client

From the Cloud Pak for Data web client, you can now configure the following information for the service instance:

• The resource quota
• The default configuration
• The default runtime

You can also view the run status of applications and the history server of your Spark instance.

For more information, see Managing Analytics Engine Powered by Apache Spark instances.

Analytics Engine powered by Apache Spark is now available on the Power platform

Starting in Cloud Pak for Data Version 4.8, you can use Analytics Engine powered by Apache Spark on the Power platform (ppc64le). R-based runtimes are not supported. For information on available runtimes, see Environments.

Development (*-devel) packages are removed

The operating system development packages (*-devel packages) are no longer available in the Spark runtime. The packages were removed to address security vulnerabilities.

If your applications use the development packages, they cannot be compiled on IBM Cloud Pak for Data 4.8.0. Contact IBM Support for assistance.

Version 4.8.0 of the Analytics Engine powered by Apache Spark includes various fixes.

For more information, see What's new and changed in Analytics Engine powered by Apache Spark.

Version 25.0.0 of the Cognos Analytics service includes various fixes.

Version 4.8.0 of the Cognos Dashboards service includes various fixes.

The 4.8.0 release of Data Privacy includes the following features and updates:

Export and import data protection rules

You can now use the Cloud Pak for Data command-line interface (cpd-cli) to export and import data protection rules across multiple instances of Cloud Pak for Data. The links to glossary artifacts, catalogs, assets, and users are maintained when you export the data protection rules. For more information, see Migrating data protection rules.

Double-byte or Unicode data is masked to X characters when you define Obfuscate masking rules

When you mask data with Identifier masking method (a data masking option for obfuscating), all the special characters in Unicode (such as double-byte characters) that are non-English-language characters is masked to X except the alphanumeric character and some special characters, such as -./@#$ %^&*()\:;?_"

For more information, see Obfuscating data method.

Version 4.8.0 of the Data Privacy service includes various fixes.

For more information, see What's new and changed in Data Privacy.

The 8.0.0 release of Data Refinery includes the following features and updates:

Control the placement of a new column in the Concatenate operation

You now have two options to specify the position of the new column is created by the Concatenate operation:

• As the rightmost column in the data set
• Next to the original column

Previously, the new column was placed at the beginning of the data set.

Important: If you have any existing Data Refinery flows that use the Concatenate operation, edit the flows to specify the new column position for the column. Otherwise, the flows with the operation might fail.

For information about Data Refinery operations, see GUI operations in Data Refinery.

Version 8.0.0 of the Data Refinery service includes various fixes.

For more information, see What's new and changed in Data Refinery.

The 4.8.0 release of Data Replication includes the following features and updates:

View and monitor replication details from the service instance page

You can now preview replicated data and track the status of the replication from a dashboard on the service instance details page for the Data Replication instance. For more information, see Monitoring your replication.

Version 4.8.0 of the Data Replication service includes various fixes.

For more information, see What's new and changed in Data Replication.

The 4.8.0 release of DataStage includes the following features and updates:

Group DataStage assets into folders

You can now group related DataStage assets by putting them in folders. Assets include connections, flows, jobs, parameter sets, and DataStage components. For more information, see Creating and managing DataStage assets.

View relationships between DataStage assets

You can now see the relationships between DataStage assets on the canvas. Select an asset such as a connection or flow to see which assets it uses and which assets it is used by. For more information, see Creating and managing DataStage assets.

Use new connectors and functions in ELT mode with SQL Pushdown

You can now use the following connectors in ELT run mode:

• IBM Db2 (optimized)

For more information, see ELT run mode in DataStage.

You can also now use parallel transform functions in ELT mode. For more information, see Parallel transform functions.

Connect to more data sources in DataStage

You can now include data from these data sources in your DataStage flows:

• Apache Derby
• IBM Cloud Databases for DataStax
• IBM Cloud Data Engine
• IBM watsonx.data (You can use this connection for source data only.)

For the full list of DataStage connectors, see Supported data sources in DataStage.

Call APIs with the new REST stage

Use the new REST stage to handle calls to REST APIs. For more information, see REST stage in DataStage.

Version 4.8.0 of the DataStage service includes various fixes.

For more information, see What's new and changed in DataStage.

The 4.8.0 release of Db2 includes the following features and updates:

Diagnostics from the db2support tool are integrated with Cloud Pak for Data diagnostics

Output from the db2support tool is now shared with the Cloud Pak for Data diagnostics tool, so that you can more easily gather Db2-specific diagnostics to resolve issues. You can still manually generate and share data from the db2support tool, if needed.

For more information, see Gathering diagnostic information.

Back up and restore databases with a custom resource

You now have the option to back up and restore your Db2 Warehouse databases with a custom resource. You can specify the backup details in the YAML file for your deployment. Backing up with a custom resource is available only for online, offline, incremental, and delta backups. For more information, see Back up and restore with a custom resource Backing up with a custom resource.

Simplified authentication for Db2 REST instances

You can now use API keys and personal access tokens from the Identity Management Service to authenticate to Db2 REST instances.

Version 4.8.0 of the Db2 service includes various fixes.

For more information, see What's new and changed in Db2.

Version 7.6.0 of the Db2 Big SQL service includes various fixes.

Version 5.0.0 of the Db2 Data Gate service includes the following features and updates:

Persistent custom indexes

Previously, custom indexes on target tables lasted only until the next full-table reload. During a reload, custom indexes were dropped, and you had to re-create the indexes manually. Starting in Cloud Pak for Data Version 4.8, you can provide an indexing data definition (DDL) job to Db2 Data Gate, which is permanently saved and rerun automatically each time tables are fully reloaded so that custom indexes are preserved. For more information, see Preserving indexes.

See the status of the target database

The status API is extended, so that it now retrieves the status of the target Db2 or Db2 Warehouse database in addition to the status of the Db2 Data Gate instance. For more information, see Retrieving the status.

Short CHAR strings require less space after conversion

When you add Db2 for z/OS source tables in EBCDIC encoding to Db2 Data Gate, the tables are converted to Unicode unless you select the query acceleration function.

Previously, 10 characters were added to the original length of short CHAR column entries during the conversion, which resulted in an excessive use of space for these values.

Now, the original string length of short CHAR entries is doubled during the conversion rather than increased by 10. For example, a source CHAR string with a length of 3, such as "YES", now uses 6 characters in the target database table, rather than 13 characters.

For more information, see Adding tables to Db2 Data Gate.

Faster container starts

The Db2 Data Gate container pod start time is a period during which the service is unavailable. Previously, the container pod used to take between 5 and 10 minutes to start. The pod start time is reduced by an average of 30 percent.

Version 5.0.0 of the Db2 Data Gate service includes various fixes.

For more information, see What's new and changed in Db2 Data Gate.

The 4.8.0 release of Db2 Data Management Console includes updates to the following features:

FIPS 140-2 compliant

Db2 Data Management Console service is now FIPS 140-2 compliant.

Version 4.8.0 of the Db2 Data Management Console service includes various fixes.

For more information, see What's new and changed in Db2 Data Management Console.

The 4.8.0 release of Db2 Warehouse includes the following features and updates:

Diagnostics from the db2support tool are integrated with Cloud Pak for Data diagnostics

Output from the db2support tool is now shared with the Cloud Pak for Data diagnostics tool, so that you can more easily gather Db2-specific diagnostics to resolve issues. You can still manually generate and share data from the db2support tool, if needed.

For more information, see Gathering diagnostic information.

Back up and restore databases with a custom resource

You now have the option to back up and restore your Db2 Warehouse databases with a custom resource. You can specify the backup details in the YAML file for your deployment. Backing up with a custom resource is available only for online, offline, incremental, and delta backups. For more information, see Backing up with a custom resource.

Simplified authentication for Db2 REST instances

You can now use API keys and personal access tokens from the Identity Management Service to authenticate to Db2 REST instances.

Version 4.8.0 of the Db2 Warehouse service includes various fixes.

For more information, see What's new and changed in Db2 Warehouse.

The 8.0.0 release of Decision Optimization includes the following features and updates:

Intermediate solutions in Decision Optimization

You can now choose to see a sample of intermediate solutions while a Decision Optimization experiment is running. Intermediate solutions can be useful for debugging or to see how the solver is progressing. For large models that take longer to solve, use intermediate solutions to quickly and easily identify any potential problems with the solve without having to wait for the solve to complete.

You can configure the Intermediate solution delivery parameter in the Run configuration and select a frequency for these solutions.

For more information, see Intermediate solutions, Solving a model and Run configuration parameters.

Decision Optimization Java models

You can now build Decision Optimization models in Java and deploy them in Watson Machine Learning. You can use the Java worker API to create optimization models locally with OPL, CPLEX, and CP Optimizer Java APIs. You can then package the models and easily deploy them on Watson Machine Learning.

To help you, the public cplex-java-worker GitHub repository includes a boilerplate that covers all the steps to build and deploy Java models.

For more information, see Deploying Java models for Decision Optimization.

Python 3.9 is now removed

The Decision Optimization environment currently supports Python 3.10. Python 3.9 is no longer supported. Python is used to run and deploy Decision Optimization models formulated in DOcplex in Decision Optimization experiments. Modeling Assistant models also use Python because DOcplex code is generated when models are run or deployed.

• To update your environment, see Configuring environments.
• To update existing deployed models, see Changing Python version for an existing deployed model with the REST API.

Version 8.0.0 of the Decision Optimization service includes various fixes.

For more information, see What's new and changed in Decision Optimization.

The 4.18.0 release of EDB Postgres includes the following features and updates:

Encrypt connections with TLS

You can use the Cloud Pak for Data web client to enable TLS encryption for an EDB Postgres database. For more information, see Creating a service instance for EDB Postgres from the web console.

Version 4.18.0 of the EDB Postgres service includes various fixes.

For more information, see What's new and changed in EDB Postgres.

The 4.8.0 release of Execution Engine for Apache Hadoop includes the following features and updates:

CES HDFS storage

You now have the option to use Cluster Export Services (CES) HDFS provided by IBM Storage Scale as the storage system in Cloudera Data Platform (CDP). For more information and to install CES HDFS, see IBM Storage Scale Big Data and Analytics Support.

Updated software versions

• Gateway Knox Service is upgraded to 2.0.0. No action is needed for this upgrade.

• Jupyter Environments Gateway (JEG) is upgraded from Version 3.1.0 to 3.2.2. No action is needed for this upgrade.

Version 4.8.0 of the Execution Engine for Apache Hadoop service includes various fixes.

For more information, see What's new and changed in Execution Engine for Apache Hadoop.

IBM Watson Knowledge Catalog is now IBM Knowledge Catalog.

The 4.8.0 release of IBM Knowledge Catalog includes the following features and updates:

Legacy features are no longer available

The legacy features of IBM Knowledge Catalog are no longer available in Cloud Pak for Data Version 4.8. These features were part of the IBM InfoSphere® Information Server components that were installed with the Watson Knowledge Catalog base configuration in Cloud Pak for Data Version 4.6. These features can no longer be installed, upgraded, or used.

If you want to upgrade a system with the base configuration to Cloud Pak for Data Version 4.8, you must migrate all data from the legacy components to their equivalent features in the IBM Knowledge Catalog core installation. For more information, see Migrating metadata and removing Knowledge Catalog legacy components.

Migrate from IBM InfoSphere Information Server to IBM Knowledge Catalog

If you are using the IBM InfoSphere Information Server product suite, you can now move all your processes and data to IBM Knowledge Catalog on Cloud Pak for Data.

When you migrate to Cloud Pak for Data, you can take advantage of:

• AI-based discovery and classification
• Advanced automated lineage
• Data privacy enforcement through attribute-based access control
• Advanced data quality management

You can migrate data from IBM InfoSphere Information Server Version 11.7.1.4 SP2. For more information, see Migrating data from IBM InfoSphere Information Server to Cloud Pak for Data.

Translate custom enumeration property names

The owner of a custom property can now specify that custom enumeration property names for assets and columns should be translated based on the user's browser language. For more information, see Creating custom properties.

More options for custom relationships

You can extend your set of custom relationship types by using multiple types on the source and target end. Use many artifact, asset and column types for more detailed relationship definition. For more information, see Creating custom relationships.

New custom property type

You can now create a custom property of type user and user group and assign specific users or user groups to it. For more information, see Creating custom properties.

Modify the default cascade behavior for some relationship types

You can now modify the default cascade delete, publish, clone, and promote settings for the following relationship types:

• Contains - Is contained in
• Is context parent of - Is context child of

To learn which cascade behaviors you can modify, see Asset relationships.

Import metadata from additional data sources

When you import metadata with the Discover import goal, new data sources are available:

• IBM Match 360
• SingleStoreDB

If the advanced metadata import option is enabled, you can import technical metadata from the following data source:

• SAP BusinessObjects

For more information, see Supported data sources for metadata import, metadata enrichment, and data quality rules.

Include or exclude hidden lineage dependencies from Oracle data sources

Now, when you import lineage data from Oracle data sources with the Get lineage import goal, you have the Import lineage for dependent assets not in scope option.

For lineage assets that have related assets that were not explicitly selected in the scope of the import, you can specify whether to include or exclude the assets from the import. For more information, see Advanced import options.

Import ETL job in a .zip file from DataStage on Cloud Pak for Data

You can now export an ETL job file from DataStage on Cloud Pak for Data and import it as a .zip file to a project by using the Import ETL job import goal. This option is useful when you have many Cloud Pak for Data instances.

For more information, see Importing ETL jobs.

Visualize relationship analysis results

In addition to the relationship table in the metadata enrichment results, you can now view a relationship diagram. This diagram helps you understand complex relationships between the assets in your organization.

For more information, see Identifying relationships.

New permissions for data quality

You can now assign the following permissions to your users to have more control over how data quality is established in IBM Knowledge Catalog:

• Manage data quality assets

  This permission replaces the Access data quality asset types permission, which was available in previous releases.

• Execute data quality rules
• Manage data quality SLA rules
• Drill down to issue details

By default, the new permissions are included in the following roles:

• Administrator
• Data Quality Analyst

For more information, see Predefined roles and permissions in Cloud Pak for Data.

Save filtering options for lineage graph

You can now save your filtering options and apply them to any lineage graph in the future. For more information, see Managing business lineage.

Data privacy enhancements in the Knowledge Accelerators

The Knowledge Accelerators include additional data classes that help identify data patterns which relate to various government IDs, such as tax IDs and Social Security numbers, across several new jurisdictions. These new data classes enhance metadata enrichment, especially when you are discovering data relevant to data privacy.

In addition, new policies and sample rules are now available for data retention and AI governance. These additional policies can help you set up appropriate data governance programs in these business areas.

For more information, see:

• Knowledge Accelerators data classes
• Knowledge Accelerators policies and rules

New taxonomy for banking financial reporting

The Knowledge Accelerators for Financial Services now has a new taxonomy of business terms to help European banking clients address their FINREP financial reporting obligations. The new taxonomy aligns with the current Data Point Model V3.3 publication from the European Banking Authority and is fully mapped to the core vocabulary of terms in the Knowledge Accelerators for Financial Services.

This new taxonomy provides a critical addition to the semantic layer of Business Terms that are the basis for the data governance activities that are needed to support this regulation.

Automatic mapping of logical data models to Knowledge Accelerators

You can now import logical data models into IBM Knowledge Catalog and automatically map them to the Knowledge Accelerators business terms.

For users of the IBM Industry Models, the same entity and attribute level mappings from IBM InfoSphere Information Governance Catalog are now available for you to use in IBM Knowledge Catalog

For more information, see Knowledge Accelerators.

More control over handling asset metadata in reporting

Catalog, project, and category collaborators with the Admin role can now decide to allow or deny reporting on the assets or artifacts metadata to the external reporting database. When creating a catalog, project, or category, they can select to opt in for reporting on such metadata. For more information, see Reporting on assets metadata.

Enhanced reporting on asset relationships

You can now get reports on the relationships between data asset column and any asset, and on data asset reference column and any asset. For more information, see Asset relationships reporting tables.

Version 4.8.0 of the IBM Knowledge Catalog service includes various fixes.

For more information, see What's new and changed in IBM Knowledge Catalog.

Version 3.3.15 of the IBM Match 360 service includes various fixes.

For more information, see What's new and changed in IBM Match 360.

The 7.0.0 release of Informix includes the following features and updates:

More integration with the Cloud Pak for Data command-line interface

Starting in Cloud Pak for Data Version 4.8, Informix uses more of the commands in the Cloud Pak for Data command-line interface (cpd-cli):

• You can create Informix instances using the cpd-cli service-instance commands.

  For more information, see Creating a service instance for Informix with the cpd-cli service-instance create command.

• You can shut down and restart the Informix service with the cpd-cli manage shutdown and cpd-cli manage restart commands.

  For more information, see Shutting down and restarting services.

Version 7.0.0 of the Informix service includes various fixes.

Version 42.0.5 of the MANTA Automated Data Lineage service includes various fixes.

The 4.18.0 release of MongoDB includes the following features and updates:

Encrypt connections with TLS

After you provision MongoDB Ops Manager and a database instance, you can configure both to use TLS to encrypt connections to the database using their custom resources. For more information, see Configuring TLS for a MongoDB database.

This release of MongoDB includes various fixes.

Version 9.000.1 of the OpenPages service includes various fixes.

The 4.8.0 release of Planning Analytics includes the following features and updates:

Updated versions of Planning Analytics software

The 4.8.0 release of the service provides the following software versions:

• Planning Analytics Workspace Version 2.0.91

  For details about this version of the software, see 2.0.91 - What's new in the Planning Analytics Workspace documentation.

• Planning Analytics Spreadsheet Services Version 2.0.91

  For details about this version of the software, see 2.0.91 - Feature updates in the TM1 Web documentation.

• Planning Analytics for Microsoft Excel Version 2.0.91

  For details about this version of the software, see 2.0.91 - Feature updates in the Planning Analytics for Microsoft Excel documentation.

• Planning Analytics Engine Version 12.3.4

  For details about this version of the software, see What's new in Planning Analytics Engine in the Planning Analytics Engine documentation.

Version 4.8.0 of the Planning Analytics service includes various fixes.

Version 5.0.0 of the Product Master service includes various fixes.

The 8.0.0 release of RStudio Server Runtimes includes the following features and updates:

Use Apache Spark 3.4

You can start using Spark 3.4 to run your scripts in RStudio. For more information, see Spark environments.

Spark 3.2 is removed

You can no longer use RStudio Server Runtimes environments with Spark 3.2. If you need Spark, you must use Spark 3.4. For more information, see Spark environments.

New flow for adding data to RStudio code

The RStudio toolbar contains a new Code snippets icon () that you can use to open the Code snippets pane, where you can read data from a file or connection that was added to the project.

To generate code that inserts data to RStudio, you can now follow these steps:

1. Click the Code snippets icon ().
2. Click Read data, select the data source from your project, and copy the code to clipboard.
3. Paste the code in the RStudio file editor.

For more information, see Analyzing data with RStudio.

Version 8.0.0 of the RStudio Server Runtimes service includes various fixes.

The 8.0.0 release of SPSS Modeler includes the following features and updates:

Traditional skewness values in the Data Audit table

The skewness value can help you understand the shape and nature of data distribution when you audit data. SPSS Modeler can calculate the skewness value in two ways:

• The adjusted skewness value
• The traditional skewness value

The traditional_skewness is calculated by Data View. The adjusted_skewness value is used by the SPSS Modeler backend and Python.

Previously, only the adjusted skewness was displayed in the Data Audit table. Now, the traditional skewness value is displayed by default. You can click the graph of the data set to view both skewness values. For more information, see Data Audit node.

Use additional files types

You can now use the following file types in SPSS Modeler:

• Parquet data files
• SAV data files

For more information, see Supported data sources for SPSS Modeler.

SQL pushback available on additional data sources

You can now connect to and use SQL pushback on the following data sources from SPSS Modeler:

• PostgreSQL
• IBM Cloud Databases for PostgreSQL
• Amazon RDS for PostgreSQL

For more information, see Supported data sources for SPSS Modeler.

Read and write access available on additional data source

You can now connect to and have read and write access on SingleStoreDB from SPSS Modeler.

Conserve resources

If multiple SPSS Modeler flows are open from the same project at the same time, your system might run low on memory. SPSS Modeler now displays a warning message when multiple flows cause your system to run low on memory. You can review all the active flows and close the ones that are not running to save resource.

Customize and reuse text analysis packages, libraries, and templates

You can customize and save linguistic resources, such as text analysis packages, libraries, and templates as project assets from the Text Analytics Workbench. You can then reuse these linguistic resources in other flows that include a Text Mining node. For more information, see Linguistic resources.

View branch status when you run your modeler flow

When you run an SPSS Modeler flow, you can see how many branches in the flow are running and its progress.

Version 8.0.0 of the SPSS Modeler service includes various fixes.

For more information, see What's new and changed in SPSS Modeler.

Version 4.8.0 of the Voice Gateway service includes various fixes.

Version 4.8.0 of the Watson Discovery service includes the following features and updates:

Enrichments are highlighted in distinct colors

When you select multiple enrichments in the advanced document view for PDF and text, each enrichment type is highlighted in the document with distinct colors. Overlapping enrichments are also highlighted in a distinct color.

Narrow the field of a search

Now, when you query data from the web client, you can specify the fields to search in. Searching specific fields improves search results when content is not indexed in the default fields. For example, content might not be indexed in the default fields when you ingest structured files or when you apply a Smart Document Understanding model. For more information, see Excerpt unavailable in the Watson Discovery documentation.

Start or stop crawling a collection

You now have the option to start or stop the scheduled crawl of a collection without changing the crawl schedule. For example, you might want to stop a crawl if you don't want the crawl to occur during a long-running task. For more information, see Stopping a crawl in the Watson Discovery documentation.

Updated tokenizer for all languages

The updated tokenizer might affect the ranking order of results for certain queries. If you observe any ranking differences in your query results, you can re-index the documents in the collection.

Watson Discovery tokenizes words when it ingests and stores data in the index, and at run time, when it analyzes queries that are submitted by users. By re-indexing the collection, you ensure that your documents are indexed with the same tokenizer that is used for matching queries.

To re-index documents, open the Manage collection page, choose a collection, and navigate to the Enrichments tab. Select a field to enrich, and then clear the field. Next, click Apply changes and reprocess and wait for the documents in the collection to be reprocessed.

For more information on re-indexing a collection, see Manage enrichments in the Watson Discovery documentation.

The part of speech enrichment is now needed only for content mining project types.

The part of speech enrichment was used for dictionary suggestion. The dictionary suggestion is now updated to work without the part of speech enrichment applied. However, for content mining projects, the part of speech enrichment is available as before.

Search for terms to label as entity examples

You can now search for terms that you want to label as entity examples in a document. You can also search for labeled and unlabeled entity examples, and you can correct any labeling inconsistencies. For more information, see Searching for examples using keywords in the Watson Discovery documentation.

Export labeled data from entity extraction

You can export labeled data that you identified when you labeled documents for entity extraction. You can use the labeled data to train or build large language models (LLMs) with services like Watson Studio and Watson Natural Language Processing (NLP). For more information, see Exporting labeled data for an entity extractor in the Watson Discovery documentation.

Version 4.8.0 of the Watson Discovery service includes various fixes.

Version 4.8.0 of the Watson Machine Learning service includes various fixes.

Version 4.8.0 of the Watson Machine Learning Accelerator service includes the following features and updates:

New deep learning libraries

You can now use the following deep learning libraries with Watson Machine Learning Accelerator:

• Python 3.10.12
• PyTorch 2.0.1

If you have existing models, update and test your models to use the latest supported frameworks. For more information, see Supported deep learning frameworks in the Watson Machine Learning Accelerator documentation.

Removal of the Watson Machine Learning Accelerator console

Starting in Cloud Pak for Data Version 4.8, the Watson Machine Learning Accelerator console is no longer available. You can manage and monitor deep learning training by using Deep Learning Experiments or the Watson Machine Learning Accelerator REST API. For more information, see:

• Training neural networks
• Watson Machine Learning Accelerator REST API in the Watson Machine Learning Accelerator documentation

See full list of What's new for Watson Machine Learning Accelerator.

Version 4.8.0 of the Watson Machine Learning Accelerator service includes various fixes.

The 4.8.0 release of Watson OpenScale includes the following features and updates:

Manage data for model evaluations with PostgreSQL

Database support for storing model transactions and model evaluation results now includes Postgre SQL database. For more information, see Configuring Watson OpenScale with manual setup

Run model evaluations with historical data

You can now use an API to evaluate historical payload and feedback data records for batch deployments to track your model performance. This gives you a wider horizon for analyzing model performance over time.

Version 4.8.0 of the Watson OpenScale service includes various fixes.

For more information, see What's new and changed in Watson OpenScale.

The 4.8.0 release of Watson Pipelines includes the following features and updates:

Secure your pipeline job runs

Secure your data by using encrypted parameters when you run a pipeline job. If you supply an encrypted parameter, your data will not be exposed by logs or written to an external drive. You can use encrypted data for a wide range of parameters, such as PROJDEF and parameter sets with environment variables.

For more information, see Configuring global objects.

Enhanced performance and configuration sizes

Watson Pipelines introduced several runtime performance improvements and expanded the limit for configuration sizes for a more optimized experience. You can add as many nodes as you need now to the pipeline canvas with no limit. At runtime, Watson Pipelines removed the node limit for single pipelines and loops in pipelines, standardizing the node limit across all active pipelines. Watson Pipelines also increased execution performance by as much as 275%. Additionally, by using shared runtime while running a Bash script can reduce the execution time of such nodes by as much as 350%.

Version 4.8.0 of the Watson Pipelines service includes various fixes.

Version 2.2.0 of the Watson Query service includes the following features and updates:

Enhanced security for the Admin role

For newly provisioned Watson Query service instances, the Admin role no longer has default access to all data. The DATAACCESS Db2 authority is removed from the Admin role. Admin users can still access data that they own or that they have been assigned to. This change does not impact existing instances that are upgraded to Version 2.2.0.

For more information, see Managing roles for users and groups in Watson Query.

IBM Knowledge Catalog data protection rules are always enabled for Watson Query data

If IBM Knowledge Catalog and Watson Query are installed in the same instance of Cloud Pak for Data, IBM Knowledge Catalog data protection is enabled and automatically applied to Watson Query data.

For more information, see Governing virtual data with data protection rules in Watson Query.

Secure your ungoverned objects

With IBM Knowledge Catalog data protection rules in Watson Query, virtualized objects that are not published in a governed catalog will now follow the Default data access convention setting from your rule settings.

For more information, see Allowing and denying access to data in Watson Query.

Improved Numeric shift masking precision

The values of the data that is masked with the Numeric shift method now correspond more closely to the percentage specified. As a result of this improvement, some values that are masked with the Numeric shift method will differ from the previous results. For more information, see Masking virtual data in Watson Query.

Improved error reporting of data protection rules

You can now access improved error reporting with enhanced and detailed information for errors that are linked to the enforcement of data protection rules. For more information, see SQL5105N error when you run a query in Watson Query.

Query Presto data

You can now create a connection to Presto to access and query data in Presto. For more information, see Supported data sources in Watson Query.

Improved auditing and detailed monitoring for Watson Query

• Monitoring data access with the Db2audit facility: When you install or upgrade Watson Query, the Db2audit facility is enabled by default. However, an administrator can turn the feature off.
• Audit events: Watson Query uses the Cloud Pak for Data platform audit logging capabilities to capture event activity at a more granular level by auditing additional events that occur in the Data virtualization engine.

  For more information, see Auditing in Watson Query.

Version 2.2.0 of the Watson Query service includes various fixes.

For more information, see What's new and changed in Watson Query.

Version 4.8.0 of the Watson Speech services includes various fixes.

For more information, see What's new and changed in Watson Speech to Text.

The 8.0.0 release of Watson Studio includes the following features and updates:

Organize project assets into folders

You can now create folders in your projects to organize assets. An administrator of the project must enable folders, and administrators and editors can create and manage them. Folders are in beta and are not yet supported for use in production environments. For more information, see Organizing assets with folders (beta).

Add data from additional sources to your notebook code with a click of a button

With the Code snippets icon () you can easily add data to your notebooks. You can now use the Code snippets icon to add data from the following sources:

• Presto connections
• Connections based on the Custom JDBC connector

  For more information about the Custom JDBC connector, see Creating a custom JDBC connector.

Version 8.0.0 of the Watson Studio service includes various fixes.

For more information, see What's new and changed in Watson Studio.

The 8.0.0 release of Watson Studio Runtimes includes the following features and updates:

Partition your GPU environment into independent instances (MIG devices)

With NVIDIA Multi-Instance GPU (MIG), you can partition a physical GPU into multiple smaller, independent instances, which are known as MIG devices. MIG devices offer isolation and allocation capabilities, so you can efficiently share resources.

Starting in Cloud Pak for Data Version 4.8, you can optionally enable MIG mode on your GPU nodes. For more information, see Administering Runtime 23.1 on Python 3.10 with GPU and Administering Runtime 22.2 on Python 3.10 with GPU.

Use Apache Spark 3.4 to run notebooks and scripts

You can start using Spark 3.4 in your notebooks and scripts in Juypter notebooks and JupyterLab. You can use Spark 3.4 in environments based on Python 3.10 and R 4.2. For more information, see Spark environments.

Environments with Spark 3.2 are removed

You can no longer use Juypter environments with Spark 3.2. If you need Spark, you must use Spark 3.4. For more information, see Spark environments.

Removal of all Runtime 22.1 environments

Runtime 22.1 environments are not available anymore. For instructions on migrating your notebooks and Git projects to latest runtime environments, see Migrating Runtime 22.1 environments from Cloud Pak for Data 4.6 or 4.7.

Version 8.0.0 of the Watson Studio Runtimes service includes various fixes. For more information, see What's new and changed in Watson Studio Runtimes.

The 4.8.0 release of watsonx Assistant includes the following features and updates:

Watson Assistant is now watsonx Assistant

The Watson Assistant service was renamed to watsonx Assistant.

Store your custom metrics

You can store your custom metrics for assistants by using the metrics method. For more information, see Monitoring the platform.

Diagnostic logs

You can now use the diagnostics logs in Cloud Pak for Data for your assistants. For more information, see Gathering diagnostic information.

Version 4.8.0 of the watsonx Assistant service includes various security fixes.

The 1.1.0 release of watsonx.data includes the following features and updates:

Time-travel and roll-back queries

You can now run the following time-travel queries to access historical data in Apache Iceberg tables:

• SELECT <columns> FROM <iceberg-table> FOR TIMESTAMP AS OF TIMESTAMP <timestamp>;
• SELECT <columns> FROM <iceberg-table> FOR VERSION AS OF <snapshotId>;

You can use time-travel queries to query and restore data that was updated or deleted in the past.

You can also roll back an Apache Iceberg table to any existing snapshot.

Capture historical data about Presto queries

The Query History Monitoring and Management (QHMM) service captures historical data about Presto queries and events. The historical data is stored in a MinIO bucket and you can use the data to understand the queries that were run and to debug the Presto engine.

For more information, see Monitoring and managing diagnostic data.

Ingest data by using Spark

You can now use the IBM Analytics Engine powered by Apache Spark to run ingestion jobs in watsonx.data.

For more information, see Ingesting data by using Spark.

Improved query performance with caching

You can use the following types of caching to improve Presto query performance:

• Metastore caching
• File list caching
• File metadata caching

For more information, see Enhancing query performance through caching.

New connectors

You can now use connectors in watsonx.data to establish connections to the following types of databases:

• SAP HANA
• SingleStoreDB
• Snowflake
• Teradata

For more information, see Adding a database.

Version 1.1.0 of the watsonx.data service includes various fixes.

For details, see What's new and changed in watsonx.data.

You can use the Synthetic Data Generator graphical flow editor tool to generate synthetic tabular data. Synthetic data is information that is generated to augment or replace real data to improve AI models, protect sensitive data, and mitigate bias. Synthetic data helps to mitigate many of the logistical, ethical, and privacy issues that come with training machine learning models on real-world examples.

You can deploy Cloud Pak for Data Version 4.8.0 on the following versions of Red Hat OpenShift Container Platform:

• Version 4.12

You can deploy Cloud Pak for Data Version 4.8.1 on the following versions of Red Hat OpenShift Container Platform:

• Version 4.12 or later fixes
• Version 4.14 or later fixes
• Version 4.15 or later fixes

Each instance of Cloud Pak for Data now has its own set of operators. The private topology simplifies the process of installing and managing multiple instances of Cloud Pak for Data at different releases on a single cluster.

The private topology replaces the express installation topology and the specialized installation topology.

If you are upgrading to IBM Cloud Pak for Data Version 4.8, you must migrate your existing installation to the private topology.

For more information, see Supported project (namespace) configurations.

You can now install or upgrade multiple components in parallel. When you run a batch installation or upgrade, the apply-cr command automatically installs or upgrades up to 4 components at a time.

The apply-cr command ensures that the specified components are installed in the correct order. For example, the command ensures that the control plane is installed before any services are installed.

You can adjust the number of components that are installed or upgraded in parallel by specifying the --parallel_num option. For more information, see the manage apply-cr command reference.

By default, IBM Cloud Pak for Data user records are stored in an internal repository. However it is strongly recommended that you use an enterprise-grade password management solution, such as single sign-on (SSO) or LDAP.

If you decide to use an LDAP server, you currently have two methods for connecting to your LDAP server:

• The LDAP integration provided by Cloud Pak for Data.
• The LDAP integration provided by the IBM Cloud Pak foundational services Identity Management Service.

The LDAP integration provided by Cloud Pak for Data is deprecated and will be removed in a future release.

Installing

The Identity Management Service is automatically enabled when you install Cloud Pak for Data Version 4.8.

Upgrading

If you are upgrading to IBM Cloud Pak for Data Version 4.8 and you currently use the LDAP integration provided by Cloud Pak for Data, it is strongly recommended that you migrate your existing configuration to the Identity Management Service. To migrate your LDAP configuration:

1. Integrate with the Identity Management Service.
2. Change the cpadmin user to admin.
3. Connect to your current LDAP server from the Identity Management Service (links to the IBM Cloud Pak foundational services documentation).

Repeat these steps for each instance of Cloud Pak for Data, that you want to integrate with the Identity Management Service.

In IBM Cloud Pak for Data Version 4.6, all instances of IBM Cloud Pak for Data on a cluster are managed by a set of shared operators. The location of the operators depended on the installation topology that you chose when you installed IBM Cloud Pak for Data:

Express installation topology

In an express installation, the shared IBM Cloud Pak for Data operators were co-located with the shared IBM Cloud Pak foundational services operators in the ibm-common-services project.

Specialized installation topology

In a specialized installation, the shared IBM Cloud Pak for Data operators were in a separate project from the shared IBM Cloud Pak foundational services operators, which were typically in the ibm-common-services project.

Starting in IBM Cloud Pak for Data Version 4.7, both of these installation topologies were replaced by the private topology. In the private topology, each instance of Cloud Pak for Data has its own set of operators.

If you are currently running IBM Cloud Pak for Data Version 4.6, you must migrate your current installation to the private topology when you upgrade to Version 4.8. For more information, see Upgrading Cloud Pak for Data.

The cpd-cli diag healthcheck is replaced by the new cpd-cli health plug-in.

For more information on the new plug-in, see the health command reference.

All Runtime 22.1 environments are removed in Cloud Pak for Data Version 4.8.

Spark was removed from the following services:

• Watson Studio
• Watson Studio Runtimes