What's new and changed in IBM Knowledge Catalog

Installing or upgrading IBM Knowledge Catalog

Ready to install or upgrade IBM Knowledge Catalog?

• To install IBM Knowledge Catalog along with the other Cloud Pak for Data services, see Installing Cloud Pak for Data.
• To upgrade IBM Knowledge Catalog along with the other Cloud Pak for Data services, see Upgrading Cloud Pak for Data.
• To install or upgrade IBM Knowledge Catalog independently, see IBM Knowledge Catalog.
  Remember: All of the Cloud Pak for Data components associated with an instance of Cloud Pak for Data must be installed at the same version.

Cloud Pak for Data Version 5.0.0

A new version of IBM Knowledge Catalog was released in June 2024 with Cloud Pak for Data 5.0.0.

Operand version: 5.0.0

This release includes the following changes:

New features

This release of IBM Knowledge Catalog includes the following features and updates:

Additional IBM Knowledge Catalog editions

You can continue to use the classic IBM Knowledge Catalog service, or you can choose one of the two new, separately priced editions of IBM Knowledge Catalog:

IBM Knowledge Catalog Standard Cartridge

This edition offers basic governance tooling for cataloging and AI-augmented data enrichment.

IBM Knowledge Catalog Premium Cartridge

This edition offers the full governance framework with data privacy, data quality, cataloging, and enrichment across the data lifecycle with a generative AI layer for enhanced data enrichment.

In addition to governance capabilities as in the classic IBM Knowledge Catalog service, the cartridges provide semantic and AI-augmented data enrichment:

• Recommend descriptive names for data assets and columns based on the collected metadata and a predefined glossary.
• Suggest and assign semantic descriptions for data assets and columns that are easy to understand. The descriptions are generated based on the surrounding columns and the context of the data assets.
• Generate semantic term assignments for data assets and columns.

For details, see IBM Knowledge Catalog.

Import metadata from additional data sources

You can now import metadata and lineage metadata from the following data sources:

MicroStrategy

Use a new connection to import data. For details, see Supported data sources for metadata import, metadata enrichment, and data quality rules.

OpenLineage

Import the data from a .zip file. For details, see Importing ETL jobs and Getting ETL job lineage.

Data quality enhancements

You can now add data assets or columns with the new relationship type Validates data quality of to any type of data quality rule to have the quality score and any data quality issues reported for this item on the Data quality page. With this enhancement, data quality rules with externally managed bindings and SQL-based data quality rules can now also contribute to the quality scores of assets and columns.

For details, see Creating rules from data quality definitions and Creating SQL-based data quality rules.

Data protection rules are no longer enforced in projects

Data protection rules are now only enforced in governed catalogs or by a deep enforcement solution. Assets that are added into projects from a governed catalog no longer have preview, download, or profiling restricted by data protection rules. For more information, see Data protection rules no longer enforced in projects.

Enhanced project list view in catalogs

Now, when you are adding assets from a catalog to a project, you can view more than 100 projects in your project list page and add up to 50 assets at a time to your project. For more information, see Add assets from within the catalog.

Enhancements in governance artifacts

• You can now make changes to multiple governance artifacts at once. Bulk edits are available when updating tags and stewards. For more information, see Managing governance artifacts.
• Now you can move any category either to the top level or to any other category as a sub-category. The collaborators are also moved provided they have required permissions on the new parent category. For more information, see Managing categories.
• You can now add custom properties and relationships for reference data sets. For more information, see Designing reference data sets.
• Notifications about changes in governance artifacts, for example, when an artifact is added, updated, or deleted, can now be forwarded to external applications or users. For more information, see Forwarding notifications generated by Cloud Pak for Data services.

Knowledge Accelerators

Additional data classes

There are over 20 new data classes that can be used to identify and classify national identifiers, tax identifiers and social security identifiers for the additional jurisdictions of Argentina, Egypt, Finland, Greece, Hong Kong, Ireland, Malaysia, New Zealand, Pakistan, Peru, Romania, Thailand, Turkey, and United Arab Emirates.

These new data classes supplement previously added data classes to provide an enhanced framework for identifying and classifying data of particular relevance to data privacy.

For more information, see Knowledge Accelerators data classes.

Updated business scopes for Relationship Explorer

The Knowledge Accelerators contain a set of predefined business scopes that group the set of business terms that are relevant to a specific business topic. Many of these scopes were reorganized to ensure that they are optimized for viewing in the new Relationship Explorer capability of IBM Knowledge Catalog. Also, new business scopes were added to Financial Services.

In addition, certain term-to-term relationships across the Knowledge Accelerators were simplified to improve clarity when viewing them in Relationship Explorer.

For more information, see Business scopes for Knowledge Accelerators.

Relationship Explorer to visualize your metadata

Relationship Explorer is now available to help better understand your data. This new feature helps you to visualize, explore and govern your metadata. Discover how your governance artifacts and data assets relate with each other in a single view. For more information, see Relationship Explorer.

Expand DataStage jobs in the lineage graph

When you are viewing a DataStage job in the lineage graph, you can expand the job to view all its stages. For more information, see Lineage.

Enhanced security for profiling results in Data Virtualization and watsonx.data views

To prevent unexpected exposure to value distributions through the profiling results of a view, all users are denied access to profiling results in Data Virtualization and watsonx.data views in all catalogs and projects.

Updates

The following updates were introduced in this release:

• New options were added to the business terms and data classes filters for enrichment results: assigned, suggested, and none.
• Updated the asset membership roles for catalogs. Now, users can hold the asset owner, asset editor, or asset viewer role. The asset editor role replaced the asset member role. Now, to complete any asset-related actions, you must be an asset owner or asset editor.

  Also, catalog assets might have more than one owner now.

  You can change asset user roles on the Access control page of an asset by selecting a role from the Role dropdown menu.

• Added a Add catalog assets to projects user permission. Now, to add assets to projects, you must have the Add catalog assets to projects, the Admin, Editor or Viewer role in the catalog, and be the asset owner or editor. Users that don't have an existing role with the Manage catalogs or Access catalogs permission must be explicitly granted the Add catalog assets to projects permission.

• You can now edit and remove the business terms, owners or tags on up to 20 catalog assets at a time.
• You can now assign one or more classifications to a column within an asset of your catalog. Bulk edits and removals of classifications are also introduced.
• The following new columns are now available in the reporting tables:
  • Column level classifications
  • SQL Query asset type
  • Asset Member Roles
  • Custom Properties of user/group type for asset and columns
  • More column attributes - native & inferred data type, mean value
• When you import metadata with the Discover import goal, you can use a new advanced option Import asset timestamp to include the information about the time when the imported asset was last modified.
• You can configure two new properties for the Qlik Sense connection: Extracted applications and Excluded applications.
• Administrator users can now configure the default reporting settings by using the ccs-features-configmap. For more information, see Configuring reporting settings for IBM Knowledge Catalog.

Customer-reported issues fixed in this release

The following issues, which were reported by customers, were fixed in this release:

• DT378506: Adding a member to a catalog results in the wrong member being added.
• DT242636: The project does not display in Add to project pull down list.
• DT386785: Metadata import job remains in running state if the Jobs API service doesn't respond at the time the job ends.
• DT386793: The Activities for this asset section for catalog assets may appear as if it not updating if there is no lineage information available for an asset.
• DT386829: For assets with short or long column names, the metadata enrichment job may complete with a HTTP 502 Bad Gateway error for ML-based term assignment.
• DT386841: When adding new assets to an existing metadata import scope, user has to re-select previously selected assets along with new assets.
• DT386843: Metadata import job may have timeout errors due to slow or delayed response from the source database.
• DT386875: User may not be able to click Add to project when assets in catalog are selected. User is also not able to see the add to project button for individual asset.
• DT386883: User gets unknown error notification after adding assets to project from catalog and these assets fail to be added to the selected project.
• DT386893: In the Cloud Pak for Data home page, the create time displayed for Catalogs is not displayed in the user’s timezone, if the user is not in the same timezone as the Cloud Pak for Data server.
• DT380903: While creating Connection for Qlik Sense results in an error: The test was not successful. An unexpected error occurred of type Null pointer error.

Security issues fixed in this release

The following security issues were fixed in this release:

CVE-2016-1000027

CVE-2022-24823, CVE-2022-41881

CVE-2023-42282, CVE-2023-44487, CVE-2023-51074

CVE-2024-21892, CVE-2024-22019, CVE-2024-22243, CVE-2024-24762, CVE-2024-28180

Deprecated features

The following features were deprecated in this release:

Egeria synchronization to external repositories is now removed

Egeria synchronization to external repositories is no longer available from IBM Knowledge Catalog.