Importing metadata (IBM Knowledge Catalog)

You can capture and import technical metadata and lineage information for the data in your organization. This data can be on a wide variety of data sources. When you import metadata, you create assets in the platform.

Required services

IBM Knowledge Catalog

MANTA Automated Data Lineage for IBM Cloud Pak for Data

See Installation and license requirements

Metadata types

Technical metadata

Lineage metadata

See Types of metadata

Asset types

Data assets:

Tables from relational and nonrelational data sources
Files from file-based connections to the data sources

Cobol copybooks

Business intelligence reports

Transformation scripts

Data models

Data integration assets

See Types of assets

Supported connections

You can import assets from the data sources listed in Supported data sources for curation and data quality.

Required permissions

To create, manage, and run a metadata import, you must have these roles and permissions:

The Manage asset discovery user permission.
The Admin or the Editor role in the project.
The Admin or the Editor role in the catalog to which you want to import or publish the assets.
Access to the connections to the data sources of the data assets to be imported and the SELECT or a similar permission on the corresponding databases.

Types of metadata

You can import these types of metadata:

Technical metadata: Technical metadata provides the information that is required to create an asset in a project or catalog. Technical metadata provides asset details, relationships, and the preview of the contents of the asset. For data assets, the technical metadata also allows for data profiling, data quality analysis, and provides access for people to work with the data.
Lineage metadata: Lineage metadata provides the lineage information for the Lineage page of an asset in a catalog and the technical lineage in the MANTA Automated Data Lineage for IBM Cloud Pak for Data UI. Lineage shows where your data comes from, how it changes, and where it moves over time.

Types of assets

You can create the following types of assets by importing metadata:

Data assets: Data tables or files from a connection. If you want to run metadata enrichment or data quality rules on the imported assets, you import them to a project.
Cobol copybooks: The data structure of a COBOL program. You can import COBOL copybook maps, virtual tables, and views into projects and catalogs. To add such assets from mainframes, you must use a Data Virtualization Manager for z/OS connection. The imported assets cannot be profiled, enriched through metadata enrichment, or used in Data Refinery.
Business intelligence assets: The components of business intelligence reports and their relationships.
Transformation script assets: The data transformations that change the format, structure, or values of data and that usually are part of ETL (extract, transform, and load) processes.
Data model assets: The components of physical or logical data models and their relationships.
Data integration assets: The data integration components for the flow of data in ETL jobs.

See Asset types created through metadata import and Supported connectors.

Installed features and license requirements

Depending on the type of metadata that you want to import, you might need to enable features when you install IBM Knowledge Catalog or after installation:

The Advanced metadata import feature. Enable this feature to import technical metadata for asset types other than data assets and COBOL copybooks and to import lineage metadata for any type of asset that has lineage.
The Knowledge graph feature. Enable this feature to be able to view lineage information in the catalog.

For importing lineage, you must also purchase MANTA Automated Data Lineage for IBM Cloud Pak for Data separately. You are provided a common license key that is automatically applied when you install MANTA Automated Data Lineage for IBM Cloud Pak for Data. This key entitles you to a certain number of lineage imports, the so-called script count. Every new lineage import for an object such as a database table counts toward this script count. Reruns of imports don't consume script count. For more information, see MANTA Automated Data Lineage for IBM Cloud Pak for Data Script counting details.

The information about the current script count is available in the Get lineage section when you define the goal of your metadata import.

The following tables lists the feature and license requirements for each type of metadata.

Requirements for metadata import
Metadata type	Asset type	Requires Advanced metadata import feature?	Requires Knowledge graph feature?	Requires MANTA Automated Data Lineage for IBM Cloud Pak for Data license key?
Technical	Data assets	No	No	No
Technical	COBOL copybook assets	No	No	No
Technical	Business intelligence report assets	Yes	No	No
Technical	Transformation script assets	Yes	No	No
Technical	Data integration assets	Yes	No	No
Technical	Data model assets	Yes	No	No
Lineage	Data assets Data integration assets	Yes	Yes	Yes

To enable the Knowledge graph feature, see Determining the optional features to enable.
To enable the Advanced metadata import feature, you must install the MANTA Automated Data Lineage service. For details, see MANTA Automated Data Lineage for IBM Cloud Pak for Data. After the service is installed, enable lineage import. For details, see Enabling lineage import.

Next steps

Learn more

Parent topic: Curating data