Ingesting data by using Iceberg copy loader

You can ingest data into IBM® watsonx.data by using Iceberg copy loader through the web console.

watsonx.data on Red Hat® OpenShift®

Before you begin

  • You must have the Administrator role and privileges in the catalog to do ingestion through the web console.
  • Add a bucket for the target catalog. See Adding a bucket-catalog pair.
  • Create a schema in the catalog for the target table. See Creating schemas.
  • Optionally, you can also create a target table in the schema. See Creating tables.

Procedure

  1. Log in to the watsonx.data console.
  2. From the navigation menu, select Data manager and open the Ingest job page in one of the following ways:
    • Select the Ingestion jobs tab and click Create ingestion job.
    • From the Data objects pane, select a Table from the list of tables under Schema and click Ingest data.
  3. In the Ingest data page, select Iceberg copy loader as the mode for ingesting data.
  4. Select a source directory from the Source bucket menu.
    Data objects in the source directory are displayed.
  5. Select the data objects to be ingested from the source directory. Click Next.
    Note: Only Parquet and CSV file formats are supported as source for ingestion.
    Note: You can apply the configuration for Encoding, Escape character, Field delimiter, and Line delimiter for the CSV files.
  6. Specify the target details for Catalog and Schema.
  7. Select Target table option as Existing table to ingest source data into an existing table. Corresponding target tables are listed in the Table drop down.
  8. Select Target table option as New table and enter the name of the target table to create a new table from the source data. Click Next.
  9. Validate the details in Summary page. Click Ingest.