Configuring the Change Data Capture node

Configure the Change Data Capture node to capture all the changed data from your database into IBM® App Connect Enterprise.

Before you begin

Your installed version of IBM App Connect Enterprise must be 12.0.10.0 or later to be able to use this function.

Watch the following video, which demonstrates how to stream changes from a Postgres database: How to use the Change Data Capture node to stream changes from a Postgres database.

If you need to capture data from Db2, obtain a license for IBM InfoSphere Data Replication (IIDR) For more information, see InfoSphere Data Replication.

The Change Data Capture node requires an integration server to run Java 11. For more information about using a specified version of the Java Runtime Environment, see Configuring an integration server to use a specified Java Runtime Environment (JRE).

Set up the database server by completing the following steps:
  1. Go to the Connectors page for Debezium.
  2. Select your database, for example, Db2.
  3. In the scrollable side menu, click the link for setting up your database server, for example Setting up Db2.
  4. Follow the instructions in the link.
The Change Data Capture node requires extra JAR files specific to your database, which you can obtain by completing the following steps:
  1. Go to the Connectors page for Debezium.
  2. Select your database, for example, Db2.
  3. In the scrollable side menu, select Deployment.
  4. Follow the link to download the plug-in archive from Maven Central.
  5. Extract the file into /opt/mqsi/server/cdc (or an equivalent path if you installed elsewhere) so that you have a directory similar to /opt/mqsi/server/cdc/debezium-connector-db2 that contains several JAR files.
  6. If your database is Db2, you must also complete the following steps:
    1. The following JDBC driver for Db2 is required: https://repo1.maven.org/maven2/com/ibm/db2/jcc/11.5.0.0/jcc-11.5.0.0.jar. Download this driver from Maven Central as described in step 3 of Debezium connector for Db2.
    2. Add the https://repo1.maven.org/maven2/com/ibm/db2/jcc/11.5.0.0/jcc-11.5.0.0.jar into the directory where you extracted the plug-in archive at step 5.
  7. Complete any setup steps that are required for your chosen database as described in step 2.b of the "Procedure" section.

About this task

When you put an instance of the Change Data Capture node into a message flow, you can configure it. For more information about configuring message flow nodes, see Configuring a message flow node. The properties of the node are displayed in the Properties view. All mandatory properties that do not have a defined default value are marked with an asterisk. By configuring a Change Data Capture node, you can enable it to respond to changes in your database.

For Db2, the Debezium setup scripts to automatically start and stop data capture work only on Linux®. On other platforms, use Db2 control commands to put tables into capture mode. For more information about Db2 commands, see https://www.ibm.com/docs/en/idr/11.4.0?topic=reference-system-commands-sql-replication-linux-unix-windows-zos.

On other platforms, use Db2 control commands to put tables into capture mode.

Where the commands are listed here: https://www.ibm.com/docs/en/idr/11.4.0?topic=reference-system-commands-sql-replication-linux-unix-windows-zos

Configure the Change Data Capture node by completing the following steps:

Procedure

  1. Optional: On the Description tab, enter a Short description, a Long description, or both. You can also rename the node on this tab.
  2. Specify a policy and update it by completing the following steps:
    1. On the Basic tab, specify a policy by completing one of the following steps:
      • Specify an existing policy by entering a value for Policy, or click Browse and select a property policy from your file system.
      • Create a new policy by clicking Browse and then clicking Create policy.
    2. Update the policy by completing the following steps:
      1. Database Name: Set to the name of the database to connect to.
      2. Connector Class: Set the entry classname for the downloaded plug-in. Some examples are shown in the following list:
        • Db2: io.debezium.connector.db2.Db2Connector.
        • PostgreSQL: io.debezium.connector.postgresql.PostgresConnector.
        • SQL Server: io.debezium.connector.sqlserver.SqlServerConnector.
      3. Hostname: Hostname or IP address of database server.
      4. Port : TCP port of database server.
      5. Security Identity: Credential to use for authentication to the server.
  3. On the Basic tab, click Add, Edit, or Delete to add, edit, or delete entries in the List of Database Tables.
    The tables that you add to the List of Database Tables are the only tables that the Change Data Capture node captures. Changes to any other tables are ignored.
  4. On the Instances tab, select an option for Additional instances pool to determine the pool from which additional instances are obtained. The options are Use Pool Associated with Message Flow, and Use Pool Associated with node.
    Specify a value for Additional instances to determine the number of additional instances that the node can start if the Additional instances pool property is set to Use Pool Associated with node. The default value is 0.
  5. On the Monitoring tab, use Add, Edit, and Delete to create, change, or delete monitoring events for the node. For more information, see Configuring monitoring event sources by using monitoring properties.