Specifying connection settings for a subscription applying to Kafka

To apply changes to Kafka, you provide the Kafka host name and connection credentials.

1. Click Configuration > Subscriptions.
2. Select the subscription and ensure that it is mapped to Kafka.
3. Right-click a subscription and select Kafka Properties.
4. Select either ZooKeeper Server or Kafka REST Proxy as the communication method for Kafka apply:
   • ZooKeeper Server: Uses Java API calls to ZooKeeper. Choose this option for highest performance.
   • Kafka REST Proxy: Uses HTTP requests to the Kafka REST proxy server. The REST API option is appropriate for situations in which communication between the CDC Replication Engine for Kafka target and the actual Kafka server must be routed over HTTP.
5. Under Communication Details, enter values for the following fields:
   • Host name: The host name of the Kafka REST proxy or ZooKeeper server.
   • Port: The port number of the Kafka REST proxy or ZooKeeper server.
     Note: As of Version 11.4.0.2-5544, you can configure the CDC Replication Engine for Kafka to use a broker list in place of a ZooKeeper server connection. To do so, add the broker list to the properties file of the CDC Kafka producer, CDC_Kafka_instance_directory/conf/kafkaproducer.properties, and the CDC Kafka consumer, CDC_Kafka_instance_directory/conf/kafkaconsumer.properties.

     The syntax for a broker list is as follows:

     bootstrap.servers=host1:port1, host2:port2, ..., hostN:portN

     Where the host name and port number in each pair are the host name and port number of a Kafka server in the cluster.

     It is recommended that both kafkaproducer.properties and kafkaconsumer.properties have the same bootstrap.server. If the bootstrap.servers for kafkaproducer.properties and kafkaconsumer.properties are different, then a not-matching warning message is issued. Producer and consumer then use their own bootstrap.servers to connect to their own Kafka clusters.

     When you specify a broker list, all subscriptions in that instance use the broker list in place of the ZooKeeper server host name and port values that are configured at the subscription level. Subscriptions that are configured to use a Kafka REST Proxy are not affected by the presence of a broker list.

   • Encrypted: Optional. Check this box to use a secure socket connection to the REST Proxy. Not functional for ZooKeeper.
   • Topic prefix: Optional. A subscription scope root prefix for topic names.
     Note: If a topic prefix is used for the subscription, the topic name uses this dot-separated structure: topicPrefix.schemaName.tableName. The prefix is used for both the topic name and the internal commit stream metadata topic name, prefix-instanceName-subscriptionName-commitstream. If no prefix is used, the topic name uses this structure: instanceName.subscriptionName.sourcedb.tableSchema.tableName, and the commit stream metadata topic name is instanceName-subscriptionName-commitstream. To configure a user-specified topic name, refer to Enabling a KCOP to specify topic names and map multiple source tables.
   • Commit Stream: Optional. Name of Commit Stream topic.
     Note: As of engine version 11.4.0.2-5503 and Management Console version 11.4.0.2-11004, you can use this field to choose the name of the Commit Stream topic for subscriptions. If the field is left unmodified, the CDC Replication Engine for Kafka uses the following structure for the Commit Stream topic:

     • (If no topic prefix is specified): instanceName-subscriptionName-commitstream
     • (If a topic prefix is specified): prefix-instanceName-subscriptionName-commitstream

     The Commit Stream topic must be unique for every subscription. The Commit Stream topic has a specific hard-coded format and uses its own serializer that is independent of the serializer that is used for data topics or that is custom selected by a KCOP.

     Ensure that the CDC Replication Engine for Kafka is the only application that is writing to the Commit Stream topic. Otherwise, errors will occur while running the CDC Replication Engine for Kafka and while using the Transactionally Consistent Consumer. Refer to standard Kafka practices such as using an Access Control List (ACL) to protect the Commit Stream topic.

6. Under Schema Registry, enter values for the following fields:
   • Host name: The host name of the Schema Registry REST server.
   • Port: The port number of the Schema Registry REST server.
   • Encrypted: Optional. Check this box to use a secure socket connection to the schema registry.
7. Click OK.