Securing communications to Kafka

IBM® Business Automation Insights relies on messages that are sent and received through Kafka topics.

About this task

IBM Business Automation Insights messages are produced by event emitters and Flink ingestion jobs. Flink jobs other than ingestion jobs also consume and process these messages. For all these messages, IBM Business Automation Insights supports SSL encryption of the communications to Kafka, and supports authentication either with a user name and password, or through Kerberos.

You secure communications to Kafka first for the initial configuration of IBM Business Automation Insights and possibly later when you work with it and need to update your configuration. In this case, see Updating your Business Automation Insights deployment to load your configuration changes.

Whichever security mode you set up, see also Securing communications from event emitters to Kafka.
Note:
  • Certificate values must be base64-encoded.
  •  For 18.0.0  and  For 18.0.1  If the server certificate is self-signed, the kafka.caCertificate parameter must also reference the server certificate.
  •  New in 18.0.2  Unless the server certificate is self-signed, the kafka.serverCertificate parameter must contain the root CA certificate.
Parent topic: Securing communications and data

Connecting to IBM Event Streams on IBM Cloud Private

Procedure

  1. Retrieve your certificate as instructed in Retrieving the Kafka broker certificate.
  2. Get an API key.

    You access IBM Event Streams by using an API key that is generated on the IBM Cloud Private management console. For instructions, see the Managing access Event Streams page. For IBM Business Automation Insights, the Operator Event Streams role (or a higher role) is required to read and write to Kafka topics.

  3. Set the security properties as follows.
    kafka.securityProtocol
    SASL_SSL
    kafka.serverCertificate
    The base64-encoded server certificate that you retrieved in step 1.
    kafka.caCertificate
     For 18.0.0  and  For 18.0.1  The base64-encoded signer CA certificate, or the self-signed server certificate that you retrieved in step 1.
     New in 18.0.2  The kafka.caCertificate parameter does not exist any more. You set only the kafka.serverCertificate.
    kafka.username
    token
    kafka.password
    The API key that you retrieved in step 2.

Example

  •  For 18.0.0  and  For 18.0.1 
    kafka.securityProtocol=SASL_SSL
kafka.serverCertificate=<server certificate>
kafka.caCertificate=<Signer CA or self-signed server certificate>
kafka.username=token
kafka.password=<API  key>
  •  New in 18.0.2 
    kafka.securityProtocol=SASL_SSL
kafka.serverCertificate=<server certificate>
kafka.username=token
kafka.password=<API  key>

Connecting to Kafka by using SSL with user name and password authentication

Procedure

Follow the procedure in Connecting to IBM Event Streams on IBM Cloud Private and set the user name and password accordingly.

Connecting to Kafka by using SSL only

Procedure

  1. Retrieve your certificate as instructed in Retrieving the Kafka broker certificate.
  2. Set the security properties as follows.
    kafka.securityProtocol
    SSL
    kafka.serverCertificate
    The base64-encoded server certificate that you retrieved in step 1.
    kafka.caCertificate
     For 18.0.0  and  For 18.0.1  The base64-encoded signer CA certificate, or the self-signed server certificate that you retrieved in step 1.
     New in 18.0.2  The kafka.caCertificate parameter does not exist any more. You set only the kafka.serverCertificate.

Example

  •  For 18.0.0  and  For 18.0.1 
    kafka.securityProtocol=SSL
kafka.serverCertificate=<server certificate>
kafka.caCertificate=<Signer CA or self-signed server certificate>
  •  New in 18.0.2 
    kafka.securityProtocol=SSL
kafka.serverCertificate=<server certificate>

Connecting to Kafka by using Kerberos

Procedure

  1. Enable Kerberos as described in Enabling Kerberos.
  2. Set the security properties as follows.
    kafka.securityProtocol
    SASL_PLAINTEXT
    kafka.saslKerberosServiceName
    The name of the Kerberos principal.
    kafka.securityProtocol=SASL_PLAINTEXT
kafka.saslKerberosServiceName=<the principal name of the Kafka broker>

Connecting to Kafka by using SSL with Kerberos authentication

Procedure

  1. Retrieve your certificate as instructed in Retrieving the Kafka broker certificate.
  2. Enable Kerberos as described in Enabling Kerberos.
  3. Set the security properties as follows.
    kafka.securityProtocol
    SASL_SSL
    kafka.serverCertificate
    The base64-encoded server certificate that you retrieved in step 1.
    kafka.caCertificate
     For 18.0.0  and  For 18.0.1  The base64-encoded signer CA certificate, or the self-signed server certificate that you retrieved in step 1.
     New in 18.0.2  The kafka.caCertificate parameter does not exist any more. You set only the kafka.serverCertificate.
    kafka.saslKerberosServiceName
    The name of the Kerberos principal.
    kafka.securityProtocol=SASL_SSL
kafka.serverCertificate=<server certificate>
kafka.caCertificate=<Signer CA or self-signed server certificate>
kafka.saslKerberosServiceName=<the principal name of the Kafka broker>

Example

  •  For 18.0.0  and  For 18.0.1 
    kafka.securityProtocol=SASL_SSL
kafka.serverCertificate=<server certificate>
kafka.caCertificate=<Signer CA or self-signed server certificate>
kafka.saslKerberosServiceName=<the principal name of the Kafka broker>
  •  New in 18.0.2 
    kafka.securityProtocol=SASL_SSL
kafka.serverCertificate=<server certificate>
kafka.saslKerberosServiceName=<the principal name of the Kafka broker>

Connecting to Kafka with no security

About this task

You might want to set up a simple development environment where you focus on developing and testing features, and keep security disabled.
Attention: This practice is not recommended because it leaves access to your data open to unauthorized users.

Procedure

Set the kafka.securityProtocol property to PLAIN_TEXT.