Controlling access to Kafka resources

The Event Gateway Service controls access to the event source by managing the incoming requests to consume from the Kafka resource described by the AsyncAPI document.

Clients connecting to the AsyncAPI are authenticated by the Event Gateway Service through the username and password credentials. These credentials are set for the client applications in the Developer Portal as required by the security protocol settings for the AsyncAPI.

The authenticated client users can then access the Kafka resource through the Event Gateway Service by using the credentials provided in your API. These credentials authenticate the Event Gateway Service with your Kafka cluster, and authorizes the actions the gateway service can perform through Kafka Access Control Lists (ACLs).

Note: It is recommended that you provide the minimum access to your Kafka resources for the authenticated client users: read-only access to the specified topic, and read-only access for all the consumer groups. This ensures that connecting clients can only consume as required, and not perform any other actions.

You can also restrict access to a specific consumer group, as explained in the following example.

Example: restricting access for a consumer group

To avoid potential group.id collisions from different clients consuming through the Event Gateway Service, the service defines its own group.id for each API and client when consuming from Kafka.

The Event Gateway Service adds additional information to any group.id presented by a client application to create a unique group.id. This group.id takes the following format when used to consume from Kafka:

group.id = <API_ID>-<PRINCIPLE>-<CLIENT_GROUP_ID>

Where:

  • API_ID: A unique identifier for the AsyncAPI which is being consumed from.

  • PRINCIPLE: The Developer Portal application username provided by a client application when connecting to the Event Gateway Service.

  • CLIENT_GROUP_ID: The group.id set in the client application that is connecting to the Event Gateway Service.

The API_ID for an API is defined in the AsyncAPI document. You can view it in the AsyncAPI editor under Gateway > Invoke > Cluster Config ID.

The Kafka ACL is created based on this information.

If you are using open-source Apache Kafka, the following is an example for updating an ACL. The following command creates a new Read ACL for any consumer group that starts with the API_ID:

bin/kafka-acls.sh --bootstrap-server <BOOTSTRAP_SERVER> --command-config ./config/admin.properties --add --allow-principal User:<KAFKA PRINCIPLE> --operation Read --resource-pattern-type prefix --group <API_ID>

This ACL can then be used in combination with other ACLs to restrict access to topics for specific users (Principals).

For an introduction to Kafka authorization and access control lists, see the following IBM Developer tutorial. For more information about Kafka authorization and ACLs, see the Apache Kafka documentation