IBM Support

IT44159: OUTOFMEMORYERROR FROM KAFKA CONSUMER NODE WHEN KAFKA SERVER IS RESTARTED

Subscribe to this APAR

By subscribing, you receive periodic emails alerting you to the status of the APAR, along with a link to the fix after it becomes available. You can track this item individually or track all items by product.

Notify me when this APAR changes.

Notify me when an APAR for this component changes.

 

APAR status

  • Closed as program error.

Error description

  • If the Kafka  Consumer Node is configured to authenticate using
    SASL over TLS then under some circumstances when the Kafka
    server is restarted this may cause an OutOfMemoryError with the
    following stack:
    
    3XMTHREADINFO3           Java callstack:
    4XESTACKTRACE                at
    java/nio/HeapByteBuffer.<init>(HeapByteBuffer.java:57)
    4XESTACKTRACE                at
    java/nio/ByteBuffer.allocate(ByteBuffer.java:335)
    4XESTACKTRACE                at org/apache/kafka/common/memory/
    MemoryPool$1.tryAllocate(MemoryPool.java:30)
    4XESTACKTRACE                at org/apache/kafka/common/network
    /NetworkReceive.readFrom(NetworkReceive.java:113)
    4XESTACKTRACE                at org/apache/kafka/common/securit
    y/authenticator/SaslClientAuthenticator.receiveResponseOrToken(
    SaslClientAuthenticator.java:475)
    4XESTACKTRACE                at org/apache/kafka/common/securit
    y/authenticator/SaslClientAuthenticator.receiveKafkaResponse(Sa
    slClientAuthenticator.java:572)
    4XESTACKTRACE                at org/apache/kafka/common/securit
    y/authenticator/SaslClientAuthenticator.authenticate(SaslClient
    Authenticator.java:250)
    4XESTACKTRACE                at org/apache/kafka/common/network
    /KafkaChannel.prepare(KafkaChannel.java:181)
    4XESTACKTRACE                at org/apache/kafka/common/network
    /Selector.pollSelectionKeys(Selector.java:543)
    4XESTACKTRACE                at
    org/apache/kafka/common/network/Selector.poll(Selector.java:481)
    
    4XESTACKTRACE                at org/apache/kafka/clients/Networ
    kClient.poll(NetworkClient.java:551)
    4XESTACKTRACE                at org/apache/kafka/clients/consum
    er/internals/ConsumerNetworkClient.poll(ConsumerNetworkClient.j
    ava:265)
    4XESTACKTRACE                at org/apache/kafka/clients/consum
    er/internals/ConsumerNetworkClient.poll(ConsumerNetworkClient.j
    ava:236)
    4XESTACKTRACE                at org/apache/kafka/clients/consum
    er/internals/ConsumerNetworkClient.poll(ConsumerNetworkClient.j
    ava:215)
    4XESTACKTRACE                at org/apache/kafka/clients/consum
    er/internals/Fetcher.getTopicMetadata(Fetcher.java:374)
    4XESTACKTRACE                at org/apache/kafka/clients/consum
    er/KafkaConsumer.partitionsFor(KafkaConsumer.java:1949)
    4XESTACKTRACE                at org/apache/kafka/clients/consum
    er/KafkaConsumer.partitionsFor(KafkaConsumer.java:1917)
    4XESTACKTRACE                at com/ibm/broker/connector/kafka/
    KafkaIIBConsumer.initialise(KafkaIIBConsumer.java:177)
    4XESTACKTRACE                at com/ibm/broker/connector/kafka/
    KafkaIIBConsumer.start(KafkaIIBConsumer.java:512)
    5XESTACKTRACE                   (entered lock: com/ibm/broker/c
    onnector/kafka/KafkaIIBConsumer@0x00000000C0A94038,entry count:
    4XESTACKTRACE                at com/ibm/broker/connector/kafka/
    KafkaInputConnector.start(KafkaInputConnector.java:250)
    
    The issue is caused by a single large allocation made by the
    Kafka Client when it reads unexpected data from a socket.
    

Local fix

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    All users of IBM Integration Bus version 10.1 or IBM App Connect
    version 11 or 12 using the Kafka Nodes.
    
    
    Platforms affected:
    MultiPlatform
    
    ****************************************************************
    PROBLEM DESCRIPTION:
    If the Kafka Consumer Node is configured to authenticate using
    SASL over TLS then under some circumstances when the Kafka
    server is restarted this may cause an OutOfMemoryError with the
    following stack:
    
    3XMTHREADINFO3           Java callstack:
    4XESTACKTRACE                at
    java/nio/HeapByteBuffer.<init>(HeapByteBuffer.java:57)
    4XESTACKTRACE                at
    java/nio/ByteBuffer.allocate(ByteBuffer.java:335)
    4XESTACKTRACE                at
    org/apache/kafka/common/memory/MemoryPool$1.tryAllocate(MemoryPo
    ol.java:30)
    4XESTACKTRACE                at
    org/apache/kafka/common/network/NetworkReceive.readFrom(NetworkR
    eceive.java:113)
    4XESTACKTRACE                at
    org/apache/kafka/common/security/authenticator/SaslClientAuthent
    icator.receiveResponseOrToken(SaslClientAuthenticator.java:475)
    4XESTACKTRACE                at
    org/apache/kafka/common/security/authenticator/SaslClientAuthent
    icator.receiveKafkaResponse(SaslClientAuthenticator.java:572)
    4XESTACKTRACE                at
    org/apache/kafka/common/security/authenticator/SaslClientAuthent
    icator.authenticate(SaslClientAuthenticator.java:250)
    4XESTACKTRACE                at
    org/apache/kafka/common/network/KafkaChannel.prepare(KafkaChanne
    l.java:181)
    4XESTACKTRACE                at
    org/apache/kafka/common/network/Selector.pollSelectionKeys(Selec
    tor.java:543)
    4XESTACKTRACE                at
    org/apache/kafka/common/network/Selector.poll(Selector.java:481)
    4XESTACKTRACE                at
    org/apache/kafka/clients/NetworkClient.poll(NetworkClient.java:5
    51)
    4XESTACKTRACE                at
    org/apache/kafka/clients/consumer/internals/ConsumerNetworkClien
    t.poll(ConsumerNetworkClient.java:265)
    4XESTACKTRACE                at
    org/apache/kafka/clients/consumer/internals/ConsumerNetworkClien
    t.poll(ConsumerNetworkClient.java:236)
    4XESTACKTRACE                at
    org/apache/kafka/clients/consumer/internals/ConsumerNetworkClien
    t.poll(ConsumerNetworkClient.java:215)
    4XESTACKTRACE                at
    org/apache/kafka/clients/consumer/internals/Fetcher.getTopicMeta
    data(Fetcher.java:374)
    4XESTACKTRACE                at
    org/apache/kafka/clients/consumer/KafkaConsumer.partitionsFor(Ka
    fkaConsumer.java:1949)
    4XESTACKTRACE                at
    org/apache/kafka/clients/consumer/KafkaConsumer.partitionsFor(Ka
    fkaConsumer.java:1917)
    4XESTACKTRACE                at
    com/ibm/broker/connector/kafka/KafkaIIBConsumer.initialise(Kafka
    IIBConsumer.java:177)
    4XESTACKTRACE                at
    com/ibm/broker/connector/kafka/KafkaIIBConsumer.start(KafkaIIBCo
    nsumer.java:512)
    5XESTACKTRACE                   (entered lock:
    com/ibm/broker/connector/kafka/KafkaIIBConsumer@0x00000000C0A940
    38,entry count:
    4XESTACKTRACE                at
    com/ibm/broker/connector/kafka/KafkaInputConnector.start(KafkaIn
    putConnector.java:250)
    
    The issue is caused by a single large allocation made by the
    Kafka Client when it reads unexpected data from a socket.
    

Problem conclusion

  • The product now catches OutOfMemoryError with this specific
    stack trace in order to prevent a failure. Connection will be
    re-attempted once the server is available again.
    
    ---------------------------------------------------------------
    The fix is targeted for delivery in the following PTFs:
    
    Version    Maintenance Level
    v11.0      11.0.0.25
    v12.0      10.1.0.3
    
    The latest available maintenance can be obtained from:
    http://www-01.ibm.com/support/docview.wss?rs=849&uid=swg27006041
    
    If the maintenance level is not yet available,information on
    its planned availability can be found on:
    http://www-1.ibm.com/support/docview.wss?rs=849&uid=swg27006308
    ---------------------------------------------------------------
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT44159

  • Reported component name

    APP CONNECT ENT

  • Reported component ID

    5724J0560

  • Reported release

    C00

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2023-07-13

  • Closed date

    2024-01-18

  • Last modified date

    2024-01-18

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    APP CONNECT ENT

  • Fixed component ID

    5724J0560

Applicable component levels

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSDR5J","label":"IBM App Connect Enterprise"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"C00","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
23 January 2024