Direct links to fixes
APAR status
Closed as program error.
Error description
Jgroups allocates 1650814064 bytes of memory ( 1.527 GB) and the noapp heap is set to maximum heap of -Xmx1280m (1 GB)so this will always causes the OOM :- 3XEHSTTYPE 07:33:24:346183000 GMT j9mm.126 - at 0000000050F8CD60 java/lang/Thread.run()V, jit 00007FCF323EA580, pc 00007FCF489E0A36 3XEHSTTYPE 07:33:24:346179000 GMT j9mm.126 - at 0000000053644748 org/jgroups/blocks/TCPConnectionMap$TCPConnection$Receiver.run() V, jit 0000000000000000, pc 00007FCF3354D334 3XEHSTTYPE 07:33:24:346175000 GMT j9mm.101 - J9AllocateIndexableObject() returning NULL! 1650814064 bytes requested for object of class 0000000050F79700 from memory space 'Generational' id=00007FCF440427C0 The heapdumps and javacore show minimal heap usage until jgroups trys to request this large amount of memory.
Local fix
STRTC 490911 AC/AC Workaround - Increase heap to 2GB
Problem summary
Users Affected: All Problem Description: JGroups is generating OOM on each node as it was requesting for more memory even though JGroups did not need that much memory for working properly historically. A ticket was raised with JGroups. Further with JGroup suggestion we were able to resolve the issue by setting the port_range = 0 ( We Ship port_range =1 ) in jgroups_cluster.property_string. Sample property string after making changes is as below. Notice that we have "port_range=0" jgroups_cluster.property_string=TCP(bind_addr=x.x.x.x;bind_port= 60061;enable_diagnostics=false;level=ERROR):TCPPING(initial_host s=x.x.x.x[60061],y.y.y.y[60061];port_range=0;timeout=5000;num_in itial_members=2):MERGE2(min_interval=3000;max_interval=5000):FD ALL(interval=5000;timeout=20000):FD(timeout=5000;max_tries=48;le vel=ERROR):VERIFY_SUSPECT(timeout=1500):pbcast.NAKACK(retransmit _timeout=100,200,300,600,1200,2400,4800;discard_delivered_msgs=t rue):pbcast.STABLE(stability_delay=1000;desired_avg_gossip=20000 ;max_bytes=0):pbcast.GMS(print_local_addr=true;join_timeout=5000 ) jgroups_cluster.distribution_property_string=TCP(bind_port=60060 ;thread_pool_rejection_policy=run;enable_diagnostics=false):TCPP ING(initial_hosts=lnvfr99720042[60060];port_range=0;timeout=5000 ;num_initial_members=2):MERGE2(min_interval=3000;max_interval=50 00):FD_SOCK:FD(timeout=5000;max_tries=48;level=ERROR):VERIFY_SUS PECT(timeout=1500):pbcast.NAKACK(retransmit_timeout=3000;discard _delivered_msgs=true):pbcast.STABLE(stability_delay=1000;desired _avg_gossip=20000;max_bytes=0):pbcast.GMS(join_timeout=5000;prin t_local_addr=true) Platforms Affected: All
Problem conclusion
Resolution Summary: A code fix is provided. Delivered In: 5020500_14 5020603 5020601_7 5020602_4
Temporary fix
Comments
APAR Information
APAR number
IT15772
Reported component name
STR B2B INTEGRA
Reported component ID
5725D0600
Reported release
525
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2016-06-17
Closed date
2016-09-29
Last modified date
2017-05-14
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
STR B2B INTEGRA
Fixed component ID
5725D0600
Applicable component levels
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS3JSW","label":"Sterling B2B Integrator"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"5.2.5","Line of Business":{"code":"LOB59","label":"Sustainability Software"}}]
Document Information
Modified date:
11 September 2023