IBM Support

Java heap corruption and crash in WebSphere Application Server

Troubleshooting


Problem

You have Java™ heap corruption and your WebSphere Application Server crashes frequently in random thread stacks. You are missing AIX 5.3 APAR IZ52719. The same issues have also been noticed in AIX 6.1.

Symptom

WebSphere Application Server frequently crashes in random thread stacks that are inconsistent. There will be frequent crashes in application code, garbage collection, Java code, and so on. The crashes will not be related to each other and will not show any consistent patterns. These random crashes are common with memory corruption issues.

This issue occurs after security patches for AIX 5.3 and AIX 6.1 have been applied to a server.

Cause

Corruption of the Java heap due to the "GETGRENT_R" routine. The problem is addressed in the following APAR reference:

IZ52585: "GETGRENT_R" ROUTINE CAUSES HEAP CORRUPTION . APPLIES TO AIX
5300-08
http://www.ibm.com/support/docview.wss?uid=isg1IZ52585

Environment

AIX development has noted the problems can occur within Aix 5.3 and AIX 6.1 bos.rte.libc levels in multiple update levels.

Review the AIX technote referenced below, IBM Fix pack information for: \"GETGRENT_R\" ROUTINE CAUSES HEAP CORRUPTION , for a listing of additional AIX versions and update levels affected by this issue.

Diagnosing The Problem

Before requesting assistance from AIX support to confirm if the condition is occurring in the client's environment, while logged into the environment as actual root user, please gather the information/output from the following AIX commands:


          > prtconf | grep "Serial Number"
          > uname -M
          > lparstat -i
          > instfix -ia
          > oslevel -s
          > emgr -lv3
          > lslpp -la | grep libc

Resolving The Problem

Individual AIX APARS are not available. They are provided in the maintenance level and service pack updates. The problems are resolved by updating AIX 5.3 to the latest service pack for the maintenance level installed that has included APAR IZ52585 and IZ52719. In the case of AIX 5.3 ML 09, the problem is resolved by updating to service pack 4, AIX 5.3_ML09_SP4. Updates for AIX 6.1 are noted in the technote referenced below as well.

See below link for AIX 5.3 and AIX 6.1 maintenance and service pack levels where the fix is included.


Fix Pack information for: \\"GETGRENT_R\\" ROUTINE CAUSES HEAP CORRUPTION .
http://www.ibm.com/support/docview.wss?uid=isg1fixinfo111945

[{"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Crash","Platform":[{"code":"PF002","label":"AIX"}],"Version":"9.0;8.5;8.0;7.0","Edition":"Base;Developer;Express;Network Deployment","Line of Business":{"code":"LOB45","label":"Automation"}},{"Product":{"code":"SS7JFU","label":"WebSphere Application Server - Express"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"General","Platform":[{"code":"PF002","label":"AIX"}],"Version":"7.0;6.1;6.0.2;6.0.1;6.0","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
19 August 2022

UID

swg21399260

Page Feedback