IBM Support

High CPU / Hang on java.util.HashMap.findNonNullKeyEntry() due to non-thread-safe usage of HashMap

Troubleshooting


Problem

High CPU or hang symptoms may be observed on java.util.HashMap.findNonNullKeyEntry when a HashMap is utilized in a non-thread-safe manner. This is an unsafe coding practice that can occur at any level of development when using the HashMap API: application, application server, third party module, etc. The following will outline how to identify the problematic code and some possible methods to resolve the issue.

Symptom

High CPU or hung threads with an executing stack similar to the following:


    at java.util.HashMap.findNonNullKeyEntry()
    at java.util.HashMap.getEntry()
    at java.util.HashMap.get()
    at com.xyz.methodABC()
    ...
or
    at java.util.HashMap.findNonNullKeyEntry()
    at java.util.HashMap.putImpl()
    at java.util.HashMap.put()
    at com.xyz.methodABC()
    ...

The specific stack may vary.

As a more general symptom of this issue, the executing stack will show the problematic code calling a method within the HashMap API which will in turn call the findNonNullKeyEntry() method.

Resolving The Problem

In order to resolve these type of issues, the problematic code must first be identified.

In the examples above, the problematic code is com.xyz.methodABC(). This is determined by looking at the code that is making the call to the HashMap API. In both cases we see that com.xyz.methodABC() is calling the HashMap API, and therefore it is the problematic code. In the first example com.xyz.methodABC() is calling the get() method, and in the second example com.xyz.methodABC() is calling the put() method.

Once the problematic code has been identified, the developers of that code will need to review their usage of the HashMap API (Java 5, Java 6, Java 7). The discovery of high CPU or hung threads in states such as those described above is compelling evidence that the usage of the HashMap API has not been done in a thread-safe manner.

To correct the issue, the usage of HashMap objects should be done in a thread-safe (synchronized) manner as the API mandates. Alternatively, an object that is similar to a HashMap but intrinsically thread-safe could be used instead, a Hashtable or a ConcurrentHashMap for example.

To understand this further, please see the HashMap API document:


http://docs.oracle.com/javase/7/docs/api/java/util/HashMap.html

HashMaps and WebSphere Application Server

As noted previously, this is not an issue that is unique to applications or third party modules. This is an issue that can occur at any level of development where the HashMap API is used, including WebSphere Application Server itself. There are a number of WebSphere Application Server APARs which address such issues. If one of these specific issues is encountered, it is recommended that the referenced Fix Pack (or greater) be applied to correct the issue.

       APAR     Affected Versions        Released in Fix Pack      
    PK74719                   7.0                     7.0.0.1
    PK97665                   7.0                     7.0.0.9
    PM00692              6.1, 7.0           6.1.0.31, 7.0.0.9
    PM02318                   7.0                    7.0.0.11
    PM48600              7.0, 8.0           7.0.0.21, 8.0.0.2
    PM44032              7.0, 8.0           7.0.0.21, 8.0.0.2
    PM53640              7.0, 8.0           7.0.0.23, 8.0.0.4
      Note: This list contains the most common WebSphere Application Server APARs related to these type of issues; this list is by no means comprehensive or exhaustive.

    A Final Note: IBM Java 6 SDK Enhancements

    There have been some enhancements to the IBM Java 6 SDK to mitigate the likelihood of problems due to non-thread-safe usage of HashMap objects:


         APAR              Released       Shipped with Fix Pack
      IZ51855            Java 6 SR6                     7.0.0.7
      IZ73767            Java 6 SR9                    7.0.0.15

    However, it is still the responsibility of the code using the HashMap objects to ensure that it is done in a thread-safe (synchronized) manner as the API requires. These enhancements cannot and will not protect against all improper usage of HashMap objects.

    [{"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"Not Applicable","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"}],"Version":"8.0;7.0","Edition":"Base;Express;Network Deployment","Line of Business":{"code":"LOB45","label":"Automation"}}]

    Document Information

    Modified date:
    15 June 2018

    UID

    swg21597581