IBM Support

High kernel CPU and memory exhaustion while forking processes on WebSphere Application Server

Troubleshooting


Problem

WebSphere Application Server hosts can experience elevated CPU utilization in the kernel space while forking processes. Hosts can also experience an exhaustion of memory which can lead to paging and thus degraded performance.

Cause

Forking processes run on the system space. The spike in system CPU utilization is expected. Due to the larger address space there is more processing needed to fork a process, which will require more CPU resources. Also, when a process is forked an exact copy of that process will be created, which will require more memory. This immediate need for large amounts of memory (depending on process size) can lead to to paging and slow server performance.

Diagnosing The Problem

The stack trace of the calling thread will show evidence of a fork. Here is an example on Linux:


at java/lang/Object.wait(Native Method)
at java/lang/Object.wait(Object.java:167(Compiled Code))
at java/lang/UNIXProcess$Gate.waitForExit(UNIXProcess.java:88(Compiled Code))
at java/lang/UNIXProcess.<init>(UNIXProcess.java:262(Compiled Code))
at java/lang/ProcessImpl.start(ProcessImpl.java:112(Compiled Code))
at java/lang/ProcessBuilder.start(ProcessBuilder.java:464(Compiled Code))
at java/lang/Runtime.exec(Runtime.java:604(Compiled Code))
at java/lang/Runtime.exec(Runtime.java:442(Compiled Code))
at java/lang/Runtime.exec(Runtime.java:339(Compiled Code))
.....

From this stack trace we can see the java Runtime.exec() method being called which prepares the current process to create a new child process, then the child process is created in the ProcessBuilder.start() method. After the child process is generated the calling thread will go into UNIXProcess$Gate.waitForExit() which will make the calling thread wait until the child process is exited.

Resolving The Problem

This is working as designed. To alleviate high CPU utilization it would be best to add more CPU resources or change the order of how the application forks processes. If too many processes are forked at once, the available memory will be exhausted and the residual required memory will be taken from the hard disk (paging), which can costly towards performance.

[{"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"100% CPU Usage","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF035","label":"z\/OS"}],"Version":"8.5;8.0;7.0;6.1","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
21 June 2018

UID

swg21678440