The leak detection policy for the WebSphere® Application
Server is turned off by default. You can configure a leak detection,
prevention, and action policy to accommodate your applications and
environment so that potential memory leaks are reported and acted
upon. Leak detection, prevention, and proactive fixing provides for
protection and resiliency for servers that face persistent out of
memory errors. When a classloader memory leak is detected, WebSphere® Application Server
notifies you with informational messages in the log and by taking
JVM heapdumps so that you can troubleshoot the problem. Optionally,
you might also choose to have WebSphere Application
Server mitigate, and if possible, fix the memory leak using reflection
and other techniques.
Before you begin
A common error in Java™ Platform,
Enterprise Edition (Java EE)
applications is a classloader memory leak. A memory leak can result
from subtle application bugs or a more complex use, such as third
party libraries. System resources, such as CPU time due to garbage
collection and the Java heap
are consumed when a leak is present. A system can become unresponsive
even though all other resources are available. Unless a protection
and early warning system is built in, the system might remain in this
degraded state and ultimately die due to an out of memory error.
Note: You can configure WebSphere Application Server to detect,
prevent, and take action, if possible, on classloader memory leaks
using the memory leak detection policy. For more information about
memory leaks, read the Memory Leaks in Java Platform,
Enterprise Edition applications topic.
About this task
The leak detection option is disabled by default. You can
use Java virtual machine (JVM)
custom properties to adjust the leak policy values, such as, enable
and disable leak detection, action, and prevention. These custom properties
are only applicable to a stand-alone server or managed application
server and not to a node agent, admin agent, job manager, or deployment
manager. When the application or the server is shutting down, WebSphere Application Server
determines the classloaders that have leaked and are held references
to all of associated loaded classes and objects. If a classloader
leak is detected, a heapdump or systemdump is taken. All persistent
configurations of this service are completed using JVM custom properties.
There are no administrative console panels. At runtime, use the MemoryLeakConfig
and MemoryLeakAdmin mbeans for configuration and administration respectively;
however, the configuration changes are not persisted until JVM custom
properties are configured.
The MemoryLeak service and its mbean
are active only in an application server that hosts applications and
services requests. This service is not active on a Deployment Manager,
node agent, administrative agent, or other servers types like WebSphere proxy server and
so on.
Tip: Configure the JVM com.ibm.ws.runtime.component.disableMemoryLeakService
custom property to permanently disable the service.
Procedure
- Create or modify JVM custom properties to enable various
aspects of the memory leak service, based on the table that follows.
Read Java virtual machine custom properties to change the values of the
JVM custom properties.
Table 1. com.ibm.ws.runtime.component.MemoryLeakConfig.detectAppCLLeaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.detectAppCLLeaks |
Description |
When the server is shutting down or an application
stops, WebSphere Application
Server determines the classloaders that have leaked and issues warnings
and other additional information that aids in debugging the memory
leak. See also the Improved classloader leak detection PMR. |
Default |
false |
Table 2. com.ibm.ws.runtime.component.MemoryLeakConfig.clearAppCLLeaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.clearAppCLLeaks |
Description |
Enable proactive classloader leak mediation
and fixing. When this property is set to true, WebSphere Application Server mediates on
behalf of the application to remedy any classloader leaks that are
detected. |
Default |
false |
Table 3. com.ibm.ws.runtime.component.MemoryLeakConfig.preventJreMemoryLeaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.preventJreMemoryLeaks |
Description |
Enable WebSphere Application
Server to eliminate certain classes of memory leaks that are caused
by the JRE loading singletons on the thread context classloader. |
Default |
true |
Table 4. com.ibm.ws.runtime.component.MemoryLeakConfig.generateHeapDumps
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.generateHeapDumps |
Description |
Set to true to cause a heapdump to be created
when a memory leak is detected . |
Default |
true |
Table 5. com.ibm.ws.runtime.component.MemoryLeakConfig.leakSweeperDelay
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.leakSweeperDelay |
Description |
Delay after an application or module stops to
check for classloader leaks. |
Default |
10000 (milliseconds) |
Table 6. com.ibm.ws.runtime.component.MemoryLeakConfig.monitorSystemApps
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.monitorSystemApps |
Description |
Set to true to watch for memory leaks on system
applications that are shipped by IBM®,
such as the administrative console. |
Default |
false |
Attention: You can fine tune the leak action policy
behavior using the following thread and timer leaks JVM custom properties.
The ThreadLocal, thread, timer and static leak custom properties are
applicable only if com.ibm.ws.runtime.component.MemoryLeakConfig.clearAppCLLeaks
is set to true.
Table 7. com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesInterruptThreads. Thread and timer leaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesInterruptThreads |
Description |
Set to true for WebSphere Application Server to attempt
to interrupt threads that are started by the web application. Interrupting
threads is performed using the Thread.interrupt() method. There is
a possibility that the target thread might not respond to the interrupt. |
Default |
true |
Table 8. com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesHttpClientKeepAliveThread. Thread and timer leaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesHttpClientKeepAliveThread |
Description |
If an HttpClient keep-alive timer thread is
not started by this web application and is still running, WebSphere Application Server
changes the context class loader from the current classloader to its
parent to prevent a memory leak. The keep-alive timer thread stops
on its own when the keep-alive threads all die. However, on a busy
system that might not happen for some time. |
Default |
true |
Table 9. com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesStopTimerThreads. Thread and timer leaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesStopTimerThreads |
Description |
Set to true for WebSphere Application Server to stop any
java.util.TimerThreads that are started by the web application. |
Default |
false |
Table 10. com.ibm.ws.runtime.component.MemoryLeakConfig.jvmThreadGroupNames. Thread and timer leaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.jvmThreadGroupNames |
Description |
List of ThreadGroup names to ignore when scanning
for threads that are started by the web application that must be shut
down. This list is delineated by underscores. |
Default |
system__RMI Runtime |
Attention: The following static class variable
leaks properties apply only if com.ibm.ws.runtime.component.MemoryLeakConfig.clearAppCLLeaks
is set to true.
Table 11. com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesStatic. Static class variable leaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesStatic |
Description |
Set to true for WebSphere Application Server to attempt
to make final fields null from loaded classes when a web application
stops, as a work around for garbage collection bugs and application
coding errors. Applications without memory leaks using recent JVMs
should operate correctly when this option is set to false. |
Default |
false |
Table 12. com.ibm.ws.runtime.component.MemoryLeakConfig.filterPrefixes. Static class variable leaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.filterPrefixes |
Description |
Member attributes starting with these filters
are not set to null when clearReferencesStatic is true. |
Default |
java javax com.ibm org sun com.sun. The list
is delineated by spaces. |
Attention: The following Threadlocal leaks properties
apply only if com.ibm.ws.runtime.component.MemoryLeakConfig.clearAppCLLeaks
is set to true.
![[AIX Solaris HP-UX Linux Windows]](../images/ngdist.svg)
![[IBM i]](../images/ngibmi.svg)
Table 13. com.ibm.ws.runtime.component.MemoryLeakConfig.checkThreadLocalLeaks. Threadlocal leaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.checkThreadLocalLeaks |
Description |
Determines if ThreadLocal leaks should be checked
when an application stops. |
Default |
false |
![[AIX Solaris HP-UX Linux Windows]](../images/ngdist.svg)
![[IBM i]](../images/ngibmi.svg)
Table 14. com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesThreadLocal. Threadlocal leaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesThreadLocal |
Description |
Eliminate threadlocal leaks by renewing threads
in the threadpool when an application stops. This property applies
only to distributed platforms. |
Default |
true |
Attention: The following properties apply only
if clearReferencesThreadLocal is set to true.
![[AIX Solaris HP-UX Linux Windows]](../images/ngdist.svg)
![[IBM i]](../images/ngibmi.svg)
Table 15. com.ibm.ws.runtime.component.MemoryLeakConfig.renewThreadPoolNames. Threadlocal leaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.renewThreadPoolNames |
Description |
Threadpool names that WebSphere Application Server must renew
when a threadlocal leak is detected. |
Default |
"WebContainer" list is delineated by spaces. |
![[AIX Solaris HP-UX Linux Windows]](../images/ngdist.svg)
![[IBM i]](../images/ngibmi.svg)
Table 16. com.ibm.ws.runtime.component.MemoryLeakConfig.threadPoolRenewalDelayFactor. Threadlocal leaks
Information |
Value |
Name |
com.ibm.ws.runtime.component.MemoryLeakConfig.threadPoolRenewalDelayFactor |
Description |
Control the total amount of time to wait for
thread pool to be renewed. The amount of time to wait for threadpool
renewal = threadPoolRenewalDelayFactor * threadPool.getKeepAliveTime() |
Default |
10. For the web container threadpool, the renewal
delay is (10 * 60000 ms) 6 minutes. |
![[AIX Solaris HP-UX Linux Windows]](../images/ngdist.svg)
![[IBM i]](../images/ngibmi.svg)
Table 17. com.ibm.ws.util.ThreadPool.DEFAULT_THREAD_RENEWAL_DELAY. Threadlocal leaks
Information |
Value |
Name |
com.ibm.ws.util.ThreadPool.DEFAULT_THREAD_RENEWAL_DELAY |
Description |
Threads in the pool renew after an application
stops. Use this custom property to avoid renewing all threads at the
same time. This delay is observed between two threads being renewed. |
Default |
1000 (value is in ms). If the value is negative,
threads are never renewed. |
These JVM custom properties are persisted in the WebSphere Application Server
configuration model in the
server.xml file. The
following code snippet displays the persisted leak policy configuration
from the
server_home/config/cells/nodes/servers/server.xml file
of an unmanaged server:
<jvmEntries xmi:id="JavaVirtualMachine_1183122130078" verboseModeClass="true" verboseModeGarbageCollection="true" verboseModeJNI="false"
runHProf="false" hprofArguments="" debugMode="false" debugArgs="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=7777"
genericJvmArguments="-agentlib:getClasses -Xquickstart -Xalwaysclassgc" executableJarFileName="" disableJIT="false">
<systemProperties xmi:id="Property_1317048628648" name="com.ibm.ws.runtime.component.MemoryLeakConfig.detectAppCLLeaks" value="true" />
<systemProperties xmi:id="Property_1318975518491" name="com.ibm.ws.runtime.component.MemoryLeakConfig.clearAppCLLeaks" value="true" />
<systemProperties xmi:id="Property_1318955284241" name="com.ibm.ws.runtime.component.MemoryLeakConfig.generateSystemDumps" value="false" />
<systemProperties xmi:id="Property_1319119976147" name="com.ibm.ws.runtime.component.MemoryLeakConfig.generateHeapDumps" value="true" />
<systemProperties xmi:id="Property_1317048628649" name="com.ibm.ws.runtime.component.MemoryLeakConfig.monitorSystemApps" value="false" />
</jvmEntries>
- Click Apply.
- Click OK.
- Save the changes.
- Restart the Application Server for the changes to take
effect.
Example
The Memory Leak policy for WebSphere Application Server can be configured
and persisted using JVM custom properties as described in this example.
At runtime the memory leak detection, prevention and policy configuration
can be changed using the MemoryLeakConfig mbean.
Administration
of the Memory leak policy can be carried out using the MemoryLeakAdmin
mbean. The leak policy affects how the application server responds
to a classloader memory leak when an application or when the server
is stopped.
You can adjust the Memory leak policy settings
by using the wsadmin scripting interface. These changes take effect
immediately, but do not persist to the server configuration, and are
lost when the server is restarted. The following script provides an
example of how to configure and administer the memory leak policy
using wsadmin jacl scripting :
# Scripting in JACL
# Get the object name of the MemoryLeak Configiration object you want to change the values on
set leakConfig [$AdminControl completeObjectName "type=MemoryLeakConfig,*"]
WebSphere:cell=smitaNode03Cell,name=LeakConfig,type=MemoryLeakConfig,node=smitaNode03,process=server1
# Get the object name of the MemoryLeak Administration object you want to issue operations
set leakAdmin [$AdminControl completeObjectName "type=MemoryLeakAdmin,*"]
WebSphere:cell=smitaNode03Cell,name=LeakAdmin,type=MemoryLeakAdmin,node=smitaNode03,process=server1
# Look at all the attributes of the MemoryLeakConfig mbean
wsadmin>$Help all $leakConfig
Name: WebSphere:cell=smitaNode03Cell,name=LeakConfig,type=MemoryLeakConfig,node=smitaNode03,process=server1
Description: Information on the management interface of the MBean
Class name: com.ibm.ws.runtime.component.MemoryLeakConfig
Attribute Type Access
JvmThreadGroupNames java.lang.String RW
FilterPrefixes java.lang.String RW
RenewThreadPoolNames java.lang.String RW
DetectAppCLLeaks boolean RW
ClearAppCLLeaks boolean RW
MonitorSystemApps boolean RW
NoDumps boolean RW
GenerateHeapDumps boolean RW
GenerateSystemDumps boolean RW
ClearReferencesStatic boolean RW
ClearReferencesInterruptThreads boolean RW
ClearReferencesStopTimerThreads boolean RW
ClearReferencesHttpClientKeepAliveThread boolean RW
ClearReferencesThreadLocal boolean RW
LeakSweeperDelay int RW
ThreadPoolRenewalDelayFactor int RW
PreventJreMemoryLeaks boolean RW
LeakConfiguration java.lang.String RO
Operation
Notifications
Constructors
# Print the current Memory Leak Policy configuration on the console
wsadmin>$AdminControl getAttribute $leakConfig LeakConfiguration
MemoryLeakConfig [getClass()=class com.ibm.ws.runtime.component.MemoryLeakConfig, hashCode()=37266644
preventJreMemoryLeaks true
detectAppCLLeaks true
monitorSystemApps false
leakSweeperDelay 10000
clearAppCLLeaks true
clearReferencesStopTimerThreads false
clearReferencesHttpClientKeepAliveThread true
clearReferencesInterruptThreads true
jvmThreadGroupNames [system, RMI Runtime]
clearReferencesStatic true
filterPrefixes [java., javax., com.ibm., org., sun., com.sun]
clearReferencesThreadLocal true
renewThreadPoolNames [WebContainer]
threadPoolRenewalDelayFactor 1
noDumps false
generateHeapDumps true
generateSystemDumps false
# Change the configuration
wsadmin>$AdminControl setAttribute $leakConfig ThreadPoolRenewalDelayFactor 10
wsadmin>$AdminControl setAttribute $leakConfig ClearReferencesStopTimerThreads true
# See the updated configuration
wsadmin>$AdminControl getAttribute $leakConfig LeakConfiguration
MemoryLeakConfig [getClass()=class com.ibm.ws.runtime.component.MemoryLeakConfig, hashCode()=37266644
preventJreMemoryLeaks true
detectAppCLLeaks true
monitorSystemApps false
leakSweeperDelay 10000
clearAppCLLeaks true
clearReferencesStopTimerThreads true
clearReferencesHttpClientKeepAliveThread true
clearReferencesInterruptThreads true
jvmThreadGroupNames [system, RMI Runtime]
clearReferencesStatic true
filterPrefixes [java., javax., com.ibm., org., sun., com.sun]
clearReferencesThreadLocal true
renewThreadPoolNames [WebContainer]
threadPoolRenewalDelayFactor 10
noDumps false
generateHeapDumps true
generateSystemDumps false
# Look at all the operations of the MemoryLeakAdmin mbean
wsadmin>$Help all $leakAdmin
Name: WebSphere:cell=smitaNode03Cell,name=LeakAdmin,type=MemoryLeakAdmin,node=smitaNode03,process=server1
Description: Information on the management interface of the MBean
Class name: com.ibm.ws.runtime.component.MemoryLeakAdmin
Operation
java.lang.String findLeaks()
java.lang.String fixLeaks()
java.lang.String fixLeaks(java.lang.String)
# Find current classloader memory leaks
wsadmin>$AdminControl invoke $leakAdmin findLeaks
CWMML0028I: The following web applications were stopped (reloaded, undeployed),
but their classes from previous runs are still loaded in memory,
thus causing a memory leak.[[78577.075.724.NWALogging#NWALoggingEJB.jar]].
# Fix ALL current classloader memory leaks
wsadmin>$AdminControl invoke $leakAdmin fixLeaks
CWMML0036I: Please watch the SystemOut log for results of the fix leak operation.
wsadmin>$AdminControl invoke $leakAdmin fixLeaks {"78577.075.724.NWALogging#NWALoggingEJB.jar"}
CWMML0036I: Please watch the SystemOut log for results of the fix leak operation.