IBM Support

TroubleShooting: Classloader problems for WebSphere Application Server

Troubleshooting


Problem

This is a problem determination document to help with resolving class loader problems for all releases and editions of WebSphere Application Server product.

Resolving The Problem

Tab navigation

  • Learning more- selectIf you have recently updated your application, or if you have reason to suspect class caches may be corrupted (for example, if the server JVM has failed to shut down properly), the first thing to try is to clear JVM's shared class cache and WebSphere Application Server's OSGi class cache. Instructions can be found in the article How to clear the WebSphere class caches ed tab,
  • Troubleshooting
  • Collecting data


If you have recently updated your application, or if you have reason to suspect class caches may be corrupted (for example, if the server JVM has failed to shut down properly), the first thing to try is to clear JVM's shared class cache and WebSphere Application Server's OSGi class cache. Instructions can be found in the article How to clear the WebSphere class caches.
If that does not resolve your issue, you will need to collect detailed information as described in MustGather: Classloader problems for WebSphere Traditional.
In the bullets below there are some examples of common class loader problems as well as some basic information on how to resolve them: 
 
  • java.lang.ClassNotFoundException - This exception is thrown when the JVM tries to load a class, but no definition for the class with the specified name could be found. When debugging ClassNotFoundExceptions we need to look at which class cannot be found and make sure that the class is visible on the logical class path of the class loader which was invoked to load the class.
    The following is an example of a ClassNotFoundException you might see:

    java.lang.ClassNotFoundException: com.ibm.faces.webapp.JSResourceServlet

    For an exception such as this one, we need to first determine where JSResourceServlet.class is located. Is the class in an application utility JAR, a shared library JAR or path, an embedded RAR, a WebSphere Application Server JAR, an EJB JAR, a WAR class or library JAR, a resource driver JAR, etc...?

    Then we need to make sure that this location is visible on the class path of the class loader expected to load the class. The Application Server class loader and JVM verbose class loader traces will assist determining whether these JARs or paths exist on the class paths of the application, JVM or Application Server class loaders, and whether any other classes were loaded from these JARs or paths.

    You should also open the jar that contains this class and ensure that the class is there, and that this jar has no permission issues, such as read only access for the root user.
     
  • java.lang.NoClassDefFoundError - This error is similar, but not identical, to a ClassNotFoundException, in that it represents the JVM’s failure to define a class. Notably, a NoClassDefFoundError is almost never a root cause error – rather, it’s generally caused by some other exception, which prevented the JVM successfully defining the class. The most essential step in investigating a NoClassDefFoundError is to determine the root cause exception – typically, there will be a “caused by” chain of exceptions that will contain the root error. As this error is often caused by a ClassNotFoundException, we typically apply the same technique described above. Here we have a sample exception:
     
    [4/16/14 14:05:08:756 CDT] 00000014 ContextLoader E org.springframework.web.context.ContextLoader initWebApplicationContext
    Context initialization failed
    java.lang.NoClassDefFoundError:org.springframework.beans.factory.config.BeanExpressionResolver
        at java.lang.J9VMInternals.verifyImpl(Native Method)
    .....
    Caused by: java.lang.ClassNotFoundException: org.springframework.beans.factory.config.BeanExpressionResolver
        at java.net.URLClassLoader.findClass(URLClassLoader.java:434)
        at com.ibm.ws.bootstrap.ExtClassLoader.findClass(ExtClassLoader.java:204)
        at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:665)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:644)
    …

    From the trace.log, look for the class load for the suspect class

    [4/16/14 14:05:08:753 CDT] 00000014 CompoundClass >  loadClass org.springframework.beans.factory.config.BeanExpressionResolver this=com.ibm.ws.classloader.CompoundClassLoader@18fd0448[PL][war:XYZ/XYZ-2.6.1.war] Entry
    [4/16/14 14:05:08:753 CDT] 00000014 SinglePathCla 3   failed to read
    org.springframework.beans.factory.config.BeanExpressionResolver from /app/WAS80/APPS/XYZ.ear/XYZ-2.6.1.war/WEB-INF/lib/spring-2.0.6.jar
    [4/16/14 14:05:08:753 CDT] 00000014 CompoundClass >  loadClass org.springframework.beans.factory.config.BeanExpressionResolver this=com.ibm.ws.classloader.CompoundClassLoader@4eb085d2[PL][app:XYZ] Entry
    [4/16/14 14:05:08:755 CDT] 00000014 CompoundClass <  loadClass org.springframework.beans.factory.config.BeanExpressionResolver failed Exit

    The above entries tell us that the class loader tried to load the class org.springframework.beans.factory.config.BeanExpressionResolver using the class loader for the web application XYZ/XYZ-2.6.1.war, and it specifically searched this jar file:
     /app/WAS80/APPS/XYZ.ear/XYZ-2.6.1.war/WEB-INF/lib/spring-2.0.6.jar  and did not find the class.

    NoClassDefFoundErrors can also be caused by a failure during the static initialization of a Class object. These errors will typically include “(initialization failure)” in the message of the NoClassDefFoundError. In that case, the root exception is less likely to be a ClassNotFoundException – any uncaught exception in a static{} block could cause the error.
     
  • java.lang.ClassCastException - In most cases, a ClassCastException is simply caused by two objects of types that are incompatible – for example, an attempt to cast an Integer to a String. In these cases, class loading is not at issue, and the developer must determine the source of the objects in question and determine why they are incompatible; class loading analysis is not necessarily of interest in this scenario.
    The interesting ClassCastExceptions from a class loading perspective are exceptions in which the objects in question should be compatible. For example:

    [10/30/23 21:06:52:750 SGT] 00000092 RAWrapperImpl E   J2CA0043E: An Exception occurred while trying to instantiate a ResourceAdapter JavaBean instance for the installed ResourceAdapter defined by key cells/myCell/applications/myapp.ear/deployments/myapp/deployment.xml#J2CResourceAdapter_1691760138477. The exception is: java.lang.ClassCastException:
    test.package.MyResourceAdapter incompatible with javax.resource.spi.ResourceAdapter
        at com.ibm.ejs.j2c.RAWrapperImpl.createAndConfigureRA(RAWrapperImpl.java:2264)
        at com.ibm.ejs.j2c.RAWrapperImpl.startRA(RAWrapperImpl.java:730)
        at com.ibm.ejs.j2c.RALifeCycleManagerImpl.startRA(RALifeCycleManagerImpl.java:1205)

    In this case, MyResourceAdapter implements the ResourceAdapter interface, so the objects should be compatible. For that reason, class loading analysis is useful – typically, one of the classes in the cast will have been loaded twice. If one of the classes is an interface, as this is, that’s the one that’s likely to have been loaded twice, so we will start by looking for that class in the verbose class loading output found in native_stderr.log:

    class load: javax.resource.spi.ResourceAdapter from: file:/usr/IBM/WebSphere/AppServer/plugins/javax.j2ee.connector.jar
    class load: javax.resource.spi.ResourceAdapter from: file:/mysharedlib/thirdparty/wasrt/com.ibm.ws.webservices.thinclient_6.1.0.jar

    As expected, two separate versions of the class were loaded, one from the WAS Java EE API jars in the WAS plugins directory, and one from a jar in a shared library. Analysis of the WAS class loader trace in trace.log will confirm the specific class loader used for the shared library class:

    [10/30/23 21:06:52:676 SGT] 00000092 CompoundClass >  loadClass javax.resource.spi.ResourceAdapter this=com.ibm.ws.classloader.CompoundClassLoader@e7d7dc59[PL][app:myapp] Entry
    [10/30/23 21:06:52:678 SGT] 00000092 CompoundClass 3   class javax.resource.spi.ResourceAdapter found in com.ibm.ws.classloader.SinglePathClassProvider@731cb421[/mysharedlib/thirdparty/wasrt /com.ibm.ws.webservices.thinclient_6.1.0.jar]
    [10/30/23 21:06:52:679 SGT] 00000092 CompoundClass <  loadClass javax.resource.spi.ResourceAdapter loader=com.ibm.ws.classloader.CompoundClassLoader@e7d7dc59[PL][app:myapp] Exit

    The trace confirms that the shared library was associated with the myapp application class loader (“app:myapp”), which is configured with parent-last delegation (“PL”), causing it to be loaded in preference to the libraries included in the server.
    The solution is to either remove the jar containing the ResourceAdapter class from the shared library (note: the WebSphere thin client jars are never supported for use in a Java EE application) or to switch the class loader’s delegation mode to parent-first, allowing the application class loader to delegate the load to its parent and find the server’s version of the class.
     
  • java.lang.NoSuchMethodError - Thrown if an application tries to call a specified method of a class, either static or instance, and that class no longer has a definition of that method. Normally, this error is caught by the compiler; it can only occur at run time if the definition of a class has incompatibly changed.
    This error is usually generated due to version incompatibility. Either the calling class is newer or the class being called is. Either way, a method is expected, but is not in the loaded version of the class being called.
    Sample error message:

    java.lang.NoSuchMethodError: javax/persistence/JoinColumn.foreignKey()Ljavax/persistence/ForeignKey; (loaded from file:/opt/WebSphere/85-64/plugins/javax.j2ee.persistence.jar by org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader@4ce9e057) called from class org.hibernate.cfg.AnnotationBinder (loaded from file:/opt/WebSphere/profiles/MyCell/MyProfile/installedApps/MyCell/MyApp.ear/lib/hibernate-core.jar by 
    com.ibm.ws.classloader.CompoundClassLoader@f84cf61c[app:MyApp]
       Local ClassPath: <classpath entries>

    The issue is caused by an incompatibility between the Hibernate version included in the application and the JPA API included in the WAS server, the solution is to set parent last delegation mode at module level to avoid conflict with WebSphere JPA component.
     
  • java.lang.VerifyError, java.lang.LinkageError and java.lang.UnsatisfiedLinkError - Those exceptions are more complex to debug, if you find one please open a case in IBM Support.
     

[{"Type":"MASTER","Line of Business":{"code":"LOB67","label":"IT Automation \u0026 App Modernization"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"ARM Category":[{"code":"a8m50000000CdLaAAK","label":"WebSphere Application Server traditional-All Platforms-\u003EClassloader-\u003EClassloader: TroubleShooting"}],"ARM Case Number":"","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"},{"code":"PF035","label":"z\/OS"}],"Version":"8.5.5;9.0.5"},{"Product":{"code":"SSNVBF","label":"Runtimes for Java Technology"},"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Component":"Java SDK","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB08","label":"Cognitive Systems"}}]

Document Information

Modified date:
11 July 2024

UID

swg21228700

Page Feedback