WebSphere Application Server Hang in Federated Repository LDAP when using the default settings

Resolving The Problem

While a restart of the application server can repair the issues temporally (that is until the next idle timeout), a long-term solution would be to configure the context pool times out.  These steps can apply to both WebSphere Application Server and WebSphere Liberty.

SET "CONTEXT POOL TIMES OUT"

WebSphere Application Server traditional

• From the WebSphere Application Server admin console, click Security > Global Security > click Configure... (with Federated Repository selected) > Manage Repositories > Repository Name > Performance
• Set the "Context Pool Times Out" setting
  • The timeout setting "Context pool times out" is located inside the Context Pool section. The value needs to be set lower than the idle timeout on a firewall (or backend server) to be effective.

    Do not set the timeout too small, as that would defeat the purpose of even using a context pool to reuse already available connections.

• Press OK or Apply

• Save the changes, and restart the appserver instances

WebSphere Liberty

• Edit the server.xml
  • Add a subelement named <contextPool> to an already defined <ldapRegistry> and configure the timeout attribute.  The value must be smaller than the firewall (or backend instance's) idle timeout to be effective.  Do not set the timeout too small, as that would defeat the purpose of using a context pool. 
    • If I use the default values as an example aside from the timeout (the example sets the timeout to 4 minutes instead of 0), it can resemble the mock example:
      <ldapRegistry ... ... ...>
         <contextPool enabled="true" intialSize="1" maxSize="0" preferredSize="3" timeout="4m" waitTime="3s"/>
      </ldapRegistry>
  • Instead, setting the timeout attribute without the others inside the contextPool is acceptable. 

DISABLE CONTEXT POOLING (recommended for testing only)

Disabling the context pool is not a recommended solution, as it disables the pooling completely. Meaning every new connection to the LDAP generates a brand new socket, which can impact performance over the long term. 
However, the procedure can be attempted in a test environment to help determine whether context pooling is a factor in the problems. 

WebSphere Application Server traditional

• From the WebSphere Application Server admin console, click Security > Global Security > click Configure... (with Federated Repository selected) > Manage Repositories > Repository Name > Performance
• Disable Context Pooling by removing the check for Enable Context Pool.
• Press OK or Apply
• Save the changes, and restart the appserver instances

WebSphere Liberty

• Edit the server.xml
  • Adjust the contextPool element to disable the context pooling by adding the attribute enabled="false". 
  • The ldapRegisry element example is provided here, where enabled="false" is added to the element <contextPool>.
      There are other attributes for the contextPool, and might be present, but they are not shown and are not required.
    <ldapRegistry ... ... ...>
       <contextPool enabled="false"/>
    </ldapRegistry>

Frequently Asked Questions (FAQ)

* Can disabling the context pool result in performance issues?

The idea of using a context pool for the LDAP connections is to allow the appserver to reuse the connections it already created, thus conserving memory resources as less individual connections are opened.  Disabling it might cause a lot more connections to be opened, temporally.  Each login attempt, for example, would then use a connection a single time, and then discard it. 

Testing user load with the configuration changes would be one way to determine whether the new configuration has any performance effect on the environment.

* Why do I need to configure a "Context Pool Times Out" timeout setting to resolve the issue?  Doesn't the timeout setting indicate how long the login can hang for?

Because the connections are being reused, by setting the timeout allows WebSphere Application Server (or WebSphere Liberty) to automatically discard the connection after a set amount of time.  The idea here is that you want to discard the connection before the firewall's idle timeout triggers, thus preventing the hangs and freezing before they can occur

* But I thought the hangs were caused by WebSphere Application Server being unable to connect to the LDAP server?

The inquiry is partially correct.  The connections to the LDAP are closed only by the firewall, but WebSphere Application Server is unaware of the closure.  The appserver then waits until all retry attempts before it closes the connection.  One might discover that the issues are resolved 10 - 15 minutes later (after the initial hangs are observed), as if it was fixed automatically.  The closed connection on the appserver is technically generating a brand new connection to the LDAP.

* I'm ready to set the timeout value for the context pool, what do I set it to?

You need to set it lower than the firewall's idle timeout.  The lower you set it determines the time in which the connections are discarded when not used.  Generally set it lower to the firewall's timeout (work with your network administrator to find out the appropriate values)

Documentation