PI17537: DEADLOCK DETECTED BETWEEN SERVICE INTEGRATION BUS THREADS

8.5.5.4: WebSphere Application Server V8.5.5 Fix Pack 4
8.0.0.10: WebSphere Application Server V8.0 Fix Pack 10
8.5.5.5: WebSphere Application Server V8.5.5 Fix Pack 5
8.5.5.6: WebSphere Application Server V8.5.5 Fix Pack 6
8.0.0.11: WebSphere Application Server V8.0 Fix Pack 11
8.5.5.7: WebSphere Application Server V8.5.5 Fix Pack 7
8.5.5.8: WebSphere Application Server V8.5.5 Fix Pack 8
8.0.0.12: WebSphere Application Server V8.0 Fix Pack 12
8.5.5.9: WebSphere Application Server V8.5.5 Fix Pack 9
8.5.5.10: WebSphere Application Server V8.5.5 Fix Pack 10
8.5.5.11: WebSphere Application Server V8.5.5 Fix Pack 11
8.0.0.13: WebSphere Application Server V8.0 Fix Pack 13
8.5.5.12: WebSphere Application Server V8.5.5 Fix Pack 12
8.0.0.14: WebSphere Application Server V8.0 Fix Pack 14
8.5.5.13: WebSphere Application Server V8.5.5 Fix Pack 13
8.0.0.15: WebSphere Application Server V8.0 Fix Pack 15
8.5.5.14: WebSphere Application Server V8.5.5 Fix Pack 14
8.5.5.15: WebSphere Application Server V8.5.5 Fix Pack 15
8.5.5.14: WebSphere Application Server V8.5.5 Fix Pack 14
8.5.5.17: WebSphere Application Server V8.5.5 Fix Pack 17
8.5.5.20: WebSphere Application Server V8.5.5.20
8.5.5.18: WebSphere Application Server V8.5.5 Fix Pack 18
8.5.5.19: WebSphere Application Server V8.5.5 Fix Pack 19
8.5.5.16: WebSphere Application Server V8.5.5 Fix Pack 16
8.5.5.21: WebSphere Application Server V8.5.5.21

APAR status

• Closed as program error.

Error description

• When trace is enabled on Websphere Application Server that
  utilizes the Service Integration Bus objectmanager, deadlock
  occurs between the following threads:
  
  CheckpointHelper:
  
  at com/ibm/wbe/objectManager/TreeMap.size(TreeMap.java:242)
  at
  com/ibm/wbe/objectManager/AbstractMap.toString(AbstractMap.java:
  394)
  at
  com/ibm/wbe/objectManager/utils/TraceImpl.getObjectArrayMsg(Trac
  eImpl.java:44)
  at
  com/ibm/wbe/objectManager/utils/TraceImpl.entry(TraceImpl.java:1
  31)
  at
  com/ibm/wbe/objectManager/SingleFileObjectStore.write(SingleFile
  ObjectStore.java:604)
  at
  com/ibm/wbe/objectManager/AbstractSingleFileObjectStore.updateDi
  rectory(AbstractSingleFileObjectStore.java:1252)
  at
  com/ibm/wbe/objectManager/AbstractSingleFileObjectStore.flush(Ab
  stractSingleFileObjectStore.java:1145)
  at
  com/ibm/wbe/objectManager/ObjectManagerState.performCheckpoint(O
  bjectManagerState.java:928)
  ..
  ..
  
  
  RulesEngineThread-7:
  
  at
  com/ibm/wbe/objectManager/AbstractSingleFileObjectStore.reserve(
  AbstractSingleFileObjectStore.java:769)
  at com/ibm/wbe/objectManager/TreeMap.remove(TreeMap.java:1018)
  at
  com/ibm/wbe/server/engine/store/om/ConcurrentHashMap.remove(Conc
  urrentHashMap.java:229)
  ..
  ..
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:  Users of the default messaging provider     *
  *                  for IBM WebSphere Application Server V8.0   *
  *                  and v8.5.                                   *
  ****************************************************************
  * PROBLEM DESCRIPTION: Deadlock between ObjectManager and      *
  *                      WebSphere Business Events (WBE)         *
  *                      RuleEngine threads when tracing         *
  *                      enabled.                                *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  As per the "CheckpointHelper" thread in the stacktrace, the
  TraceImpl.entry() method prints the method entry trace for
  SingleFileObjectStore.write() method. It performs a
  .toString() operation which locks "TreeMap" to get the list of
  entries locked by the transaction, adjust the size of the
  tree, and then print the size.
  However, there is no need to lock the "TreeMap" as the
  .toString() method invokes the TreeMap.size() with a
  transaction id as an argument which is passed as "null".
  

Problem conclusion

• Source code has been modified not to lock the "TreeMap" object
  when the passed transaction is "null".
  
  The fix for this APAR is currently targeted for inclusion in
  fix packs 8.0.0.10 and 8.5.5.4. Please refer to the
  Recommended Updates page for delivery information:
  http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
  

Temporary fix

• Remove "ObjectManager*=all" tracing
  (or)
  Disable class level tracing by adding,
  com.ibm.wbe.objectManager.SingleFileObjectStore=all=disabled
  

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WAS SIB & SIBWS

• Fixed component ID

  620800101

Applicable component levels

• R800 PSY

     UP

• R850 PSY

     UP