PH55687: When running Liberty JVM servers in CICS excessive CPU usage is possible on systems with high processor counts

Obtain the fix for this APAR.

APAR status

• Closed as program error.

Error description

• After an upgrade to CICS 5.6 high CPU usage (both CP & zIIP
  engines) was observed for CICS regions with Liberty JVM servers.
  
  This was using most of the CPU available on the LPAR, impacting
  the processing of other jobs running in the LPAR.
  
  The high uncaptured CPU time was due to high STIMERM rates
  causing contention for the DISP spin lock.
  
  STIMERM calls are made as part of the CICS handling of Runaway
  timing.
  
  Because of the way the Liberty threadpooling works on systems
  with a lot of processors high Liberty executor default
  coreThreads values for the CICS/Liberty threadpool lead to a
  high number of CICS T8 TCBs that are subject to STIMERM calls.
  

Local fix

• Replace
  <executor id="allowCICSconfigure" maxThreads="256"/>
  
  with
  
  <executor coreThreads="10" maxThreads="256"/>
  
  to set maxThreads and coreThreads in server.xml.  This will
  allow the Liberty threadpool size to float between 10 and 256
  threads as per the load.
  Note: you must remove the
  id="allowCICSconfigure"
  in order for this Local Fix to work.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All CICS users.                              *
  ****************************************************************
  * PROBLEM DESCRIPTION: CICS regions with Liberty JVM servers   *
  *                      can trigger excessive CPU usage on      *
  *                      systems with high processor counts.     *
  ****************************************************************
  Liberty calculates a coreThreads value based on the formula
  2*number of processors. Since CICS does not provide a
  coreThread value in the CICS managed default executor, this Open
  Liberty calculated value is used, up to a maximum of 256.
  On systems with high numbers of processors this means that when
  a Liberty server is idle, the number of threads will not drop
  below the coreThreads value.
  In a CICS setting, this means unusually high numbers of CICS T8
  TCBs are subject to Runaway related STIMERM processing on a
  250ms cadence, which can result in high CPU usage due to MVS DSP
  Lock contention when serializing MVS TQE chains.
  

Problem conclusion

• CICS was modified so that it now sets a coreThreads value of
  either 6 or 2*number of processors, whichever is lower.
  This is written into the CICS managed executor in server.xml
  when a Liberty JVM server initializes.
  
  Keywords: Upgrade2CICS56
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

  PH51513

• APAR is sysrouted TO one or more of the following:

  UI93909

Modules/Macros

• DFJ@H571
  

Fix information

• Fixed component name

  CICS TS Z/OS V6

• Fixed component ID

  5655YA100

Applicable component levels

• R400 PSY UI93909

     UP23/10/10 P F310 ¢

Fix is available

• Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.