PI79536: DFHKE0002 SEVERE ERROR (CODE X'0504') IN DFHKEDS

Obtain the fix for this APAR.

APAR status

• Closed as program error.

Error description

• A JVM Server is starting up and fails to start due the
  following message.
  .
  DFHKE0002 REGNNAME A severe error (code X'0504') has occurred
  in module DFHKEDS.
  .
  In the DFHJVMTRC file from Liberty we see.
  .
  2017/03/29 16:20:37.021000 CDT E [Router-Init-Thread]
  [com.ibm.cics.router.Router] Problem detected in Router main
  method com.ibm.cics.router.RouterException: Unable to create
  Liberty profile server
  .
  Caused by: java.lang.Exception: Liberty profile: Unable to
  upgrade existing configuration of CICS TS default application
  in server.xml file
  .
  Caused by: com.ibm.cics.router.RouterException: The Liberty
  profile CICS feature failed to initialize after 300000
  milliseconds. Either a problem exists with the Server, or the
  system is busy.  The JVM server will terminate. This timeout
  can be controlled by the System Property
  'com.ibm.cics.jvmserver.controller.timeout=n' where n is the
  value of milliseconds to wait. Further information may be
  available in the Liberty messages.log file.
  .
  In the CICS Trace we see.
  .
  0202 SJIN  EXIT  - FUNCTION(NOTIFY_DELETE_TCB) RESPONSE(OK)
        TASK-00044
  0009 DSIT  EXIT - FUNCTION(DELETE_TCB) RESPONSE(OK)
        TASK-00044
  .
  .
  0008 DSIT  ENTRY - FUNCTION(DELETE_TCB)
                     TCB_TOKEN(2C345400 , 00000001)
        TASK-00044
  .
  0009 DSIT  EXIT - FUNCTION(DELETE_TCB) RESPONSE(INVALID)
                    REASON(INVALID_TCB_TOKEN)
        TASK-00044
  .
  .
  Notice it's a different TASK below.
  .
  .
  0502 KEDS  ENTRY - FUNCTION(CREATE_TCB)
                     ATTACH_TOKEN(2C345600)
                     MODENAME(T8) PRTY_RELATIVE_TO_QR(FFFB)
                     EXEC_CAPABLE(EXEC_YES)
                     LE_ENVIRONMENT(LE_MVS) TCB_KEY(KEY8)
                     INHERIT_SUBSPACE(INHERIT_NO)
                     ESSENTIAL_TCB(ESSENTIAL_NO) PTHREAD(YES)
                     SZERO(SZERO_NO)
                     PARENT_TCB_TOKEN(2C345400 , 00000001)
        TASK-00042
  .
  0504 KEDS  *EXC*  - CHANGE_MODE_ERROR - FUNCTION(CREATE_TCB)
                      ATTACH_TOKEN(2C345600) MODENAME(T8)
                      PRTY_RELATIVE_TO_QR(FFFB)
                      EXEC_CAPABLE(EXEC_YES)
                      LE_ENVIRONMENT(LE_MVS) TCB_KEY(KEY8)
                      INHERIT_SUBSPACE(INHERIT_NO)
                      ESSENTIAL_TCB(ESSENTIAL_NO) PTHREAD(YES)
                      SZERO(SZERO_NO)
                      PARENT_TCB_TOKEN(2C345400 , 00000001)
                      FUNCTION(CHANGE_MODE)
                      TCB_TOKEN(2C345400 , 00000001)
        TASK-00042
  .
  As the JVM server is starting up it fails to start, this
  appears to be caused by a timeout waiting for Liberty to
  successfully startup.  A Webservice is installing at the same
  time and does not respect an exception that is returned and so
  continues processing when it really shouldn't.  This leads to a
  race condition with T8's and the TP.  The problem we believe is
  occurring because as the JVMSERVER decides to disable it sets
  itself enabled very briefly before going into disabling.  We
  believe this then allows the INSTALL_WSBIND to execute.
  Additional Symptom(s) Search Keyword(s): KIXREVDAM
  

Local fix

• n/a
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All CICS Users.                              *
  ****************************************************************
  * PROBLEM DESCRIPTION: z/OS Connect JVMSERVER fails to start   *
  *                      with DFHKE0002 severe error code '0504' *
  *                      in DFHKEDS.                             *
  ****************************************************************
  The Liberty server is configured to use embedded xml within the
  server.xml.  When the z/OS Connect configured Liberty JVMSERVER
  attempts to start CICS tries to configure the server.xml because
  autoconfigure is enabled.  The embedded xml contains a reference
  to the CICS Default app that is now a feature, CICS is unable to
  change this because of them being in the embedded files.  This
  causes the JVM server to fail to start.
  
  While the JVM server was starting another task is installing
  WEBSERVICEs required for the z/OS Connect configuration.  As
  part of the install the JVM server responsible for these
  WEBSERVICEs is called to attempt to install them.  This call
  happens before the JVMSERVER is enabled, and if the timing is
  correct can happen during the disabling of the JVMSERVER that
  has failed to enable.  During the install of the WEBSERVICE
  a CHANGE_MODE is called to switch to the TP TCB which was
  deleted during the disabling of the JVMSERVER and this causes
  the DFHKE0002 abend.
  
  Additional Keywords: msgDFHKE0002
  

Problem conclusion

• DFHPIWR has been updated so that a WEBSERVICE install will only
  trigger the JVMSERVER install parts if the JVMSERVER is in an
  enabled state.
  
  DFHSJJS has been updated to fix a problem with the JVMSERVER
  state during a failure to start.
  
  Liberty Autoconfigure has been updated to handle the upgrading
  of the CICS default app with embedded xml files.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  UI49001 UI49002 UI49003 UI49004

Modules/Macros

• DFHPIWR  DFHSJJS  DFJ@H350
  

Fix information

• Fixed component name

  CICS TS Z/OS V5

• Fixed component ID

  5655Y0400

Applicable component levels

• R000 PSY UI49001

     UP17/07/22 P F707

• R00D PSY UI49002

     UP17/07/22 P F707

• R100 PSY UI49003

     UP17/07/22 P F707

• R10D PSY UI49004

     UP17/07/22 P F707

Fix is available

• Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.