PH46815: MQ Z/OS: DIAGNOSTICS ARE NEEDED TO DETECT MULTIPLE LOCAL QUEUE MANAGER CACHE RECORDS ON SYSTEM.CLUSTER.REPOSITORY.QUEUE

Obtain the fix for this APAR.

APAR status

• Closed as program error.

Error description

• MQ clustering code only expects a single local queue manager
  record at any one time.  A situation occurred where there were
  two of these records, and one had an unexpected flag setting.
  It caused unexpected results in the cluster.
  
  An application attempted to put to clustered queues hosted on
  other queue managers and received:
   2085 MQRC_UNKNOWN_OBJECT_NAME for a clustered QLOCAL or
   2082 MQRC_UNKNOWN_ALIAS_BASE_Q for a QALIAS with a clustered
        target queue.
  
  Often, those symptoms are because:
    - a configuration error exists for the cluster queues or
      channels
    - an application specifies the name of the local queue manager
      in MQOD.ObjectQMgrName, but the local queue manager does not
      host the cluster queue specified in ObjectName.
      ObjectQMgrName should be left blank for the MQOPEN.
    - cluster channels fail to run and exchange information
      between a partial repository and full repository
    - the repository manager failed with CSQX448E (z/OS) or
      AMQ9448 (distributed)
  
  For the situation in this APAR, a dump of the cluster repository
  cache showed that there were no internal cluster subscriptions
  for the queue.
  
  The problem was that the cluster cache had its "subscriptions
  allowed" internal flag turned off. This flag is off when a queue
  manager is first added to a cluster or for other situations,
  such as when a REFRESH CLUSTER is done.  The flag is turned on
  when the queue manager first receives a communication from one
  of the full repositories. This flag is stored in a hardened
  state on SYSTEM.CLUSTER.REPOSITORY.QUEUE (SCRQ) in a local QMGR
  record. After a restart the queue manager reads this record to
  rebuild the contents of its cluster cache.
  
  In the reported case, the dump showed that there were two local
  QMGR records on the SYSTEM.CLUSTER.REPOSITORY.QUEUE, one with
  the "subscriptions allowed" flag on and the other with the flag
  off. This was unexpected as the code logic only expects a single
  local QMGR record at any one time. The result of this is that
  the flag's value in the cache can alternate between on and off
  between queue manager restarts. When the flag is off, clustered
  queues that already have subscriptions will be able to be used,
  but new clustered queues (or those with expired subscriptions)
  are not looked up during cluster name resolution, and MQOPEN
  fails.
  
  This APAR is being created to investigate detecting if the
  unexpected duplicate messages exist.
  ADDITIONAL SEARCH KEYWORDS :
  AMQ9456I FFST rfxCheckForDupes 00E70022 94013401
  ABEND5C6
  .
  After this diagnostic fix is applied, if duplicate Queue Manager
  messages are found, a dump is created for:
    ABN= 5C6-00E70022 LOC=CSQXRCTL.FFST(94013401)
  .
  The CHIN log also contains message CSQX053E, which indicates
  that CSQSNAP information was created. The eyecatchers for the
  CSQSNAP information include:
    XFFSrfxCheckForDupes
    First SCRQ Record MsgToken:
    Duplicate SCRQ Record MsgToken:
  

Local fix

• The Support Center can provide instructions to clean up the
  duplicate repository messages.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All users of IBM MQ for z/OS Version 9       *
  *                 Release 1 Modification 0 and                 *
  *                 Release 2 Modification 0 and                 *
  *                 Release 3 Modification 0.                    *
  ****************************************************************
  * PROBLEM DESCRIPTION: An application attempting to open a     *
  *                      clustered queue hosted on another       *
  *                      queue manager when no clustered         *
  *                      subscriptions exist for that object     *
  *                      receives either:                        *
  *                      - 2085 MQRC_UNKNOWN_OBJECT_NAME for a   *
  *                        clustered QLOCAL                      *
  *                      - 2082 MQRC_UNKNOWN_ALIAS_BASE_Q for a  *
  *                        QALIAS with a clustered target queue. *
  ****************************************************************
  An application attempting to open a clustered queue hosted on
  another queue manager in a cluster that the local Queue Manager
  is not a Full Repository for and no cluster subscriptions for
  that object already exist receives either:
  - 2085 MQRC_UNKNOWN_OBJECT_NAME for a clustered QLOCAL
  - 2082 MQRC_UNKNOWN_ALIAS_BASE_Q for a QALIAS with a clustered
    target queue.
  
  No cluster subscriptions for the target object existed or were
  created at MQOPEN time when they should have been. This was
  caused by a duplicate Queue Manager record existing on the
  SYSTEM.CLUSTER.REPOSITORY.QUEUE that was in an incorrect state.
  This prevented the Queue Manager from creating new cluster
  subscriptions for clustered objects leading to the MQRC 2085
  or MQRC 2082 return codes by applications attempting to open
  clustered objects hosted elsewhere in the cluster.
  

Problem conclusion

• The original mechanism that allowed the duplicate Queue Manager
  record to be put to the SYSTEM.CLUSTER.REPOSITORY.QUEUE is not
  currently known.
  
  In order to diagnose this problem further, diagnostics have been
  added to monitor the SYSTEM.CLUSTER.REPOSITORY.QUEUE for
  duplicate Queue Manager records during cluster maintenance and
  when the Queue Manager Record is updated by MQ and capture
  documentation to aid IBM Support to diagnose the problem if it
  were to occur again.
  
  The diagnostics are enabled by default but can be disabled,
  please contact IBM Support for instructions on how to do this.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  UI91856 UI91857 UI91858

Modules/Macros

• CMQSTRC  CMQXRMNT CSQURFXC CSQXRFXC CSQXTRTB CSQXXFS2 CSQXXFST
  

Fix information

• Fixed component name

  IBM MQ Z/OS V9

• Fixed component ID

  5655MQ900

Applicable component levels

• R100 PSY UI91858

     UP23/06/15 P F306

• R200 PSY UI91857

     UP23/06/15 P F306

• R300 PSY UI91856

     UP23/06/15 P F306

Fix is available

• Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.