IT43806: MFT resource monitor fails to re-trigger on an item that has been successfully processed and is then manually put back

APAR status

• Closed as program error.

Error description

• An IBM MQ Managed File Transfer (MFT) resource monitor is
  created using the fteCreateMonitor command, with the following
  parameters specified:
  
  - pi 1
  - pu minutes
  -tr noSizeChange=3,C:\Input\*.csv
  
  This meant that the monitor will
  
  - Poll the C:\Input directory every minute, looking for files
  that ended with the *.csv suffix.
  - And only trigger if the size of the file remains the same for
  three consecutive polls
  
  To test the monitor, the file test.csv is put into the
  directory. Approximately three minutes later, the monitor
  triggers on the file and submits a managed transfer request to
  process it. The managed transfer request specifies a source
  disposition of "delete", so once it completes the file is
  deleted.
  
  About 10 minutes (and a number of polls) later, the test.csv
  file is put back into the directory. The expectation is that the
  monitor should retrigger on the file. Instead, however, the file
  is ignored and the monitor writes messages similar to the ones
  shown below during each poll:
  
  [18/05/2023 06:14:04:278 CDT] 00000020 TEST_MONITOR Start Poll
  New poll cycle started
  [18/05/2023 06:14:04:497 CDT] 00000020 TEST_MONITOR Pattern
  Match Trigger [C:\Input\test.csv] last modified 0 hours 8
  minutes 16 seconds ago. The monitor has not been triggered.
  [18/05/2023 06:14:04:512 CDT] 00000020 TEST_MONITOR End Poll
  Poll cycle completed in 234 milliseconds. Total items found in
  this poll [ 1 ]. Total items that matched the trigger condition
  [ 0 ]
  
  [18/05/2023 06:15:04:533 CDT] 00000020 TEST_MONITOR Start Poll
  New poll cycle started
  [18/05/2023 06:15:04:737 CDT] 00000020 TEST_MONITOR Pattern
  Match Trigger [C:\Input\test.csv] last modified 0 hours 9
  minutes 16 seconds ago. The monitor has not been triggered.
  [18/05/2023 06:15:04:752 CDT] 00000020 TEST_MONITOR End Poll
  Poll cycle completed in 219 milliseconds. Total items found in
  this poll [ 1 ]. Total items that matched the trigger condition
  [ 0 ]
  
  When this issue occurs, the only way to get the monitor to
  retrigger on the file is to clear the monitor's history.
  

Local fix

Problem summary

• ****************************************************************
  USERS AFFECTED:
  This issue affects users of IBM MQ 9.3 Managed File Transfer
  (MFT), who have resource monitors defined that:
  
  - Trigger on an item.
  - And then submit a managed transfer request to an agent to
  process the item that it triggered on, specifying a source
  disposition of "delete" (to ensure that the item is deleted if
  the managed transfer is successful).
  
  
  Platforms affected:
  MultiPlatform
  
  ****************************************************************
  PROBLEM DESCRIPTION:
  IBM MQ Managed File Transfer (MFT) resource monitors maintain a
  history of the items that they have triggered on before. Items
  are added to the history when a monitor triggers on them. If a
  monitor subsequently performs a poll and finds that an item in
  its history is no longer present, then it removes it from the
  history. This prevents the history from continually growing over
  time.
  
  The history is stored in a message on the
  SYSTEM.FTE.EVENT.<agent name> queue.
  
  Now, the history message was read at the start of each poll, and
  should have been updated at the end of every poll. However, due
  to a logic error a monitor would only store the message
  containing the updated history if it successfully triggered on
  an item. If a monitor performed a poll and didn't trigger on
  anything, the updated history message would not be written to
  the SYSTEM.FTE.EVENT.<agent name> queue.
  
  As a result, it was possible for the following sequence of
  events to occur:
  
  - A resource monitor started a poll, and triggered on an item.
  - Next, the monitor submitted a managed transfer request to the
  agent to process that item.
  - Finally, the monitor updated its history message to include
  details of the item, and then wrote it to the
  SYSTEM.FTE.EVENT.<agent name> queue,
  - The poll then completed.
  
  - The managed transfer request completed successfully, and the
  item was deleted.
  
  - The monitor performed another poll.
  - It read its history message, and found an entry for the item.
  - Next, the monitor detected that the item was no longer present
  and so removed it from the message.
  - However, the monitor did not write the updated history message
  to the SYSTEM.FTE.EVENT.<agent name> queue before it finished
  the poll.
  
  - The same behaviour happened for some more polls.
  
  - Some time later, the item was put back into the location that
  the monitor was polling.
  - The monitor found the item during the next poll.
  - It checked its history message, and found it already contained
  an entry for the item.
  - As a result, the monitor didn't trigger on it.
  

Problem conclusion

• To resolve this issue, IBM MQ Managed File Transfer (MFT)
  resource monitors have been updated to ensure that they always
  write the updated history message to the SYSTEM.FTE.EVENT.<agent
  name> queue at the end of every poll. This means that monitors
  will remove entries for items that:
  
  - They have triggered on during one poll.
  - And then don't find during subsequent polls
  
  which allows it to correctly retrigger on a item that it has
  seen before, if that item is put back into the monitored
  resource at a later time.
  
  ---------------------------------------------------------------
  The fix is targeted for delivery in the following PTFs:
  
  Version    Maintenance Level
  v9.3 LTS   9.3.0.10
  v9.x CD    9.3.4
  
  The latest available maintenance can be obtained from
  'WebSphere MQ Recommended Fixes'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
  
  If the maintenance level is not yet available information on
  its planned availability can be found in 'WebSphere MQ
  Planned Maintenance Release Dates'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
  ---------------------------------------------------------------
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  MQ BASE V9.2

• Fixed component ID

  5724H7281

Applicable component levels