IT45206: Intermittent timing issue where an MFT resource monitor may sendan unexpected 0 byte file.

APAR status

• Closed as program error.

Error description

• An IBM MQ Managed File Transfer (MFT) resource monitor has been
  configured to poll a directory, looking for files that have the
  extension *.txt.
  
  When performing a poll, the monitor finds a file called
  File1.txt and submits a managed transfer request to the agent to
  process it. Shortly afterwards, the monitor performs another
  poll and finds a file called File1.txt again. However, this file
  has a size of 0 bytes. Although the monitor has seen a file
  called File1.txt before, the size of the file has changed. This
  causes the monitor to submit a second managed transfer request
  to the agent for it. As a result, a file called File1.txt is
  transferred twice in quick succession:
  
  - The first time, the file is transferred with the expected
  contents.
  - The second time, a zero byte file is transferred.
  

Local fix

• Adjust the MFT Resource Monitor Poll interval to make it larger
  than the expected time that it would take for a Managed File
  Transfer to complete within the Customer Environment.
  
  Note, as the time that it would take for a Managed File
  Transfer to complete is likely to be different for each
  Customers Environment for setting the polling interval we would
  recommend that Customers run some tests with different polling
  intervals, to find the optimal value for their setup. The
  setting would depend on how long the transfers are taking to
  complete. Ideally, the polling interval should be set slightly
  higher than that.
  

Problem summary

• ****************************************************************
  USERS AFFECTED:
  This issue affects users of IBM MQ Managed File Transfer (MFT),
  who have resource monitors that:
  
  - Are configured to poll directories looking for files that
  match a trigger condition.
  - And submit managed transfer requests containing a source
  disposition of "delete" when they find a match.
  
  
  Platforms affected:
  MultiPlatform
  
  ****************************************************************
  PROBLEM DESCRIPTION:
  IBM MQ Managed File Transfer (MFT) resource monitors run as part
  of an agent, and are used to scan a resource (either a directory
  or a queue) looking for items that match a trigger condition. If
  an item is found, then the monitor will submit a managed
  transfer request to the agent. Typically, this is used to
  transfer the file that the monitor triggered on.
  
  If a resource monitor is configured to scan a directory, then
  whenever it finds a file that matches the trigger condition it
  checks to see if that file is locked by another process. In
  order to do this, the monitor performs the following steps:
  
  - Check that the file still exists.
  - If it does, then:
      - Create a Java FileOutputStream for the file.
      - Get a Java channel from the FileOutputStream.
      - Call the method Channel.tryLock() to lock the file.
      - If the call to Channel.tryLock() works then:
         - Unlock the file.
      - End if
  - End if
  
  If the file is not in use, the monitor adds it the list of files
  seen during the current poll.
  
  Once the monitor has a list of files that match the trigger
  condition and aren't in use by another process, it checks each
  file in the list in turn to see if it has seen it before. If it
  has, and the file size hasn't changed, then the monitor removes
  the file from the list (because it has triggered on it before).
  
  The monitor now has a list of files that match the trigger
  condition and have not been seen (or triggered on) before. It
  then submits one or more managed transfer requests to the agent
  to process them.
  
  
  Now, if the managed transfer requests submitted by the monitor
  had a source disposition of "delete" (indicating that the source
  files should be deleted if the managed transfer was successful),
  then it was possible for the following sequence of events to
  occur:
  
  - A resource monitor thread performed a poll, and found a file
  that matched its trigger condition.
  - The monitor then submitted a managed transfer request to the
  agent to process it.
  - The agent received the managed transfer request and started to
  process it.
  
  - Sometime later, the resource monitor thread performed another
  poll, found the file again and checked to see if it was locked
  by another process.
  - The thread checked that the file still existed, and then moved
  onto the next part of its check.
  
  - In the meantime, the original managed transfer request
  completed successfully and an internal FileIOWorker thread
  within the agent deleted the file.
  
  - The resource monitor thread then created a Java
  FileOutputStream for the file, and got the Java channel from it.
  - Because the file no longer existed, this resulted in the file
  being recreated with a size of zero bytes.
  - Next, the thread called Channel.tryLock() to see if another
  process had a lock on the file. This call worked, and so the
  monitor added the files that matched the trigger condition.
  - The monitor then checked the size of the file, and found that
  it had changed since the last time it saw it. Because of this,
  the monitor assumed the file was a new file and submitted
  another managed transfer for it
  
  As a result, a file with the same name was transferred twice in
  quick succession. These were:
  
  - The original file (which contains data).
  - Followed shortly afterwards by the zero byte file.
  

Problem conclusion

• To resolve this issue, MQ Managed File Transfer has been updated
  to include some internal locking which ensures that only one
  thread can either check if a file is in use, or delete a file,
  at any one time. This means that the resource monitor thread
  will no longer create a zero byte file when checking if a file
  is locked by another process.
  
  ---------------------------------------------------------------
  The fix is targeted for delivery in the following PTFs:
  
  Version    Maintenance Level
  v9.3 LTS   9.3.0.20
  v9.4 LTS   9.4.0.0
  
  The latest available maintenance can be obtained from
  'IBM MQ Recommended Fixes'
  https://www.ibm.com/support/pages/recommended-fixes-ibm-mq
  
  If the maintenance level is not yet available information on
  its planned availability can be found in 'IBM MQ
  Planned Maintenance Release Dates'
  https://ibm.biz/mqplannedmaintenance
  
  ---------------------------------------------------------------
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  MQ BASE V9.3

• Fixed component ID

  5724H7291

Applicable component levels