Guidance for configuring an MFT resource monitor to avoid overloading an agent
You can configure the property and parameter values of a Managed File Transfer resource monitor to reduce the load on an agent. Reducing the load on the agent improves the performance of that agent. There are several settings you can use, and you may need to use trial and error to find the best settings for your system configuration.
Overview of resource monitoring
- Finds all the files that match a trigger pattern (for example,
all the
*.txt
files in the directory). Or finds all complete groups of messages on the queue. - Determines which files are new or changed, or determines which groups are new on the queue.
- Initiates transfers for the files or groups that match the criteria in the two previous stages.
- Adds to the list of files and groups already transferred so they are not transferred again until they change.
For a queue monitor, the more groups on the queue the bigger the list of groups the agent has to compare against the list of groups already transferred.
Consider the following key settings:
- Use agent property monitorMaxResourcesInPoll to set the maximum number of files or groups the agent includes on each poll. Using this parameter limits the number of transfers in a polling interval. It also means that the agent has less parsing to do before initiating a transfer for that number of files or groups. The next time the directory monitor or queue monitor polls, the agent includes the next set of files or groups. Agent property monitorMaxResourcesInPoll is available in IBM® WebSphere® MQ File Transfer Edition 7.0.4 and later, for earlier versions of IBM WebSphere MQ File Transfer Edition it is available as an interim fix for APAR IC78011.
- When creating a directory monitor, ensure that the transfer definition you configure has a source disposition of delete. Setting this disposition means that when the file transfer completes it is removed from the monitored directory and the agent no longer keeps it on its internal list.
- When creating a directory monitor, use the -rl parameter in the fteCreateMonitor command to limit the number of levels of the directory the agent has to recurse through. Using this parameter means that lower-level directories are not scanned unnecessarily.
Further considerations when creating a resource monitor
- How quickly you need a transfer to be initiated after a file is placed in a directory, or a group on a queue.
- The rate which files are placed into a directory, or groups onto a queue.
- The maximum transfer rate of the agent. The agent must be able to handle all the transfers that a monitor generates.
The polling interval is specified when the resource monitor is created with the fteCreateMonitor command by specifying the -pi (polling interval) and -pu (polling interval units) parameters. You may need to experiment to determine the best settings for your configuration.
An option to improve
the stability of highly loaded agents that run resource monitors,
is to reduce the agent property value of maxSourceTransfers
.
With this option the agent splits its processing time between the
resource monitor and transferring files. The higher the value of agent
property maxSourceTransfers
, the more processing
time is consumed by transferring files and less is available for the
resource monitor. If you reduce the value of agent property maxSourceTransfers
,
the agent does fewer transfers in parallel, but it should have enough
processing time to poll its resource monitors. If you lower the value
of this agent property you should consider increasing the value of
agent property maxQueuedTransfers
because the number
of queued transfers may increase.
If after optimizing your monitor
you find that some transfers enter recovery, consider increasing an
agent timeout value. The heavy load placed on the agent, may mean
that the transfers timeout when negotiating the start of the transfer
with the destination agent. This timeout causes the transfer to go
into recovery and delays the completion of the transfer. The agent
property maxTransferNegotiationTime
specifies the
time the source agent waits for a response from the destination agent.
If this time is exceeded the transfer goes into recovery. The default
value of this property is 30000 milliseconds (30 seconds). Increasing
the value of the property, for example to 300000 Milliseconds (5 minutes),
may allow the transfers to continue without timing out and avoid going
into recovery.