Collector node
Use the Collector node to create message collections based on rules that you configure in the node.
- Developer
- Application Integration Suite
- Standard
- Advanced
Information about the state of in-flight messages is held on storage queues that are controlled by WebSphere® MQ, so you must install WebSphere MQ on the same computer as your integration node if you want to use the capabilities provided by the Collector node. The storage queues that hold the state information are owned by the queue manager that is associated with the integration node, and you specify this queue manager by using the -q property of the mqsicreatebroker command. For more information about the queues that are required by the Collector node, see Configuring the storage of events for Collector nodes.
This topic contains the following sections:
Purpose
- Number of messages
- Collect messages for a set period
- Match the contents of a correlation path
- Match the contents against a correlation pattern
A message collection is created when the first message arrives at any of the dynamic input terminals on the Collector node. Message collections are stored on a WebSphere MQ queue.
When the conditions set by the event handlers for a message collection have been met, the message collection is complete and ready to be propagated. For example, if you set the event handlers on the Collector node to wait for two messages from each input terminal, the message collection is complete when two messages have been received by every terminal. When the next message arrives on an input terminal, it is added to a new message collection. You can select from a number of options to determine how the propagation of the message collection is coordinated. You can specify that the message collection is propagated automatically for processing, or alternatively that the message collection is propagated when a control message is received.
You can also set an expiry timeout for message collections that fail to be completed in a satisfactory time by using a property on the Collector node. The timeout starts when the first message is added to a message collection. If the timeout expires before the message collection is complete, the incomplete message collection is propagated to the Expire terminal. Set a value for the collection expiry to ensure that incomplete message collections do not remain stored on a queue indefinitely. Add appropriate processing to your message flow to handle incomplete message collections.
The Collector node is contained in the Routing drawer of the message flow node palette, and is represented in the IBM® Integration Toolkit by the following icon:
Using this node in a message flow
- Compute
- JavaCompute
- .NETCompute
The Collector node has one static input terminal, Control, and four static output terminals: Out, Expire, Failure and Catch. These static terminals are always present on the node. In addition to the static input and output terminals, you can add dynamic input terminals for each input source that you want to use with the Collector node.
You can add and configure as many input terminals as required to the Collector node. You can configure the properties of each input terminal separately to control how the messages received on each input terminal are added to the appropriate message collection.
You can use the Control terminal to trigger the output of completed message collections from the Collector node. Configure the Event coordination property to set the behavior of the Collector node when messages are received on the Control terminal.
When a message collection is successfully completed, it is ready to be propagated to the Out terminal. If a value greater than zero is set on the Collection expiry property, any incomplete message collections are propagated to the Expire terminal.
A new transaction is created when a message collection is complete, and is propagated to the next node. Any exceptions that are caught from downstream nodes cause the message collection to be propagated to the Catch terminal on the Collector node, together with the exception list. If the Catch terminal is not connected to any other nodes, the transaction is caused to roll back. Messages in the message collection are backed out onto the queue of the Collector node. The exception list is written to the system log. This step is repeated until the message collection is successfully processed. To avoid an exception that causes the message collection to fail to be propagated successfully, ensure that you connect the Catch terminal to a flow to handle any exceptions. Also, ensure that you set an expiry timeout to propagate incomplete message collections.
If you use additional instances of a message flow or multiple inputs to the Collector node, you can use the Correlation path and Correlation pattern properties to ensure that related messages are added to the same message collection. If you use additional instances, or multiple inputs to the Collector node the order of messages in the message collection can be unpredictable. The order of messages is also unpredictable if you use WebSphere MQ cluster queues as inputs to the Collector node.
Configuring the Collector node
When you have put an instance of the Collector node into a message flow, you can configure it; see Configuring the Collector node. The properties of the node are displayed in the Properties view. All mandatory properties for which you must enter a value (those properties that do not have a default value defined) are marked with an asterisk.
Terminals and properties
The Collector node terminals are described in the following table.
Terminal | Description |
---|---|
Control | The static input terminal that accepts control messages. Any message received by the Control terminal is treated as a control message. |
Out | The output terminal to which the complete message collection is routed if the received messages satisfy the configured conditions for the message collection. |
Expire | The output terminal to which the incomplete message collection is routed if the received messages do not satisfy the configured conditions within the time specified on the Collection expiry property. If you have not set a value for the Collection expiry property this terminal is not used. |
Failure | The output terminal to which the message collection is routed if a failure is detected during processing. |
Catch | The output terminal to which the message collection is routed if an exception is thrown downstream and caught by this node. |
The Collector node can have further dynamic input terminals. You can create numeric terminal labels for the Collector node; however, the Compute node does not support numeric labels. Therefore, when you are defining a custom terminal for the Collector node, ensure that the name begins with an alphabetic character.
The following tables describe the node properties. The column headed M indicates whether the property is mandatory (marked with an asterisk if you must enter a value when no default is defined); the column headed C indicates whether the property is configurable (you can change the value when you add the message flow to the BAR file to deploy it).
The Collector node Description properties are described in the following table:
Property | M | C | Default | Description |
---|---|---|---|---|
Node name | No | No | The node type, Collector | The name of the node. |
Short Description | No | No | A brief description of the node. | |
Long Description | No | No | Text that describes the purpose of the node in the message flow. |
The Collector node has two types of Basic properties. You can set properties for each dynamic input terminal that you add to the Collector node in the Collection Definition table in the Basic properties. The properties in the Collection Definition table define the event handlers for the messages arriving on the individual input terminals. The properties that you can set for each of the dynamic input terminals are described in the following table:
Property | M | C | Default | Description |
---|---|---|---|---|
Terminal | Yes | No | The terminal name | Terminal is
not a property of the node, but a label to show the name of the dynamic
input terminal. Enter values for the event handler properties for each dynamic input terminal that you have added to the Collector node in the Collection Definition table. |
Quantity | Yes | No | 1 | This property specifies the number of messages
that the input terminal accepts to add to a message collection. The default value is 1; if you accept this default value, only one message is added to a collection. If a second message is received on the terminal, a new collection instance is created for it. If you select 0 (zero) or do not specify a value, there is no limit to the number of messages accepted. In this case, the value that is set on the Timeout property must be greater than zero. If you set both the Timeout and Quantity properties to values that are greater than zero, the terminal stops accepting messages when the first of the two thresholds is reached. When this threshold is reached, and other criteria have also been satisfied, the collection is complete and it is propagated to the Out terminal. |
Timeout | Yes | No | 0 | This property specifies the maximum time in
seconds for which the input terminal accepts messages. If you select 0 (zero), the timeout is disabled and there is no limit on the time to wait for messages. In this case, the value that is set on the Quantity property must be greater than zero. If you set both the Timeout and Quantity properties to values that are greater than zero, the terminal stops accepting messages when the first of the two thresholds is reached. When this threshold is reached, and other criteria have also been satisfied, the collection is complete and it is propagated to the Out terminal. |
Correlation path | No | No | Messages are only accepted into a message collection
if they have the same correlation string. If the message has a different
correlation string, it is offered to the next collection in the queue.
If none of the collections accept the message, a new collection is
created with correlation string set to the value of the correlation
string in the message. Messages are grouped by the value from the
correlation path. The correlation path is defined by using XPath.
You can define your own correlation path by using XPath, or select
from the following predefined paths:
If you define a value for Correlation path, you can optionally configure a Correlation pattern. |
|
Correlation pattern | No | No | This property specifies a pattern to match the
contents of a correlation path value against. You must set the Correlation path property before
you set the value for the Correlation
pattern property. If you set the correlation pattern, you
must use one * character,
optionally surrounded by other text. For example, *.dat. If the correlation pattern is blank, the entire text from the correlation path must be matched by the incoming message. |
The remaining Basic properties for the Collector node are shown in the following table:
Property | M | C | Default | Description | mqsiapplybaroverride command property |
---|---|---|---|---|---|
Collection name | No | No | This property specifies the name of the message
collection.
|
||
Collection expiry | No | Yes | The amount of time, in seconds, that the Collector node waits for messages
to arrive. After this time an incomplete message collection is expired
and propagated to the Expire output terminal. If you set this property to zero, the collection expiry is disabled and the Collector node waits for messages indefinitely. Set a value greater than zero to ensure that the message collection is processed, even if not all messages are received. A warning is issued if this property is not set. This property overrides the Timeout properties that are set on the input terminals. The Timeout property specifies the maximum time in seconds for which the input terminal accepts messages. After this time, the collection is complete and is propagated to the Out terminal. If one input terminal does not have a timeout set, or if its timer has not started, a message collection might wait for an input message indefinitely. In this situation, the Collection expiry property is used to ensure that the incomplete message collection is not left in an unresolved state, but is propagated to the Expiry terminal. This property is overridden by the Collection expiry property, if set, in the Collector configurable service. |
collectionExpiry |
The Collector node Advanced properties are described in the following table:
Property | M | C | Default | Description |
---|---|---|---|---|
Persistence mode | No | No | The node type, Collector | This property specifies whether messages are stored on the queues of the Collector node persistently. |
Event coordination | Yes | No | Disabled | This property specifies how messages received
on the Control terminal for event coordination processing are handled
in the Collector node.
|
Configurable service | No | Yes | None set | This property specifies the name of the Collector
configurable service to be used by the Collector node. The properties set by the Collector configurable service override the equivalent properties set on the Collector node. For more information about the properties than you can set with this configurable service, see Configurable services properties. |
Property | M | C | Default | Description |
---|---|---|---|---|
Events | No | No | None | Events that you have defined for the node are
displayed on this tab. By default, no monitoring events are defined
on any node in a message flow. Use Add, Edit,
and Delete to create, change or delete monitoring
events for the node; see Configuring monitoring event sources by using monitoring properties for details. You can enable and disable events that are shown here by selecting or clearing the Enabled check box. |