Configuring business transaction monitoring

Configure your integration server and message flows to enable business transaction monitoring.

About this task

You configure the message flows that you want to participate in the business transaction, and create the business transaction definitions and policies that define the business events that will be monitored end-to-end throughout the business transaction. You must also configure your integration server to enable business transaction monitoring, and define the data stores that are to be used for recording and viewing the business events.

This video demonstrates in detail how to configure IBM® App Connect Enterprise to use Business Transaction Monitoring.

Procedure

Configure your integration server and message flows to enable business transaction monitoring, by completing the following steps:

Configure monitoring on the message flows that are to participate in the business transaction, either by setting properties on the message flow nodes or by using a monitoring profile.

Ensure that the correlation information is set correctly. To be part of a business transaction, a flow must have events that are correctly defined and enabled, and that contain a global transaction ID when they are emitted by the flow. The global transaction ID is a unique ID such as a reference number or an order processing number, which is used to correlate all the events for a single business transaction. The global ID can either be derived from a previous node in the flow by using the automatic setting, or it can be set explicitly by using an XPath expression to specify a location in the message. A global transaction ID links events from a message flow to one or more related message flows or external applications. For more information, see Configuring monitoring event sources by using monitoring properties and Configuring monitoring event sources by using a monitoring profile.
Enable monitoring for the integration servers on which the flows are running (and which will therefore be publishing the events), by setting properties in the Monitoring/MessageFlow section of the integration server's server.conf.yaml file. Set the publicationOn property to active, and the eventFormat property to MonitoringEventV2, as shown in the following example:
```
Monitoring:
  MessageFlow:
    publicationOn: 'active'           # choose 1 of : active|inactive, default is inactive
                                      # Ensure Events.BusinessEvents.MQ|MQTT is set
    eventFormat: 'MonitoringEventV2'  # choose 1 of : MonitoringEventV2|WMB
```

Enable business monitoring events for the integration servers on which the flows are running, by setting the enabled property to true, in the Events/BusinessEvents/MQ section of the integration server's server.conf.yaml file:


Events:
  BusinessEvents: # Business monitoring events
    MQ:
      policy: ''                # Specify a policy in the form {policy project}:policy if not using 'defaultQueueManager'
      enabled: true             # Set true or false, default is false
      format: ''                # Set string or none
      outputFormat: 'xml'       # Set to xml or json, default is xml      
      publishRetryInterval: 0   # Set the retry interval (in milliseconds), to pause all publications and retry when publication failures are causing serious delay to the transaction.

The integration servers that are publishing the events must point to the same queue manager that is recording the events, either by specifying the defaultQueueManager property or by setting the policy property in the Events/BusinessEvents/MQ section.

Install and configure a database for recording monitoring events and business transaction data, and configure the integration server to record and access the data:

Install and configure a DB2® or Oracle database for recording monitoring events and business transaction data, and configure a connection to the integration, as described in Enabling ODBC connections to the databases.
For more information, see Configuring databases.
Run the DataCaptureSchema.sql script to create the monitoring events tables that are required for business transaction monitoring.
If these tables have already been created for use by the record and replay function, you can use the same tables for business transaction monitoring events. However, if the tables do not exist, or if you want to use a different schema for business transaction monitoring from the one that you use for record and replay, you must run the DataCaptureSchema.sql script as described in the relevant topic:
- Creating and configuring a DB2 database for recording data
- Creating and configuring an Oracle database for recording data
In addition to the DataCaptureSchema.sql script that you run as part of this configuration procedure, a file called DataCaptureSchema_v2.sql is stored in the same location in the file system. However, the DataCaptureSchema_v2.sql script can be used only if you are using V2 monitoring events exclusively, and cannot be used in any architecture that combines V1 and V2 monitoring events.
Run the BusinessCaptureSchema.sql script to create the database table that is required for recording and storing business transaction data, as shown in the following examples:
- DB2:
```
db2 -tvf BusinessCaptureSchema.sql
```
- Oracle:
```
sqlplus user/password @BusinessCaptureSchema.sql
```
The script is stored in the following location, depending on your database and operating system:
DB2
Windows: install_dir\server\ddl\db2\BusinessCaptureSchema.sql

Linux® or UNIX: install_dir/server/ddl/db2/BusinessCaptureSchema.sql
Oracle
Windows: install_dir\server\ddl\oracle\BusinessCaptureSchema.sql

Linux or UNIX: install_dir/server/ddl/oracle/BusinessCaptureSchema.sql
where install_dir is the location of your IBM App Connect Enterprise installation.
The database tables are now ready to be used for recording and storing your business transaction data.

Configure the Record and Replay store for the integration server, by setting properties in the Stores section of the integration server's server.conf.yaml file.

The name of this data store must be unique. By default, business transaction definitions created in the web user interface are configured to reference a Record and Replay store called BTMDataStore, as shown in the following example:


Stores:
    BTMDataStore:
       dataSource: ''                    # The ODBC data source name (DSN) that is used to connect to the database that stores the recorded data. This property is mandatory and has no default value.
       schema: ''                        # The schema name that owns the database tables that are used for storing recorded data. This property has no default value. If no value is set, either the default database schema is used (if there is one), or no schema is used, depending on the database.
       storeMode: 'all'                  # The mode for the store to operate in. Valid values are record, view, and all. Default is all.
       queue: 'SYSTEM.BROKER.DC.RECORD'  # The name of the queue to which event messages will be published before being recorded to the database. The queue must exist.
                                          # Default is SYSTEM.BROKER.DC.RECORD. The queue SYSTEM.BROKER.DC.RECORD must be created manually if you use Record and Replay. The same queue can be specified for multiple Record and Replay stores.
                                          # Change the value of this property to distribute the data from multiple sources across multiple queues.
       backoutQueue: 'SYSTEM.BROKER.DC.BACKOUT'  # The name of the backout queue used by the recorder. Messages that cannot be processed (for example, because the specified database does not exist) are sent to this queue.
                                                  # Default is SYSTEM.BROKER.DC.BACKOUT. The queue SYSTEM.BROKER.DC.BACKOUT must be created manually if you use Record and Replay. If a data capture source refers to this data capture store, and no backoutQueue has been specified, an error occurs. The same backoutQueue can be specified for multiple Record and Replay stores.

Specify the ODBC data source name (DSN) that is used to connect to the database where the recorded data is stored, by setting the dataSource property.
Specify the schema name for the database tables that are used for storing recorded data, by setting the schema property.
If you do not set this property, the default database schema is used.
Specify the names of the queue and backout queue to be used by the recorder, by setting the queue and backoutQueue properties. By default, these properties are set to SYSTEM.BROKER.DC.RECORD and SYSTEM.BROKER.DC.BACKOUT.

If you specify non-default queues in the queue and backoutQueue properties, you must also specify them in the Record and Replay store in which the ODBC data source name is specified (using the dataSource property), and the Record and Replay store must be the one that is referenced from the business transaction definition (by default, BTMDataStore).

Enable Record and Replay for the integration server, by editing the server.conf.yaml file and setting the recordReplayEnabled property to true.


# Record and Replay requires a default queue manager to be associated with the integration server.
   RecordReplay:
     recordReplayEnabled: true           # Set to true to enable all Record and Replay functionality. Default is true.

Configure the integration server to use a default queue manager, and create the default system queues on the queue manager:

Configure the integration server to use a local or remote default queue manager, by specifying either the defaultQueueManager or remoteDefaultQueueManager property in the integration server's server.conf.yaml file. For example:


# The remoteDefaultQueueManager and defaultQueueManager properties are mutually exclusive. Uncomment only one of these two options.
      defaultQueueManager: 'defaultQM'            # Set non-empty string to specify a default queue manager


# The remoteDefaultQueueManager and defaultQueueManager properties are mutually exclusive. Uncomment only one of these two options.
      remoteDefaultQueueManager: '{myPolicyProject}:myMQEndpointPolicy'      # Specify an MQEndpoint policy in the format {policy project}:policy

Ensure that the SYSTEM.BROKER.DC.RECORD and SYSTEM.BROKER.DC.BACKOUT system queues have been created on the queue manager. If they do not exist, you can create them by running the iib_createqueues script, as described in Creating the default system queues on an IBM MQ queue manager.

Configure the required security permissions for users of the web user interface:

If administration security has been enabled on the integration server, ensure that users have the required permissions, by setting properties in the Security section of the integration server's server.conf.yaml file:

For viewing transactions, users require read access on the DataPermissions object and the Permissions object.
For creating and editing business transaction definitions and business transaction policies, users require write access on the Permissions object.
For starting and stopping the recording of business transactions, users require execute access on the Permissions object.


Security:
  Permissions:
    # Set Admin Security Authorization file permissions by web user role using 'read+:write+:execute+' , or 'all+'
    # '+' grants permission, '-' denies permission
    # e.g. define the following web user roles 'viewRole' and 'adminRole'
    #viewRole:  'read+:write-:execute-'
    #adminRole: 'all+'
  DataPermissions:
    # Set Admin Security Authorization file permissions for Record and Replay web user role using 'read+:write+:execute+' , or 'all+'
    # '+' grants permission, '-' denies permission.  Record and Replay roles also require 'read+' permission to be defined
    # in the Permissions section above.
    # e.g. define the following web user roles 'dataViewer', 'dataReplayer' and 'adminRole'
    #dataViewer:  'read+:write-:execute-'
    #dataReplayer: 'read+:write-:execute+'
    #adminRole: 'all+'

Ensure that the web user interface is configured. For more information, see Configuring the IBM App Connect Enterprise web user interface.