IBM Support

IBM App Connect Enterprise Tracing Support

Fix Readme


Abstract

The document describes how to install and enable IBM App Connect Enterprise tracing on your IBM App Connect Enterprise hosts. The tracing data can be sent to Instana or Jaeger depending on the configurations. Current version of IBM ACE Tracing user exits is 2023.1.0.
Update Name/Fix ID: IBM-ACE-TRACING-USER-EXIT

Content

Download location

IMPORTANT NOTE: To download this update, you must first log in to IBM Fix Central by using the following link. After you log in, you can select from the individual download packages. When selecting fixes, ensure that your download options are set to "Include requisites: Yes". 
http://www.ibm.com/support/fixcentral/

Fix Download for Linux

The following table shows the components, platforms, and file names that apply to this readme file.
Product / Component Name Platform Fix
IBM App Connect Enterprise Tracing User Exit
AIX pSeries 

Linux x86_64

Linux ppc64le

Linux s390x
Windows
 IBM-ACE-TRACING-USER-EXIT

Prerequisites and co-requisites

General description

This is a technical preview for enabling IBM App Connect Enterprise tracing capability. You can download user exit implementation and configure your IBM App Connect Enterprise integration server to enable tracing. The tracing data is sent to Instana or Jaeger depending on your configuration.

Platforms and prerequisites 

This component supports Linux x86_64, Linux ppc64le, Linux s390x, AIX 7.2, AIX 7.3, and Windows. 
On AIX 7.2, ensure that XL C/C++ Runtime for AIX 16.1.0 Fix Pack 7 or later, including the file set libc++.rte is installed.  Use command "lslpp -l |grep libc++.rte" to check whether the fileset, libc++.rte, is installed.
On Windows, ensure that MSVC Runtime v2019 is installed.
This component supports IBM App Connect Enterprise v11.0.0.8 or later.
Instana and Jaeger are supported tracing systems. 
Ensure that IBM App Connect Enterprise is installed and configured before installing and configuring IBM App Connect Enterprise Tracing User Exit component. 

Known limitations

IBM App Connect Enterprise Tracing User Exit component supports HTTP requests, IBM MQ requests, and Kafka requests only.
IBM App Connect Enterprise Tracing User Exit component supports message flows with input node as the entry point only.

Known issues

None

Installing

1. Download the package from Fix Central to your IBM App Connect Enterprise host. The package includes the following files:
  • ACEOpenTracingUserExit.lel: The IBM App Connect Enterprise User Exit, intercepting the HTTP request, IBM MQ request, and Kafka request, and invoking the wrapped OpenTelemetry C++ client library to create spans.
  • tracelibrary.so:  The wrapped OpenTelemetry C++ client, provides functions to manage the lifecycle of spans and sends spans to the target tracing system.
  • acetracingexit.conf: The configuration file to specify the target tracing system (Jaeger or Instana) and the parameters for span generation.
2. Extract tar file into /var/mqsi/shared-classes directory on your IBM App Connect Enterprise host.
Note: If the ACE server is not installed with global installation, the /var/mqsi/shared-classes directory doesn't exist on your IBM ACE host. Create the directory, /opt/acetracingexit, manually and extract the tar file into /opt/acetracingexit directory on your IBM ACE host.
3. Make sure the system account that runs integration node has read and run accesses to the files in /var/mqsi/shared-classes directory. 

Configuring

This section describes how to configure your IBM App Connect Enterprise to enable tracing, including two parts - Configuring User Exit and Configuring IBM App Connect Enterprise tracing.

Configuring User Exit

Complete the following steps to enable tracing for IBM App Connect Enterprise.

1. Stop the integration node.
mqsistop <integrationNodeName>
2. Install the user exit code on an integration node by setting the UserExitPath property by using the mqsichangeflowuserexits command.
mqsichangeflowuserexits <integrationNodeName> -o -x /var/mqsi/shared-classes
3. Activate the user exit.
User exits can be active or inactive. User exits are inactive by default. You can activate user exit for an integration node, an integration server, or a specific message flow.
  • Activate the user exit for an integration node. 
mqsichangeflowuserexits <integrationNodeName> -o -a ACEOpenTracingUserExit
Note: Replace /var/mqsi/shared-classes with /opt/acetracingexit if you extract ace tracing tar file into /opt/acetracingexit directory.
Verify the user exit.
mqsireportflowuserexits <integrationNodeName> -o
Example output:
# mqsireportflowuserexits BK3 -o
BIP8854I: User Exits active for integration server 'BK3': ACEOpenTracingUserExit.
BIP8855I: User Exits inactive for integration server 'BK3': .
BIP8741I: User Exit path for integration server 'BK3': /var/mqsi/shared-classes.
BIP8071I: Successful command completion.
Start the integration node.
mqsistart <integrationNodeName>
  • Activate the user exit for an integration server:
Start the integration node.
mqsistart <integrationNodeName>
Activate the user exit for an integration server.
mqsichangeflowuserexits <integrationNodeName> -e <integrationServerName> -a ACEOpenTracingUserExit
  • Activate the user exit for a message flow.
Start the integration node.
mqsistart <integrationNodeName>
Activate the user exit for a message flow.
mqsichangeflowuserexits <integrationNodeName> -e <integrationServerName> -k <applicationName> -f <messageFlow> -a ACEOpenTracingUserExit
Repeat the steps for other integration nodes, integration servers, or message flows that you want to activate the user exit.
See more information at following links.
IBM App Connect Enterprise - Deploying a user exit

Configuring IBM App Connect Enterprise tracing

1. Go to /var/mqsi/shared-classes directory.

2. Edit acetracingexit.conf file.

image-20220718115021-3

Where:

  • LOG_LEVEL specifies the log level, which can be either info, warn, error, or debug. The log file is located at /tmp/trace directory.
  • SPAN_FORMAT specifies where the span data is sent. The SPAN format can be set to instana or jaeger. If SPAN_FORMAT is set to instana, span data is sent to Instana and you can ignore JAEGER_* parameters in acetracingexit.conf file. If SPAN_FORMAT is set to jaeger, you need to configure JAEGER_* parameters. You must use the same SPAN_FORMAT setting for all IBM App Connect Enterprise instances.
  • MONITOR_LEVEL specifies the tracing level of IBM ACE, which can be one of the following, off, normal, or verbose. If MONITOR_LEVEL is set to off, then tracing context is not appended to outgoing request. If MONITOR_LEVEL is set to normal, then tracing context will be appended only when the IBM MQ message includes the RFH2 header. If MONITOR_LEVEL is set to verbose, then the tracing context will be appended to all outgoing HTTP/MQ requests.
  • INSTANA_AGENT_HOST specifies the agent host where the Instana format span data is sent to, localhost is used by default. If you specify a remote agent host, you also need to add a line http.listen=* in <instana-agent-dir>/etc/instana/com.instana.agent.main.config.Agent.cfg for the remote host agent first as the host agent is not reachable from other hosts by default.
  • INSTANA_AGENT_PROTO specifies the connection type between IBM ACE tracing user exit and the host agent, and https are supported, http is used by default. If you want to change it to https, you need to follow Setup TLS Encryption for Agent Endpoint to secure the Instana agent endpoint first.
  • HOST_ALIAS specifies a host alias for the span data that is collected by IBM ACE Tracing user exit, so calls to IBM ACE can be linked to the infrastructure entity if the integration node or integration server is also monitored by the IBM ACE sensor. The FQDN of IBM ACE host is used by default. The host alias value must match with the IBM ACE host that is specified in host agent configuration yaml. You only need to specify a host alias if the FQDN of IBM ACE host is not used in IBM ACE sensor configuration when the host agent is not on the local IBM ACE host. The IBM ACE sensor can discover the FQDN for the local integration nodes or integration servers.
3. Save the file and restart integration node or integration server.
Repeat these installation and configuration steps on other IBM App Connect Enterprise hosts that you want to enable tracing for.
You can view IBM App Connect Enterprise tracing data on Instana UI or Jaeger UI depending on the SPAN_FORMAT settings in acetracingexit.conf file.

Unconfiguring

1. Unconfigure user exit.
  • Unconfigure user exit for an integration node.
Stop the integration node.
mqsistop <integrationNodeName>
Deactivate the user exit.
mqsichangeflowuserexits <integrationNodeName> -o -a ""
Restart the integration nodes.
  • Deactivate user exit for an integration server.
mqsichangeflowuserexits <integrationNodeName> -e <integrationServerName> -a ""
  • Deactivate user exit for a message flow.
mqsichangeflowuserexits <integrationNodeName> -e <integrationServerName> -k <applicationName> -f <messageFlow> -a ""
2. Repeat steps on other integration nodes.

Troubleshooting

The log files are at /tmp/trace. The file name is aceExit*.log. 
You can set log level in acetracingexit.conf as described in Configuring IBM App Connect Enterprise tracing section.

Additional information

The Secure Hash Algorithm 1 (SHA256) checksums of the images are as follows:
ibm-ace-tracing-exit_aix_64bit.tar.gz:
72e43c3a2d97ba3b785cb9f5a23650db371fd6d0951c97924fc98ab3a71acba6
 
ibm-ace-tracing-exit_xLinux_64bit.tar.gz:
1bfe7d62a0c4bc47c38b754ba3fc9abf667aab7c2c6fc8f069f61dbad06db778
ibm-ace-tracing-exit_pLinuxle_64bit.tar.gz
88639ae4e6e8792ed1ab644ae14ea52b1a029cf477415a1b11137819edd9d36e
ibm-ace-tracing-exit_zLinux_64bit.tar.gz:
5285446e7c278c10c297fff7cfb0b089764c2ce000461e1bc750195a9d1a54a9
ibm-ace-tracing-exit_win_64bit.zip
92ac7f014ac3fa83b83de5e226bb8b09d2c8f2c7c2a5e3eec8cf2ebf4c8d8a96

Document change history

Version Date Description of change
1.0 24 Aug 2021 Initial Version.
1.1 29 Sec 2021 Update trace library path.
2.0 18 Mar 2022 Updated IBM ACE Tracing user exits version to 2022.1.0.
3.0 11 Apr 2022 Updated IBM ACE Tracing user exits version to 2022.1.0, added support for AIX 7.2.
4.0 19 May 2022 Updated IBM ACE Tracing user exits version to 2022.2.0, added support for Linux s390x.
4.1 21 Jun 2022 Changed the sequence of msgFlowTransaction span and added MQ message RFH2 header detection.
5.0 01 Aug 2022 Updated IBM ACE Tracing user exits version to 2022.3.0, added support for Linux ppc64le and AIX 7.3.
5.1 28 Oct 2022 Fixed an issue that failed to associate with the upstream application when ACE server received a request only including W3C tracing format.
5.2 1 Dec 2022 Updated IBM ACE Tracing user exits version to 2022.4.2.
Added support for Windows.
Upgraded the openssl version to 1.1.1s.
5.3 27 Feb 2023
Updated IBM ACE Tracing user exits version to 2023.1.0.
Enriched tracing call tags information for HTTP sync request/reply node.
Enhanced HTTP async request node support.
Fixed an issue that incorrect HTTPRequestHeader cause message flow execution failed.
5.4 19 May 2023 The HTTP request header `X-INSTANA-T` and the IBM MQ message header `X_INSTANA_T`, which contain the Instana trace ID, are changed to 16 characters in length.

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSVJUL","label":"IBM Application Performance Management"},"Component":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"8.1.4","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
19 May 2023

UID

ibm16483297

Page Feedback