Troubleshooting
Problem
You are experiencing High CPU and Performance problems in IBM Integration Bus (IIB) or WebSphere Message Broker (WMB). You would like to know what documentation you must collect (MustGather) so that the IBM Integration Bus / WebSphere Message Broker Support team can diagnose your problem. If you gather this documentation before contacting support it will expedite the troubleshooting process, and save you time.
Resolving The Problem
- Message flow accounting and statistics can be started and viewed in IBM Integration Explorer to monitor the message flows processing messages when such a problem is experienced. Also check the following example in the Knowledge Center.
By reviewing the data, you can isolate the problem down to a specific message flow node where the problem may be occurring. For example: average elapsed time (in milliseconds), average CPU time and the number of times a node is invoked can help you identify a particular node that may be a bottleneck.
- Check for known issues:
IIB diagnostic tools guide
How to use the flow analysis to find further performance problems
IIB and WMB FAQ for memory
Dynamic guide to help troubleshooting Message Broker
Troubleshooting performance problems
Message splitter pattern
IBM Integration Bus support site
- mqsidc
See: mqsidc usage instructions
- This command line tool is only available for WMB 7.0.0.5+, WMB 8.0.0.1+, IIB 9.0.0.0+, and IIB 10.0.0.0+.
- When running the tool, select the option for 'Broker Collector', when prompted.
- Message flow accounting and statistics
See: Procedure for collecting accounting and statistics data as XML messages - Stack dump
In hang situations, collect 4-5 stack dumps of the threads running in the DataFlowEngine with a gap of about 10 seconds each.
Use these commands, where <PID> is the process ID of the DataFlowEngine:
- procstack <PID> (On AIX)
- pstack <PID> (On Linux, Solaris)
- Project Interchange files for the problematic flow/set/ESQL/model
- You can export your Message Flow and Message Set project(s) into an archive file for easy transmission to IBM Support.
See: Exporting files from the Workbench
- You can export your Message Flow and Message Set project(s) into an archive file for easy transmission to IBM Support.
- Traces
If the problem occurs during a runtime test or with the Toolkit test facility, WMB execution group traces can be used to gain a better understanding of the problem.
- A service level trace is intended to be provided to IBM Support to assist in the diagnosis of your issue. Please run the service level trace independently and not while the above other items are being collected
- General Broker information
If the mqsidc tool was not run, then capture this information manually- Record the IBM Integration Bus or WebSphere Message Broker version, release, and maintenance level.
This can be captured using the command 'mqsiservice -v'.. - Record the operating system version, release, and maintenance level.
- Record the version, release, and maintenance level of any related products and components for the problematic application.
- Collect the local error log. On UNIX and Linux systems, the local error log is the syslog. The location of your syslog is configured in the syslog daemon.
See: Configuring the syslog daemon - Collect the Standard Output/Error logs.
WMB writes information to both STDOUT and STDERR. These files are located under the Message Broker workpath.
See: Standard System Logs
- Record the IBM Integration Bus or WebSphere Message Broker version, release, and maintenance level.
- Additional information
- Output of command: mqsilist -r -d2
ATTENTION: A good problem description is one of the most important tools IBM needs to analyze your data!
When sending data to IBM, be sure to update your PMR or send a note with the following information:
- Use IBM Service Request to open or view a problem record with IBM.
- Send your data to IBM for further analysis.
Please see the IBM Software Support Handbook for more information on working with IBM support.
Was this topic helpful?
Document Information
Modified date:
23 March 2020
UID
swg21694056