Troubleshooting
Problem
Diagnosing The Problem
- If verboseGC is not enabled, enable it per "WebSphere Liberty verbosegc from Startup Recipe"
- Download the attached script and put it in the liberty_home/bin folder.
linperf.sh
NOTE: If the script fails with the errors "Bad Interpreter: No Such File or Directory" or "syntax error near unexpected token `$'do\r'`", please refer to the FAQ section at the bottom for instructions .
CAUTION: In certain Linux systems, using 'kill -3' or the script on the JVM PID may cause a 20-second JVM pause. - At the time of the problem, run the linperf.sh script so it generates javacores/thread dumps and important OS level data:
- Type: ./linperf.sh [pid] (where [pid] is the process ID of one or more problematic JVMs). If specifying multiple PIDs, ensure they are separated by a space. This script needs to be executed as the root user or the owner of provided PIDs. As with any script you might need to add execute permissions before executing the script (chmod).
- Skip this step if the issue you are experiencing does not involve high CPU usage. Run this command with -c option and specify the CPU threshold value. For example, to monitor PID(s) with a CPU usage of 80%. Replace the CPU_THRESHOLD_VALUE with 80 or little lower (without percentage). It will monitor the CPU usage of provided PID(s) every 30 seconds. If the CPU usage exceeds the threshold, it will automatically collect data and exit.
./linperf.sh -c CPU_THRESHOLD_VALUE 1234 5678 - The script completes in four minutes and creates a file named linperf_RESULTS.<DATE>.<TIMESTAMP>.tar.gz. It also generates:
- IBM JDK: a series of javacore.*.txt files
- Oracle JDK: thread dumps in the console.log file
- Once the script finishes its work, collect the following files:
- Logs (messages.log, console.log, etc)
- linperf_RESULTS.<DATE>.<TIMESTAMP>.tar.gz (including javacores).
- server.env, server.xml, and jvm.options
Frequently asked questions (FAQs):
What is the impact of enabling verboseGC?
verboseGC Data is critical to diagnosing these issues. This can be enabled on production systems because it has a negligible impact on performance (< 2%).
What is the linperf_RESULTS.tar.gz file and where can I find it?
The linperf_RESULTS.
Does the script gather javacores?
Yes if javacores are in the default directory or in the user-specified directory set by -Xdump:directory generic JVM argument or IBM_JAVACOREDIR environment variable.
What are 'javacores' and where do I find them?
javacores are snapshots of the JVM activity and are essential to troubleshooting these issues. These files will usually be found in the <profile_root>. If you don't find the files here, you can search your entire system for them using the following command: find / -name "*javacore*"
The script fails with with the errors "Bad Interpreter: No Such File or Directory" or "syntax error near unexpected token `$'do\r'`"
The following methods can lead to script fails when used in Windows: - Copied the script’s content from a browser into a local file and then copied it to a Linux box using FTP or SCP. - Copied the script’s content directly into a new file on a Linux box. For details and solutions, see this Fix: “Bad Interpreter: No Such File or Directory” Error in Linux . Instead of manually fixing the script after doing above actions, this approach is easier and faster to get the job done:
1. Download the script using a browser (or a curl command).
2. Use the following scp command in Command Prompt or PowerShell to copy the script to a Linux box:
scp c:/path/to/linperf.sh user_name@<ip_address>:/path/to/put
curl -o linperfh.sh https://www.ibm.com/support/pages/system/files/inline-files/linperf.sh && scp linperf.sh user_name@<ip_address>:/path/to/put
Document Location
Worldwide
Was this topic helpful?
Document Information
Modified date:
25 March 2024
UID
ibm11073532