Technical Blog Post
Abstract
(Wily) Java Agent Performance Tuning Recommendations
Body
The WebSphere L2 Support team handles its fair share of WebSphere Application Server performance analysis. Often it appears as though the Monitoring Agent (that thing that reports whether the system is behaving - or not) is the problem itself. The truth is, the more problems that the agent has to report, the more noise the agent is going to interject into the diagnostic data. It's a vicious cycle.
One of the common agents that are encountered is Wily. Like any performance monitoring tool, Wily can introduce a significant amount of noise into WebSphere Application Server diagnostic data. Wily's performance overhead along with the noisy logging can often skew performance analysis.
In order to eliminate agent noise and false positives, WebSphere Support often requests that clients disable Wily. Disabling the agent is only a temporary measure for testing purposes. Once the diagnostic data is captured, Wily can be re-enabled. However, most clients are reluctant to do disable Wily (even temporarily) since they depend on Wily to monitor the health of their system. So how is a proper analysis to be done?
Although IBM Support is not directly qualified to assist clients in tuning their Wily environment, there exists tuning recommendations that are written by the Wily team. Wily tuning can help reduce Wily noise that will assist IBM Support to positively identify the underlying issue. Proper tuning of the Java™ agent promotes harmony between the application server environment and the Java agent:
The Java Agent Performance Tuning Recommendations document
- Provides Wily performance tuning recommendations
- Reminds Wily users that all processes that interact with WebSphere Application Server require tuning for optimal performance
Java Agent Performance Tuning Recommendations
CPU
Disabling the aggregate metrics will help reduce the overall metric load per agent. Metrics show CPU utilization from both JVM and non-JVM processes.
Introscope.agent.disableAggregateCPUUtilization=true
SQL
SQLAgent could potentially generate a large number of unique SQL metrics (aka metric explosion). Reducing the length of SQL statements will help agents to manage the metric load.
Introscope.agent.sqlagent.sql.maxlength=xxx where xxx between 1 and 990
Modify sqlagent.pbd and remove the substring {sql}, which represents the normalized SQL statement. You will only get summary SQL statistics for databases.
Dynamic Instrumentation
Disabling dynamic instrumentation will help reduce overhead on the application caused by the agent. Consider disabling on all production-like agents.
introscope.autoprobe.dynamicinstrument.enabled=false
Remote Dynamic Instrumentation
Disabling remote dynamic instrumentation will help reduce overhead on the application caused by the agent. Consider disabling on all production-like agents.
introscope.agent.remotedynamicinstrumention.enabled=false
Logging
Logging produces I/O overhead for every agent instance. Consider disabling agent logging on production-like agents and leaving the autoprobe for troubleshooting.
introscope.autoprobe.logfile=logs/AutoProbe.log
log4j .appender.logfile=/dev/null
log4j.logger.IntroscopeAgent=OFF
Reduce Memory Overhead
Reducing memory overhead while trading some impact to response times.
introscope.agent.reduceAgentMemoryOverhead=true
The original document here:
Java Agent Performance Tuning Recommendations | CA Technologies Communities
⇒https://communities.ca.com/docs/DOC-231148897
Another Wily tuning document:
CA Technologies Global Webcast Jan 27 2011 Introscope 8x Per... | CA Technologies Communities
⇒https://communities.ca.com/docs/DOC-14500143
title image (modified) credit: (cc) Some rights reserved by geralt
UID
ibm11081317