IBM Support

nmon2WLE using nmon data for IBM WorkLoad Estimator Input

How To


Summary

Use nmon data as the input into the Workload Estimator (WLE) sizing tool for a new POWER Computer

Objective

Nigels Banner

Steps

Update Version 16 - released 18 December 2023

Changed format a little in light of the experience of uploading to the IBM Server Work Load Estimator:

  • It is mandatory to include the header line. Now generated by nmon2WLE -c

  • IBM WLE tool needs a specific string to identify the System = the web tool pops-up the list for you to select from. Unfortunately, these strings and user friendly server names (like Power Systems E850) are not available within AIX so nmon reports just the System Model: IBM,8408-E8E.  You need to fix the model name before the upload in the nmon2WLE file or afterwards in the WLE tool.

  • Version 12: now with average, max and 95 = 95th Percentile support for CPU, disks, network

  • Version 11: runs faster ~100 nmon files a minute

The IBM systems Workload Estimator is the official tool for estimating and consolidating workloads and sizing Power Systems Machines (and other platforms)
  • You can find it here for use directly on the Internet: https://www-912.ibm.com/wle/EstimatorServlet
  • It is a clever, useful, powerful tool and includes lots of calculations for memory requirements and VIO Servers that use "rule of thumb" that are not otherwise public.
Many AIX users collect nmon data files daily and these performance stats could be used as input to the tool.  nmon collects 1000's of performance statistics for performance tuning. For Capacity Planning estimation, we need to pick out the dozen or so statistics needed.  
The Workload Estimator uses a low number core set of statistics and allows bulk import using a Comma Separated Values (CSV) file. Given server consolidation sizing from older machines to a nice new flesh POWER8, we have one line of input for each current Logical Partition (also known as a virtual machine). Focusing on the performance stats - there are some you leave blank to fill in with the online tool (in green).
  • Name - For the user to recognize the workload. We could use the Logical Partition or virtual machine name.
  • Quantity - WLE use: to quickly allow for multiples of this workload.
  • Consolidate to - WLE use: target machine.
  • System - WLE use: Source machine. We could use the Serial number of the source machine.
  • Active Cores - in Power that is a vague term. When the LPAR is in shared CPU and capped mode then the Max cores is the Entitlement otherwise the Max cores are the Virtual CPU or Dedicated CPUs.
  • CPU Utilisation - I prefer Physical CPU use as it is a real number.  The bulk of users are using shared processors and utilisation tops out at 100% when the Entitlement is used then it uses up to the Virtual Processor number on top but still says 100%.
  • Memory in MB
  • Disk read operations per second
  • Disk read block sizes in bytes
  • Disk write operations per second
  • Disk write block sizes in bytes
  • Storage in GB
  • Network MB per second
  • Network operations per seconds
More information is documented here:
  • http://www-912.ibm.com/wle/EstimatorServlet?view=scon
The alternative is switching on nmon_topas to directly collect WLE formatted files - you can find out how here:
  • https://www-912.ibm.com/wle/HTML/en/Tutorial/TopasWalkthru.pdf  
But not many people think about this Capacity Planning far enough and advance to capture the heaviest workloads of the year.
 I ask a WLE developer "The CPU numbers are they the average numbers or the maximum?" and the answer was "yes, it depends".  From which I gather, if the data was gathered all day then we ignore the quiet periods by using the maximums and hope we don't have a 5 second massive peak that will skew the numbers.  If the nmon file was collected during a busy peak the averages might be better to eliminate occasional peaks.
So 3 hours "hacking" in the old sense of the word and we get a 40 line ksh script which outputs for my tiny nmon sample file - currently it includes both average and maximum values.  Version 3 output:
INPUT file is blue_150824_1017.nmon
blue LPAR name
100271P SerialNumber
  5.3 CPU Utilisation percent Average (User+Sys)
15.0 CPU Utilisation percent Maximum
4 Virtual CPUs
2.00 Entitlement
Uncapped CPU Mode Note:If Uncapped then ActiveCores=VirtualCPUs If Capped then ActiveCores=Entitlement
4096 MB of memory
7.9 Disk Read IOs per second Average
59.7 Disk Read IOs per second Maximum
374.2 DiskRead KB per second Average
54272 DiskRead Size in bytes Average
9.2 Disk Write IOs per second Average
48.4 Disk Write IOs per second Maximum
67.4 Disk Write per second Average
7168 Disk Write Size in bytes Average
511.0 Storage GB in Volume groups
244.0 Network MB/s Average
664.5 Network MB/s Maximum
294.7 Network OP/s Average
437.9 Network OP/s Maximum
Version 14 output one line ready for input in the WLE:
$ ./nmon2WLE  -h
nmon2WLE formatting script for nmon files to IBM Workload Estimator format.
Which can be found here - https://www-912.ibm.com/estimator
Testing Limit: Assumes AIX nmon files - Linux may work at your own risk
Limitation: only includes stats from the first 150 disks
Version 14, Nigel Griffiths, ksh shell code example i.e. Copyright is impossible.
Usage nmon2WLE {-h | -c}
-h = this help info
-c = output the WLE column names - mandatory 1st line
Usage nmon2WLE {average | max | 95} filename.nmon
  *  average - Average of the stats - Good if you captured a busy period (hour)
  *  max - Maximum peak stats - Good if captured whole day with idle periods
  *  95 - Calculate 95th percentile - Like max but ignores top 5% peaks
IBM Workload Estimator Colunm names are:
Name, Quantity, System, ActiveCores, StorageGB, CPUutilization, MemoryMB, DiskReadsPerSecond, BytesPerReadOp, DiskWritesPerSecond, BytesPerWriteOp, StorageGB, NetworkMBPerSecond, NetworkOpsPerSecond, OverheadPercent, ConcurrencyPercent
Example output:
Identifier,Quantity,System,Active Cores,Storage (GB),CPU Utilization,Memory (MB),Disk Reads Per Second,Bytes Per Read Op,Disk Writes Per Second,Bytes Per Write Op,Network (MB/S),Network Ops/Sec,Overhead,Concurrency
0299C7B-green10-22-FEB-2015, 1,Power System E880 MHE (22U) POWER8 4.02GHz (16ch/192co), 24, 910,   64, 131072, 21947, 32042, 18, 6144, 102454, 67532, 5, 70
0299CBC-green11-22-FEB-2015, 1,Power System E880 MHE (22U) POWER8 4.02GHz (16ch/192co), 24, 910,   50, 131072, 73943, 31744, 18, 6144, 59341, 44801, 5, 70
0299CCE-green12-22-FEB-2015, 1,Power System E880 MHE (22U) POWER8 4.02GHz (16ch/192co), 24, 910,   64, 131072, 994245, 25600, 19, 5120, 100324, 70579, 5, 70
Warnings:
  1. The script assumes AIX as some of the numbers are extracted from AIX 6 or 7 only data.  AIX 5.3 does not include many mandatory statistics that are used by the script.
  2. Linux might be possible but with so many variations of distributions, versions, and platforms it would take ages to do them all, so I am ignoring Linux for now.
  3. The current version covers machines with up to 150 disks - over that nmon has have to do strange things to stop originally Lotus 1-2-3 but now Microsoft Excel from crashing with too many columns.
What is next?
  • Volunteers with WLM experience to check we are going in the right direction.
  • Need some testing against real nmon data sets - and some hand cranked checking of the numbers or compare to nmonchart graphs.
  • Volunteers are welcome. 
Download: 

Additional Information

Other places to find content from Nigel Griffiths IBM (retired)

Document Location

Worldwide

[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"Component":"","Platform":[{"code":"PF002","label":"AIX"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB08","label":"Cognitive Systems"}},{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"HW1W1","label":"Power -\u003EPowerLinux"},"Component":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"","label":""}}]

Document Information

Modified date:
19 December 2023

UID

ibm11116219

Page Feedback