IBM Support

Sizing the Virtual I/O Server (VIOS) - My Rules of Thumb

How To


Summary

Sizing the VIOS is tricky without "unknowable" network I/O and disk I/O stats.
Here is what I recommend as a VIOS start point. Once the VIOS is operating, monitor it and adjust, as necessary.

Objective

Nigels Banner

Steps

I often get asked: How large to make a pair of Virtual I/O Server (VIOS)?

The classic consultant answer is "it depends on what you are doing with Disk I/O & Network I/O" is not very useful to the practical guy that has to size a server nor people defining the VIOS partition to install it!  The problem is no one know the I/O that is generated for the specific workload of your tens to hundreds of virtual machines.
Note: Added updated guidance at the end of this article in RED.

These are my personal Rules of Thumb and not official numbers.


The VIOS server unfairly gets a bad press but note:
  1. Physical adapters are now in the VIOS, so device driver CPU cycles (normally hidden and roughly half of the OS CPU System time) move to the VIOS - this is not new CPU cycles.
  2. Extra CPU work involves function shipping the request from client to VIOS and back but this is a function call to the Hypervisor = "small beer" means not many CPU instructions.
  3. Data shipping is efficient as the Hypervisor uses virtual memory references rather that raw data moving.
  4. Aggregating the adapters in one place means that all client virtual machines have access to much larger and redundant data channels at reduced cost so it is a win-win situation.
Who knows the I/O details to rates and packet sizes?
  • Answer: No one (in my experience) knows the disk and network mixture of block or packet sizes, the read and write rates for each size or the periods of time, which cause a peak in workload. In new workloads, it is all guess work anyway - lets be generous that would be + or - 25%.
  • If you do know, IBM can do some maths to estimate the CPU cycles at the peak period.
  • But most of the time that peak sizing would be total overkill.
My "Rules of Thumb" (ROT):
  • Trick 1 - "Use the PowerVM, Luke!"
    • Use PowerVM to reuse unused VIOS CPU cycles in the client virtual machines
    • VIOS = Micro-partition Shared CPU, Uncapped, high weight factor, with virtual processor minimum +1 or +2 headroom (virtual processor would be better called spreading factor)
    • PowerVM allows for workload peaks but doesn't waste CPU resources
  • Trick 2 – Don’t worry about the tea bags!
    • No one calculates the number of tea bags they need per year
    • In my house, we simply have some in reserve and monitor the use of tea bags and then purchase more when needed
    • Likewise, start with sensible VIOS resources and monitor the situation
  • Trick 3 - Go dual VIOS
    • Use a pair of VIOS to allow VIOS upgrades
    • In practice, we have low failure rates in the VIOS - mostly because systems administrators are recommended NOT to fiddle!
    • If the VIOS is using high-speed adapters like 10+ Gbps Ethernet or 16+ Gbps SAN, then VIOS buffering space is needed, so double the RAM to 4 GB.
      • Ignore this rule, if you have these adapters but are only likely to use a fraction of the bandwidth like 1 Gbps.
    • If you know that your applications are I/O intensive (that is likely to stress the high-speed adapters) then go to 6 GB or 8 GB of RAM.
    • If you are using large numbers of tiny virtual machines (lots of virtual connections) also go to 6 GB or 8GB.
    • On large machines, like 32 processor (cores) or more, many customers use a pair of VIOS for production and further a pair for other workloads.
  • Trick 4 - the actual "Rule of Thumb"
    • VIOS base memory = 4 GB
    • Each VIOS: for every 16 CPUs – 1.0 Shared CPU and 2 GB of memory
    • This assumes that your virtual machines are roughly 1 to 2 CPUs each and not extremely I/O intensive (more CPU limited)
  • Trick 5 - Check VIOS performance regularly
    • As workloads are added to a machine in the first few months, monitor VIOS CPU and memory use & tune as necessary
    • See other AIXpert blog entries for monitoring tools - whole machine and including VIOS
  • Trick 6 - Driving system utilisation beyond, say, 70%
    • As you drive system utilisation up by adding more workloads, you need more pro-active monitoring
    • Implement some tools for Automatic Alerting of VIOS stress

Remember that these rules are a starting point:  Monitor the VIOS as workloads increases by using your favorite tools.  I suggest the VIOS "part" command, nmon or njmon for VIOS.

Starving your VIOS is a bad idea as it impacts all virtual machines. 

Update for 2015 and assuming POWER8 and beyond

IBM WorkLoad Estimator (WLE)
Ball park starting points then mandatory monitoring and use the VIOS Performance Advisor (VIOS command called: part) plus my njmon for VIOS
  • Tiny machine (~4 CPU cores)
    • Entitlement = 0.5 probably one VIOS
    • Virtual Processors = 1    
    • RAM = 4 GB
    • Uncapped, weight=200
  • Small machine (12-16 CPU cores)
    • Entitlement = 1 each VIOS (more than 1, if using high-speed adapters that are used at high speed)
    • Virtual Processors = 2   
    • RAM = 6 GB
    • Uncapped, weight=200
  • Larger
    • Entitlement = 2 to 8 each VIOS
    • Virtual Processors = E+1  
    • RAM = 8 GB to 16 GB (depends on high-speed adapters being using at high speed)
    • Uncapped, weight=200

Additional Information


Other places to find Nigel Griffiths IBM (retired)

Document Location

Worldwide

[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"Component":"","Platform":[{"code":"PF002","label":"AIX"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB08","label":"Cognitive Systems"}},{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"HW1W1","label":"Power -\u003EPowerLinux"},"Component":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"","label":""}},{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG60","label":"IBM i"},"Component":"","Platform":[{"code":"PF012","label":"IBM i"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB57","label":"Power"}}]

Document Information

Modified date:
09 January 2024

UID

ibm11165174