Active Memory Expansion (AME)
Active Memory Expansion (AME) is a new technology for expanding a system's effective memory capacity. AME employs memory compression technology to transparently compress in-memory data, allowing more data to be placed into memory and thus expanding the memory capacity of configured systems.
Overview
Active Memory Expansion (AME) relies on compression of in-memory data to increase the amount of data that can be placed into memory and thus expand the effective memory capacity of a IBM® Power Systems processor-based server. The in-memory data compression is managed by the operating system, and this compression is transparent to applications and users. AME is configurable on a per-logical partition (LPAR) basis. Thus, AME can be selectively enabled for one or more LPARs on a system.
When Active Memory Expansion is enabled for an LPAR, the operating system compresses a portion of the LPAR's memory and leaves the remaining portion of memory uncompressed. This results in the memory being effectively broken up into two pools. They are:
- Compressed pool
- Uncompressed pool
The operating system varies dynamically the amount of memory that is compressed based on the workload and the configuration of the LPAR.
The operating system moves data between the compressed and uncompressed memory pools based on the memory access patterns of applications. When an application needs to access data that is compressed, the operating system automatically extracts the data and moves it from the compressed pool to the uncompressed pool, making it available to the application. When the uncompressed pool is full, the operating system compresses the data and moves it from the uncompressed pool to the compressed pool.
This compression and decompression activity is transparent to the application. As AME relies on memory compression, some additional CPU utilization are consumed when the AME is in use. The amount of additional CPU utilization needed for AME varies based on the workload and the level of memory expansion being used.
Memory expansion factor and expanded memory size
LPAR_expanded_mem_size = LPAR_true_mem_size * LPAR_mem_exp_factor 40 GB = 20 GB * 2.0 The operating system compresses enough in-memory data to fit 40 GB of data into 20 GB of memory. The memory expansion factor and the expanded memory size can be dynamically changed at runtime by using the Hardware Management Console (HMC) through dynamic LPAR operations. The expanded memory size is always rounded down to the nearest logical memory block (LMB) multiple.
Memory Deficit
When configuring the memory expansion factor for an LPAR, it is possible that a memory expansion factor might be chosen that is too large and cannot be achieved based on the compressibility of the workload. When the memory expansion factor for an LPAR is too large, then a memory expansion deficit forms, indicating that the LPAR cannot achieve its memory expansion factor target. For example, if an LPAR is configured with a memory size of 20 GB and a memory expansion factor of 1.5, which results in a total target expanded memory size of 30 GB. However, the workload running in the LPAR does not compress well, and the workload's data only compresses by a ratio of 1.4 to 1. In this case, it is impossible for the workload to achieve the targeted memory expansion factor of 1.5. The operating system limits the amount of physical memory that can be used in a compressed pool to a maximum of 95%. In the preceding example with the LPAR memory size as 20 GB, if the amount of physical memory that can be used in a compressed pool is limited to 90%, 18 GB will be reserved for compressed pool. The maximum achievable expanded memory size would be 27.2 GB (2 GB + 1.4 x 18 GB) . The result is a 2.8 GB shortfall. This shortfall is referred to as the memory deficit.
The effect of a memory deficit is the same as the effect of configuring an LPAR with too little memory. When a memory deficit occurs, the operating system cannot achieve the expanded memory target configured for the LPAR, and the operating system might have to resort to paging out virtual memory pages to paging space. Thus, in the previously mentioned example, if the workload used more than 27.2 GB of memory, the operating system would start paging out virtual memory pages to paging space. To get an indication of whether a workload can achieve its expanded memory size, the operating system reports a memory deficit metric. This is a “hole” in the expanded memory size that cannot be achieved. If this deficit is zero, the target memory expansion factor can be achieved, and the LPAR's memory expansion factor is configured correctly. If the expanded memory deficit metric is non-zero, then the workload falls short of achieving its expanded memory size by the size of the deficit. To eliminate a memory deficit, the LPAR's memory expansion factor should be reduced. However, reducing the memory expansion factor reduces the LPAR's expanded memory size. Thus to keep the LPAR's expanded memory size the same, the memory expansion factor must be reduced and more memory must be added to the LPAR. Both the LPAR's memory size and memory expansion factor can be changed dynamically.
Planning Considerations
Before deploying a workload in the Active Memory Expansion (AME) environment, some initial planning is required to ensure that a workload gets the maximum benefit from AME. The benefit of AME to a workload varies based on the workload's characteristics. Some workloads can get a higher level of memory expansion than other workloads. The Active Memory Expansion Planning and Advisory Tool amepat assists in planning the deployment of a workload in the Active Memory Expansion environment and provides guidance on the level of memory expansion a workload can achieve.
AME Planning Tool
The AME Planning Tool (located in /usr/bin/amepat) serves two primary purposes. They are
- To plan an initial Active Memory Expansion configuration
- To monitor and fine-tune an active AME configuration
AME Planning Tool can run on LPARs with and without AME enabled. In an LPAR where AME has not been enabled, run amepat with a representative workload. It should be set to monitor that workload for a meaningful period of time. For example, the amepat tool is set to run for the duration of a workload's peak resource usage. Once completed, the tool displays a report with a variety of potential memory expansion factors and the expected CPU utilization attributable to an AME for each factor. The tool also provides a recommended memory expansion factor that seeks to maximize memory savings while minimizing additional CPU utilizationThe report and recommendation can be a useful initial configuration for an AME deployment. In an LPAR where AME is enabled, amepat serves a similar purpose. When run at peak-time for a representative workload, the tool provides a report with the actual CPU utilization attributable to AME at the current memory expansion factor. It also displays memory deficit information, if one is present. Because the AME is enabled, the tool can also provide a more accurate representation of what CPU utilization levels can be expected at different memory expansion factors. A new recommendation based on this information will be presented to the user.
# amepat 5 2
Command Invoked : amepat 2 5
Date/Time of invocation : Wed Dec 2 11:29:29 PAKST 2009
Total Monitored time : 10 mins 58 secs
Total Samples Collected : 5
System Configuration:
---------------------
Partition Name : aixfvt19
Processor Implementation Mode : POWER5
Number Of Logical CPUs : 8
Processor Entitled Capacity : 4.00
Processor Max. Capacity : 4.00
True Memory : 4.25 GB
SMT Threads : 2
Shared Processor Mode : Disabled
Active Memory Sharing : Disabled
Active Memory Expansion : Disabled
System Resource Statistics: Average Min Max
--------------------------- ----------- ---------- -----------
CPU Util (Phys. Processors) 2.00 [ 50%] 1.00 [ 25%] 3.00 [ 75%]
Virtual Memory Size (MB) 1366 [ 31%] 1113 [ 26%] 2377 [ 55%]
True Memory In-Use (MB) 1758 [ 40%] 1234 [ 28%] 3834 [ 88%]
Pinned Memory (MB) 673 [ 15%] 673 [ 15%] 675 [ 16%]
File Cache Size (MB) 391 [ 9%] 124 [ 3%] 1437 [ 33%]
Available Memory (MB) 841 [ 65%] 1812 [ 42%] 3099 [ 71%]
Active Memory Expansion Modeled Statistics
-------------------------------------------
Modeled Expanded Memory Size : 4.25 GB
Average Compression Ratio : 5.29
Expansion Modeled True Modeled CPU Usage
Factor Memory Size Memory Gain Estimate
--------- ------------- ------------------ -----------
1.00 4.25 GB 0.00 KB [ 0%] 0.00 [ 0%]
1.31 3.25 GB 1.00 GB [ 31%] 0.34 [ 8%]
1.55 2.75 GB 1.50 GB [ 55%] 0.39 [ 10%]
1.89 2.25 GB 2.00 GB [ 89%] 0.45 [ 11%]
2.12 2.00 GB 2.25 GB [112%] 0.50 [ 12%]
2.43 1.75 GB 2.50 GB [143%] 0.65 [ 16%]
2.83 1.50 GB 2.75 GB [183%] 0.70 [ 18%]
Active Memory Expansion Recommendation:
---------------------------------------
The recommended AME configuration for this workload is to configure the LPAR
with a memory size of 1.50 GB and to configure a memory expansion factor
of 2.83. This will result in a memory gain of 183%. With this
configuration, the estimated CPU usage due to AME is approximately 0.50
physical processors, and the estimated overall peak CPU resource required for
the LPAR is 3.50 physical processors.
NOTE: amepat's recommendations are based on the workload's utilization level
during the monitored period. If there is a change in the workload's utilization
level or a change in workload itself, amepat should be run again.
The modeled Active Memory Expansion CPU usage reported by amepat is just an
estimate. The actual CPU usage used for Active Memory Expansion may be lower
or higher depending on the workload.The report comprises six sections that are explained in order.
Command Information
This section provides details about the arguments passed to amepat, time of invocation, the total time the system is monitored and the number of samples collected. In the report, the amepat
System Configuration
System Resource Statistics
This section displays the Utilization of system resources over the monitoring period. It shows the average, minimum, and maximum values along with the corresponding percentages for system resource utilization.
In the report mentioned, the workload on average utilizes 2.00 physical processors (as shown in the CPU Util row) which is 50% of maximum physical capacity (4.00 as shown in the Processor Max Capacity row) of the LPAR.
In the report mentioned above, the memory utilization of the workload can also be observed. All the memory percentages are relative to the total true memory size of LPAR
Active Memory Expansion Statistics
This section is displayed only when the amepat tool is invoked in an AME enabled LPAR.
Find an example of the sample output:
AME Statistics: Average Min Max
--------------- ----------- -------- ---------
AME CPU Usage (Phy. Proc Units) 0.25 [ 6%] 0.01 [ 0%] 0.50 [ 13%]
Compressed Memory (MB) 264 [ 13%] 264 [ 13%] 264 [ 13%]
Compression Ratio 2.15 2.15 2.16
Deficit Memory Size (MB) 562 [ 55%] 562 [ 55%] 562 [ 55%]
This section of the report shows the AME CPU Usage, Compressed Memory,
Compression Ratio & Deficit Memory.
The Deficit Memory will be reported only if there is a memory deficit
in achieving the expanded memory size.
Otherwise the tool will not report this information.
For example in the above report, it can be observed that there is
an average memory deficit of 562 MB which is 55% of the Target
Expanded Memory Size of 2 GB (which is reported in the
System Configuration Section when AME is enabled).
The report also shows on an average 264 MB out of 2GB of expanded memory is in
compressed form as reported against Compressed Memory.Active Memory Expansion Modeled Statistics
This section displays the Modeled Expanded Memory size, compression ratio, and a table with a number of possible AME configurations. In the mentioned report, the Modeled Expanded Memory Size is
reported as 4.25 GB, which is the LPAR's True Memory Size. By default amepat uses the partition's true memory as the Modeled Expanded Memory. This can be altered by using the
-t or -a options. The report shows an average compression ratio of 5.29 which indicates that the workload compresses well. Compression ratios close to 1 might indicate that little
memory expansion is possible. The configurations displayed in the modeling table are based on the Modeled Expanded Memory Size as the target memory.
The table shows Modeled True Memory Size, Modeled Memory Gain, and additional CPU usage attributed to AME for various expansion factors.
For example, see the following row in the table
1.55 2.75 GB 1.50 GB [ 55%] 0.39 [ 10%]Here the original true memory size of 4.25 GB can be achieved with 2.75 GB physical memory size and an expansion factor of 1.55. This configuration may result in the CPU usage increasing by
0.39 physical processors (10% of maximum capacity).
Active Memory Expansion Recommendation
This section displays an Active Memory Expansion Configuration recommendation that is made for the workload. The optimal configuration will never have an AME CPU usage of over 15%. This section also provides information on the AME CPU Usage and Memory Gain for the recommended configuration. In the report mentioned, it is observed that an expansion factor of 2.12 is recommended. The AME CPU usage target can be altered using the -c or -C options.
WARNING: This LPAR currently has a memory deficit of 562 MB.
A memory deficit is caused by a memory expansion factor that is too
high for the current workload. It is recommended that you reconfigure
the LPAR to eliminate this memory deficit. Reconfiguring the LPAR
with one of the recommended configurations in the above table should
eliminate this memory deficit.
The recommended AME configuration for this workload is to configure
the LPAR with a memory size of ...For more information on these and other uses of the AME Planning tool, please refer to the amepat man page.
Performance Monitoring
Several AIX performance tools can be used to monitor Active Memory Expansion statistics and gather information about Active Memory Expansion.
| Tool | Option | Description |
|---|---|---|
| amepat | -N | Provides general CPU and memory utilization statistics. Also provides an indication of the CPU utilization for AME compression and decompression activity, as well as compression, decompression, and memory deficit statistics. |
| vmstat | -c | Provides compression, decompression, and deficit statistics. |
| lparstat | -c | Provides an indication of the CPU utilization for AME compression and decompression activity. Also provides memory deficit information. |
| svmon | -O summary=ame | Provides a summary view of memory usage broken down into compressed and uncompressed pages. |
| topas | The default topas screen displays the memory compression statistics when it is run in the AME environment. |
vmstat command
The vmstat command can be used with it's –c option to display AME statistics.
# vmstat –c 2 1
System configuration: lcpu=2 mem=1024MB tmem=512MB ent=0.40 mmode=dedicated-E
kthr memory page faults
r b avm fre csz cfr dxm ci co pi po in sy cs
0 0 309635 2163 43332 943 26267 174 386 0 0 93 351 339
cpu
us sy id wa pc ec
2 3 89 7 0.02 5.3
In the output above, the following memory compression statistics are provided:
- Expanded memory size mem of the LPAR is 1024 MB.
- True memory size tmem of the LPAR is 512 MB.
- The memory mode mmode of the LPAR is Active Memory Sharing disabled and Active Memory Expansion enabled.
- Compressed Pool size csz is 43332 4K- pages.
- Amount of free memory cfr in the compressed pool is 943 4K- pages.
- Size of expanded memory deficit dxm is 26267 4K- pages.
- Number of compression operations or page-outs to the compressed pool per second co is 386.
- Number of decompression operations or page-ins from the compressed pool per second ci is 174.
lparstat command
The lparstat command can be used with –c option to display AME statistics.
# lparstat -c 2 5
System configuration: type=Shared mode=Uncapped mmode=Ded-E smt=On
lcpu=2 mem=1024MB tmem=512MB psize=14 ent=0.40
%user %sys %wait %idle physc %entc lbusy app vcsw phint %xcpu dxm
----- ----- ------ ------ ----- ----- ------ --- ----- ----- ------ ------
45.6 51.3 0.2 2.8 0.95 236.5 62.6 11.82 7024 2 5.8 165
46.1 50.9 0.1 2.8 0.98 243.8 64.5 11.80 7088 7 6.0 162
46.8 50.9 0.3 2.1 0.96 241.1 69.6 11.30 5413 6 19.4 163
49.1 50.7 0.0 0.3 0.99 247.3 60.8 10.82 636 4 8.6 152
49.3 50.5 0.0 0.3 1.00 248.9 56.7 11.47 659 1 0.3 153In the output, the following memory compression statistics are provided
- Memory mode mmode of the LPAR is Active Memory Sharing disabled and AME enabled.
- Expanded memory size mem of the LPAR is 1024 MB.
- True memory size tmem of the LPAR is 512 MB.
- Percentage of CPU utilized for Active Memory Expansion activity %xcpu.
- Size of expanded memory deficit dxm in megabytes.
topas command
The topas main panel in an LPAR with Active Memory Expansion enabled
displays memory compression statistics automatically under the sub-section AME.
Topas Monitor for host: proc7 EVENTS/QUEUES FILE/TTY
Mon Dec 14 16:30:50 2009 Interval: 2 Cswitch 1240 Readch 43.2M
Syscall 110.8K Writech 102.5K
CPU User% Kern% Wait% Idle% Physc Entc Reads 12594 Rawin 0
ALL 49.1 50.7 0.0 0.3 1.00 249.7 Writes 515 Ttyout 388
Forks 218 Igets 0
Network KBPS I-Pack O-Pack KB-In KB-Out Execs 218 Namei 5898
Total 1.2 7.5 1.0 0.9 0.3 Runqueue 1.0 Dirblk 0
Waitqueue 0.0
Disk Busy% KBPS TPS KB-Read KB-Writ MEMORY
Total 0.0 0.0 0.0 0.0 0.0 PAGING Real,MB 1024
Faults 53184 % Comp 85
FileSystem KBPS TPS KB-Read KB-Writ Steals 0 % Noncomp 0
Total 75.4K 21.1K 75.3K 95.4 PgspIn 0 % Client 0
PgspOut 0
WLM-Class (Active) CPU% Mem% Blk-I/O% PageIn 0 PAGING SPACE
System 0 61 0 PageOut 0 Size,MB 512
Default 0 4 0 Sios 0 % Used 1
% Free 99
Name PID CPU% PgSp Class AME
inetd 364682 3.5 0.5 wpar1 TMEM,MB 512 WPAR Activ 1
xmtopas 622740 0.4 0.7 wpar1 CMEM,MB 114 WPAR Total 1
topas 413712 0.1 1.5 System EF[T/A] 2.0/1.5 Press: "h"-help
random 204934 0.1 0.1 System CI:5.5 CO:0.0 "q"-quitIn the above output, the following memory compression statistics are provided.
- True memory size TMEM,MB of the LPAR is 512 MB.
- Compressed pool size CMEM,MB of the LPAR is 114 MB.
- EF[T/A] – Target Expansion Factor is 2.0 and Achieved Expansion Factor is 1.5.
- Rate of compressions co and decompressions ci per second are 0.0 and 5.5 pages respectively.
svmon command
The svmon tool can provide a detailed view of AME usage on an LPAR.
# svmon -G -O summary=ame,pgsz=on,unit=MB
Unit: MB
-------------------------------------------------------------------------------
size inuse free pin virtual available mmode
memory 1024.00 607.54 144.11 306.29 559.75 136.61 Ded-E
ucomprsd - 387.55 -
comprsd - 219.98 -
pg space 512.00 5.08
work pers clnt other
pin 213.34 0 0 28.9
in use 534.12 0 9.42
ucomprsd 314.13
comprsd 219.98
PageSize PoolSize inuse pgsp pin virtual ucomprsd
s 4 KB - 543.54 5.02 242.27 560.59 323.55
L 16 MB 4 0 0 64.0 0 0
-------------------------------------------------------------------------------
True Memory: 512.00
CurSz %Cur TgtSz %Tgt MaxSz %Max CRatio
ucomprsd 405.93 79.28 168.38 32.89 - - -
comprsd 106.07 20.72 343.62 67.11 159.59 31.17 2.51
txf cxf dxf dxm
AME 2.00 1.46 0.54 274.21In the output above, the following memory compression statistics are provided:
- Memory mode mmode of the LPAR is Active Memory Sharing disabled and AME enabled.
- Out of a total of 607.54 MB in use memory_inuse, uncompressed pages ucomprsd_inuse constitute 387.55 MB and compressed pages comprsd_inuse constitute the remaining 219.98 MB
- Out of a total of 534.12 MB working pages in use inuse_work, uncompressed pages ucomprsd_work constitute 314.13 MB and compressed pages comprsd_work constitute 219.98 MB.
- Out of a total of 543.54 MB of in use pages 4KB_inuse in 4K-PageSize Pool, uncompressed pages 4KB_ucomprsd constitute 323.55 MB.
- Expanded memory size memory_size of the LPAR is 1024 MB.
- True memory size True Memory of the LPAR is 512 MB.
- Current size of the uncompressed pool ucomprsd_CurSz is 405.93 MB (79.28% of the total true memory size of the LPAR).
- Current size of the compressed pool comprsd_CurSz is 106.07 MB (20.72% of the total true memory size of the LPAR).
- The target size of the compressed pool comprsd_TgtSz needed to achieve the target memory expansion factor txf of 2.00 is 343.62 MB (67.11% of the total true memory size of the LPAR).
- The size of the uncompressed pool ucomprsd_TgtSz in that case becomes 168.38 MB (32.89% of the total true memory size of the LPAR).
- The maximum size of the compressed pool comprsd_MaxSz is 159.59 MB (31.17% of the total true memory size of the LPAR).
- The current compression ratio CRatio is 2.51 and the current expansion factor cxf achieved is 1.46
- The amount of expanded memory deficit dxm is 274.21 MB and the deficit expansion factor dxf is 0.54.
The –O summary=longame option provides summary of memory compression details as follows:
# svmon -G -O summary=longame,unit=MB
Unit: MB
Active Memory Expansion
--------------------------------------------------------------------
Size Inuse Free DXMSz UCMInuse CMInuse TMSz TMFr
1024.00 607.91 142.82 274.96 388.56 219.35 512.00 17.4
CPSz CPFr txf cxf CR
106.07 18.7 2.00 1.46 2.50In the output, the following memory compression statistics are provided:
- Out of the total expanded memory size Size of 1024.00 MB, 607.91 MB is in use Inuse and 142.82 MB is free Free. The deficit in expanded memory size DXMSz is 274.96 MB.
- Out of the total in use memory Inuse of 607.91 MB, uncompressed pages UCMInuse constitute 388.56 MB and the compressed pages CMInuse constitute the remaining 219.35 MB.
- Out of the true memory size TMSz of 512.00 MB, only 17.4 MB of True Free memory TMFr is available.
- Out of the compressed pool size CPSz of 106.07 MB, only 18.7 MB of free memory CPFr is available in the compressed pool.
- Whereas the target expansion factor txf is 2.00, the current expansion factor cxf achieved is 1.46.
- The compression ratio (CR) is 2.50.