Large pages and hotplug memory

6.6 LPAR mode z/VM guest KVM guest

You configure hugetlbfs huge-page support by adding parameters to the kernel parameter line.

With huge-page support built into the kernel, you can use 1 MB huge pages without further configuration. Preallocate huge pages through kernel parameters to reserve continuous large blocks of memory and so assure that a sufficient number of huge pages is available when required.

Huge-page support kernel parameter syntax

Notes:

¹ Relevant only to KVM hosts.

where:

default_hugepagesz=<size>: specifies the default page size in byte. You can use suffixes K, M, and G to specify KB, MB, and GB. The default value is 1 MB.
hugepages=<number>: is the number of huge pages to be allocated at boot time.
hugepagesz=<size>: specifies the page size in byte. You can use suffixes K, M, and G to specify KB, MB, and GB.
kvm.hpage=1: enables KVM hosts to back the memory of their guests with huge pages. For more information about kvm module parameters, including the corresponding parameter for kvm as a separate module, see Setting up KVM host support.

Note: If you specify more pages than available, Linux® reserves as many as possible. As a likely result, too few general pages remain for the boot process, and your system stops with an out-of-memory error.

Pre-allocating 2 GB huge pages

If Linux is running in an LPAR or as a KVM guest, you can use 2 GB huge pages.

Before you can use 2 GB huge pages, you must pre-allocate them to the kernel page pool. To pre-allocate 2 GB pages, precede the hugepages= parameter with the page size selection parameter, hugepagesz=2G.

Tip: Memory quickly becomes fragmented after booting, and new 2 GB huge pages cannot be allocated. Use kernel boot parameters to allocate 2 GB huge pages to avoid the memory fragmentation problem.

To pre-allocate a number of pages of 2 GB size and also set the default size to 2 GB:

default_hugepagesz=2G hugepagesz=2G hugepages=<number>

Setting up multiple huge-page pools

You can allocate multiple huge-page pools and use them simultaneously. To allocate multiple huge-page pools, specify the hugepagesz= parameter several times, each time followed by a corresponding hugepages= parameter.

For example, to specify two pools, one with 1 MB pages and one with 2 GB pages, specify:

hugepagesz=1M hugepages=8 hugepagesz=2G hugepages=2

This creates a sysfs attribute for each pool, /sys/kernel/mm/hugepages/hugepages-<size>kB/nr_hugepages, where <size> is the page size in KB. For the example given, the following attributes are created:

/sys/kernel/mm/hugepages/hugepages-1024kB/nr_hugepages
/sys/kernel/mm/hugepages/hugepages-2097152kB/nr_hugepages

Huge pages and hotplug memory

Hotplug memory that is added to a running Linux instance is movable and can be allocated to movable resources only.

By default, huge pages are not movable and cannot be allocated from movable memory. You can enable allocation from movable memory with the sysctl setting hugepages_treat_as_movable.

To enable allocation of huge pages from movable hotplug memory, issue:

# echo 1 > /proc/sys/vm/hugepages_treat_as_movable

Although this setting makes huge pages eligible for allocation through movable memory, it does not make huge pages movable. As a result, the allocated hotplug memory cannot be set offline until all huge pages are released from that memory.

To disable allocation of huge pages from movable hotplug memory, issue:

# echo 0 > /proc/sys/vm/hugepages_treat_as_movable