General system settings
The following topics describe the general operating system settings used for the testing.
randomize_va_space
This setting controls Address Space Layout Randomization (ASLR). ASLR is a security features that can help protect against certain types of buffer overflow attacks by randomizing the base address of code, stack, heap and other program elements to prevent attacks at known, predicted or assumed locations. This feature has been available since kernel 2.6.12 back in 2005.
- 0
- No randomization. Everything is static.
- 1
- Conservative randomization. Shared libraries, stack, mmap(), VDSO and heap are randomized.
- 2
- Full randomization. In addition to elements listed in the previous point, memory managed through brk() is also randomized.
The default setting adopted by most Linux® distributions is full randomization
(2).
Changing the setting to no randomization
(0) permits the greatest chance of sharing and
enhanced cache efficiency. This setting was changed to reduce the variations of the measurement
results in a controlled lab environment. Users should consider the security implications when
changing this setting in their environments.
Kernel Same-page Merging
[root@kvmhost ~] # chkconfig ksm off
[root@kvmhost ~] # service ksm stop
Stopping ksm: [ OK ]
This setting was changed to reduce the variations of the measurement results in a controlled lab environment.
Receive Packet Steering
Modern system configurations rely heavily on network connectivity for many functions. Additionally, newer network components are getting increasingly faster and faster. This results in an ever increasing load on the OS and subsystems.
While newer processors increase capacity, much of these gains are due to additional cores rather than more powerful cores. In order to keep pace with the growing network requirements and load, many newer network adapters support multiple send and receive queues and can use multiple cores to process these queues concurrently.
For adapters that do not support multiple send / receive queues, there remains a desire and need to distribute large/high network loads across multiple processor cores. To address this, the feature Receive Packet Steering (RPS) was developed.
RPS uses a hash algorithm, based on packet IP addresses and ports, to distribute received network traffic across multiple cores. The hash ensures packets for the same data stream are processed on the same CPU.
/sys/class/net/<device/queues/rx-<queue#>/rps_cpus
- <device>
- specifies the actual name of the interface device
- rx-<queue#>
- represents the network queue number being set
[root@kvmhost ~] # cat /sys/class/net/eth0/queues/rx-0/rps_cpus
0000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000
[root@kvmhost ~] # echo 0xf > /sys/class/net/eth0/queus/rx-0/rps_cpus
[root@kvmhost ~] # cat /sys/class/net/eth0/queues/rx-0/rps_cpus
0000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,0000000F
The benefits realized from setting RPS bitmap will vary depending on workload characteristics. Based on measurements conducted on our KVM test systems, setting the bitmap value corresponding to the total number of CPUs configured for the KVM host has obtained the best overall improvement.
This setting is recommended for the KVM host only.
Sharing an OSA adapter across multiple LPARs
IBM Z systems are designed to share many system resources across multiple LPARs configured on the system. This is true for CPUs, memory, IO and other adapters.
The tests conducted for this paper have shown that while an OSA adapter is easily shared across several LPARs, when multiple LPARs are driving extremely high loads using a shared OSA adapter, that the peak network performance is less than when each LPAR is using a separate OSA adapter.
To obtain the highest level of network performance across multiple LPARs running KVM guests on a single system, it is recommended that each KVM host LPAR be configured to use a separate OSA adapter.