Question & Answer
Question
What basic TCP tunings are recommended to improve performance of WAN connections between AIX virtual machines?
Answer
The AIX virtual machine (VM) is used in WAN environment as shown below.
The following basic TCP tunings are recommended on AIX VM:
• tcp_sendspace=1048576 tcp_recvspace=1048576
The tcp_sendspace and tcp_recvspace need to be set to a higher value than LAN setup to keep long pipe populated due to high RTT. These options can be set per interface by using chdev command or system wide using no command or by application by using setsockopt() subroutine.
(1) If application is using setsockopt() subroutine to set tcp_sendspace and tcp_recvspace then change application or its configuration file to set options to 1048576.
(2) Use following command to set per interface:
chdev -l enX -a tcp_sendspace=1048576 -a tcp_recvspace=1048576
Where enX is actual interface name. For example, en0, en1, en2.
Where enX is actual interface name. For example, en0, en1, en2.
(3) Use following command to set system wide: (only used if no options are set per interface and by application)
no -p -o tcp_sendspace=1048576
no -p -o tcp_recvspace=1048576
Note: The options set by an application override option set per interface and set system wide. The options set per interface overrides options set system wide.
• sb_max=2097152
sb_max sets an upper limit for tcp_sendspace and tcp_recvspace. It is generally 2 or 4 times of tcp_sendspace and tcp_recvspace. This option can be set system wide only.
Use following command to set system wide:
no -p -o sb_max=2097152
• rfc1323=1
The rfc1323 needs to be enabled at both ends to use tcp_sendspace and tcp_recvspace higher than 64 Kbytes. These options can be set per interface by using chdev command or system wide using no command.
Use following command to set per interface:
chdev -l enX -a rfc1323=1
Where enX is actual interface name. For example, en0, en1, en2.
Where enX is actual interface name. For example, en0, en1, en2.
Use following command to set system wide:
no -p -o rfc1323=1
• tcp_nodelayack=0
When tcp_nodelayack is set to 0, the TCP stack delays sending ACK packets by up to 200ms. This minimizes the system overhead and number of packets on the network. This option can be set system wide only.
Use following command to set system wide:
no -p -o tcp_nodelayack=0
• sack=1
The sack needs to be enabled at both ends. It is useful for fast recovery from multiple packets drop in a window of data. This option can be set system wide only.
Use following command to set system wide:
no -p -o sack=1
• tcp_cubic=1
tcp_cubic enables CUBIC congestion algorithm as specified in RFC 8312. This modifies the congestion control mechanism for use with TCP connections with long and large bandwidth networks to improve the average throughput. This option is only available in AIX 7.3 and above.
Use following command to set system wide:
no -p -o tcp_cubic=1
• mtu_bypass=on
mtu_bypass enables largesend when largesend is enabled on VIOS. The largesend is enabled on VIOS by default. This option can be set per interface only by using chdev command.
Use following command to set per interface:
chdev -l enX -a mtu_bypass=on
Where enX is actual interface name. For example, en0, en1, en2.
Where enX is actual interface name. For example, en0, en1, en2.
• tcp_init_window=10
Changing tcp_init_window allows to tune the TCP slow start to control the number of TCP segments (packets) outstanding before an ACK is received. For example, setting value to 6 would allow 6 packets to be sent initially, instead of the normal 2 or 3 packets, thus speeding up the initial packet rate. Changing the value to a value of 10 or above may improve the performance of some workloads, especially where there are bursts of traffic, or for short lived connections.
Use following command to set this value to 10:
no -p -o tcp_init_window=10
Note:
(a) The tcp_init_window is used only when rfc2414 is turned on (ignored otherwise). Use following command to enable rfc2414 .
no -p -o rfc2414=1
(b) If IPsec is configured then traffic flow usually have more packet processing overhead compared to unencrypted TCP/IP traffic without IPsec. To achieve better throughput, it is recommended to increase the tcp_init_window to a higher value starting from 10, 64, 128, 256, 512, 1K, 4K, etc. Experiment with lower value to higher values till the bandwidth is saturated. Use command "ipsecstat" to check if IPsec is configured.
Author: Darshan Patel
Platform: AIX on Power
Feedback: aix_feedback@wwpdl.vnet.ibm.com
Platform: AIX on Power
Feedback: aix_feedback@wwpdl.vnet.ibm.com
[{"Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"ARM Category":[{"code":"a8m0z000000cw0AAAQ","label":"Networking-\u003ETCPIP"},{"code":"a8m0z000000cw0qAAA","label":"Performance-\u003ENetwork"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)"}]
Was this topic helpful?
Document Information
Modified date:
16 May 2024
UID
ibm16410510