Transport mode and tunnel mode

The manner in which the original IP packet is modified depends on the encapsulation mode used. There are two encapsulation modes used by AH and ESP, transport and tunnel.

Transport mode encapsulation retains the original IP header. Therefore, when transport mode is used, the IP header reflects the original source and destination of the packet. Transport is most often used in a host-to-host scenario, where the data endpoints and the security endpoints are the same. A transport mode encapsulated datagram is routed, or transported, in the same manner as the original packet.

Figure 1 shows an IPv4 packet that is encapsulated using AH in transport mode:

Figure 1. IPv4 packet encapsulated using AH in transport mode
Shows IP header, authentication header, IP payload, all authenticated.

Figure 2 shows an IPv4 packet that is encapsulated using ESP in transport mode:

Figure 2. IPv4 packet encapsulated using ESP in transport mode
IP head,ESP head,IP payload,ESP trail,ESP auth data; IP payload - ESP trail encrypted; ESP head - trail authenticated

Figure 3 shows an IPv6 packet that is encapsulated using AH in transport mode:

Figure 3. IPv6 packet encapsulated using AH in transport mode
Orig IP header, ext headers, AH, ext headers, IP payload, all authenticated except for mutable fields.

For a description of the IPv6 mutable fields, see RFC 2402. For information about accessing RFCs, see Related protocol specifications.

Figure 4 shows an IPv6 packet that is encapsulated using ESP in transport mode:

Figure 4. IPv6 packet encapsulated using ESP in transport mode
Orig IP header,ext headers,begin auth,ESP,begin encryption,ext headers,IP payload,end encryption/auth,ESP auth data

Tunnel mode encapsulation builds a new IP header containing the source and destination address of the security endpoints. When tunnel mode is used, the outer IP header reflects the source and destination of the security endpoints, which might or might not be the same as the original source and destination IP address of the data connection. The choice of transport or tunnel mode depends on the structure of the network and relies heavily on logical connections between the endpoints. Tunnel mode is required if one of the IKE peers is a security gateway that is applying IPSec on behalf of another host or hosts. A datagram that is encapsulated in tunnel mode is routed, or tunneled, through the security gateways, with the possibility that the secure IPSec packet will not flow through the same network path as the original datagram. To successfully encapsulate and send an outbound packet, the route table must contain a route that can be used to reach the security gateway, as well as a route that can be used to reach the data endpoint. If policy-based routing is being used on a TCP/IP stack where IP security is active, it is important to understand how the two functions interact. For more information, see Considerations for using policy-based routing with IP security.

Figure 5 shows an IPv4 packet that is encapsulated using AH in tunnel mode:

Figure 5. IPv4 packet encapsulated using AH in tunnel mode
Shows new IP header, authentication header, IP packet (original IP header and IP payload), all authenticated.

Figure 6 shows an IPv4 packet that is encapsulated using ESP in tunnel mode:

Figure 6. IPv4 packet encapsulated using ESP in tunnel mode
New IP head,ESP head,IP packet,ESP trail, ESP auth data;IP packet - ESP trailer encrypted, ESP header - ESP trail auth

Figure 7 shows an IPv6 packet that is encapsulated using AH in tunnel mode:

Figure 7. IPv6 packet encapsulated using AH in tunnel mode
New IP head, ext heads, AH, orig IP head, ext heads, IP payload, authenticated except mutable fields in new head

For a description of the IPv6 mutable fields, see RFC 2402.

Figure 8 shows an IPv6 packet that is encapsulated using ESP in tunnel mode:

Figure 8. IPv6 packet encapsulated using ESP in tunnel mode
New IP head;ext heads;begin auth;ESP;begin encrypt;orig IP head;ext heads;IP payload;end auth/encrypt;ESP auth data

Do not confuse tunnel mode encapsulation with IKE tunnel or IPSec tunnel. In this context, tunnel refers only to the method by which IPSec packets are constructed, while IKE and IPSec tunnels are conceptually defined as secure logical connections between hosts. IPSec tunnels can use transport mode or tunnel mode encapsulation.

For a dynamic tunnel, the choice of encapsulation mode is configured using the IpDataOffer statement in an IP security policy configuration file. For a manual tunnel, the choice of IPSec protocol is configured using the IpManVpnAction statement in an IP security policy configuration file. For more details about the IpDataOffer statement and the IpManVpnAction statement, see z/OS Communications Server: IP Configuration Reference.

Guideline: In host-to-host scenarios, transport mode is commonly used because it does not incur the overhead of having to build an additional IP header. However, tunnel mode is equally valid.
Tip: Any configurations that include a secure gateway as an IKE endpoint require tunnel mode; however, if IKEv2 is used, then the selection of encapsulation mode is performed automatically. The IKED will use transport mode if it determines that the connection is host-to-host; otherwise the IKED will use tunnel mode encapsulation. This behavior can be overridden by using the HowToEncapIKEv2 parameter on the associated KeyExchangeAction statement; see the KeyExchangeAction statement in z/OS Communications Server: IP Configuration Reference for details.
Rule: If you are using Sysplex-Wide Security Associations, a dynamic Security Association cannot use a subnet or range that encompasses a DVIPA address. For takeover to work consistently, Security Associations must be negotiated for single IP addresses only. If two DVIPAs that are covered by the same Security Association are subsequently taken over by two different backup stacks, the coverage of the Security Association is ambiguous because it is then linked to two DVIPAs on two different stacks.