After two Shared Memory Communications over RDMA (SMC-R) peers recognize during rendezvous processing that shared memory communications are possible, a logical point-to-point SMC-R link is established between the stacks over the RDMA over Converged Ethernet (RoCE) fabric. An SMC-R link, as shown in Figure 1, is uniquely defined by a combination of the following information:
A VMAC is a 6-byte value that is a virtual representation of the physical MAC address for an IBM® 10GbE RoCE Express® interface (shown as RNIC in Figure 1).
Each TCP/IP stack that activates a particular 10GbE RoCE Express interface is assigned a different VMAC value.
A GID is a 16-byte value. z/OS® Communications Server generates the GID values by converting the VMAC address of the 10GbE RoCE Express interface into an IPv6 link-local address.
A QP represents one end of the logical connection between two RDMA peers. A combination of two reliable connected queue pairs (RC QPs) forms a single logical point-to-point link. The link enables exactly one pair of communicating RDMA peers to send and receive messages and initiate RDMA activities between themselves. A 10GbE RoCE Express interface associates units of work, such as confirmation of sent data or indication of received data, to a specific QP to enable the SMC-R protocols to identify which TCP/IP stack to notify for the unit of work. The stack then determines which TCP connection that uses that RC QP is to process the data.
You can optionally use VLANs to isolate application traffic into different virtual networks on the same physical Ethernet.
For more information about using VLANs, see VLANID considerations.
Application traffic between the two peers that uses the same remote and local VMACs, GIDs, and QPs, and that is associated with the same VLAN when VLANs are defined, can use the same SMC-R link.
In addition to the 7-tuple (local VMAC, GID, QP# + remote VMAC, GID, QP# + VLAN ID) that uniquely defines an SMC-R link, each peer assigns a 4-byte SMC-R link ID value that uniquely identifies the SMC-R link within its own resource space. This SMC-R link ID is exchanged between peers and is intended to be used for network management and diagnostic purposes. For instance, you can use the SMC-R link ID to filter Netstat report information that is related to a specific SMC-R link. For more information, see Displaying SMC-R information.
An SMC-R link supports multiple TCP connections between the same two peers, as shown in Figure 2. The first TCP connection between the peers establishes the SMC-R link, and subsequent TCP connections between the peers can use the previously established link. Because subsequent TCP connections between the peers can use the previously established link, extra SMC-R link setup costs between the peers are avoided.