Using the MacVTap driver

Edit online

Another alternative to using a bridge to enable a KVM guest to communicate externally is to use the Linux® MacVTap driver.

“Macvtap is a new device driver meant to simplify virtualized bridged networking. It replaces the combination of the tun/tap and bridge drivers with a single module based on the macvlan device driver. A macvtap endpoint is a character device that largely follows the tun/tap ioctl interface and can be used directly by kvm/qemu and other hypervisors that support the tun/tap interface. The endpoint extends an existing network interface, the lower device, and has its own mac address on the same ethernet segment. Typically, this is used to make both the guest and the host show up directly on the switch to which the host is connected.1

A key difference between using a bridge is that MacVTap connects directly to the network interface in the KVM host. This direct connection effectively shortens the codepath, by bypassing much of the code and components in the KVM host associated with connecting to and using a software bridge. This shorter codepath usually improves throughput and reduces latencies to external systems.

MacVTap modes

MacVTap can be configured in any of three different modes which determine how the macvtap device communicates with the lower device in the KVM host. The three possible modes are VEPA, Bridge and private mode.

  1. Virtual Ethernet Port Aggregator (VEPA) is the default mode. Data flows from one endpoint down through the source device in the KVM host out to the external switch. If the switch supports hairpin mode, the data is sent back to the source device in the KVM host and from there sent to the destination endpoint.

    Most switches today do not support hairpin mode, so the two endpoints are not able to exchange ethernet frames, although they might still be able to communicate using an tcp/ip router. A linux host used as the adjacent bridge can be put into hairpin mode by writing to /sys/class/net/dev/brif/port/hairpin_mode. This mode is particularly interesting for data-centers and cloud provides where the ability to manage virtual machine networking at the switch level is desirable. Switches aware of the VEPA guests can enforce filtering and bandwidth limits per MAC address without the Linux host knowing about it.

  2. Bridge, connecting all endpoints directly to each other. Two endpoints that are both in bridge mode can exchange frames directly, without the round trip through the external bridge. This is the most useful mode for setups with classic switches, and when inter-guest communication is performance critical. (Figure 1 shows the communication path using Bridge mode.)
  3. For completeness, a private mode exists that behaves like a VEPA mode endpoint in the absence of a hairpin aware switch. Even when the switch is in hairpin mode, a private endpoint can never communicate to any other endpoint on the same lowerdev.2
Figure 1. Single LPAR using MacVTap bridge mode for direct connection between KVM guests

MacVTap isolation / limitations

Each of these modes provide different levels of endpoint isolation. Consider your needs when choosing which macvtap mode is right for your environment.

Bridge mode provides the greatest degree of flexibility enabling inter-guest communication within a single KVM host.

VEPA mode, without a network switch thats supports hairpin mode, prevents any endpoints using the same KVM host interface from being able to communicate.

In Private mode, even with a network switch thats supports hairpin mode, a KVM guest endpoint will not be able to communicate with its KVM host using the same lower level source device.

Without a switch that supports hairpin mode, KVM guests configured to use Bridge or VEPA modes will not be able to directly communicate with the KVM host using the same KVM host interface. This limitation can be overcome if the KVM host has multiple interfaces using different ethernet segments (subnets).

To configure a KVM guest interface to use the MacVTap driver, the XML stanza to define the interface for KVM guest the in the guest's libvirt XML configuration file should include the following:

<interface type=”direct”> 
    <mac address=12:34:56:78:9a:bc/> 
    <source dev=private1 mode=bridge/> 
    <model type=virtio/>    
    <driver name=vhost/> 
</interface>
The relevant XML tags here are:
  • <interface type=direct>
  • <source dev=kvm-host-device mode=bridge>

The interface type for macvtap is direct, which specifies a direct mapping to an existing KVM host device. The source dev keyword specifies the KVM host network interface name that will be used by the macvtap interface for the KVM guest. The mode keyword defines which macvtap mode will be used.

The MAC address field is optional and if omitted, the libvirt daemon will generate a unique address.

After the changes have been saved in the KVM guest's libvirt XML configuration file, the libvirt daemon needs to be informed, using the following commands:

[root@kvmhost ~] # virsh undefine <kvm-guest-name> 
[root@kvmhost ~] # virsh define <kvm-guest-libvirt-xml-file>

And lastly, remember to restart the KVM guest for the changes to take affect. The defined macvtap interface will be persistent and automatically started whenever the KVM guest restarts.

All of MacVTap testing done in this paper used the macvtap mode=”bridge” which allowed KVM guests on the same KVM host attached to the same KVM host network interface to communicate with each other. This enabled KVM guests on the same KVM host with uperf client and server roles to be paired.

1 MacVTap - Linux Virtualization Wiki
2 MacVTap - Linux Virtualization Wiki