Messenger v2 protocol

Edit online

Messenger v2 (msgr2) is the new protocol for Ceph clients, or other Ceph daemons, to connect to the Ceph Monitor daemon. Ceph daemons bind to multiple ports, allowing both the legacy v1-compatible, and the new, v2-compatible Ceph clients to connect to the same storage cluster.

The second version of Ceph’s on-wire protocol, msgr2, has the following features:

  • A secure mode encrypting all data moving through the network.

  • Encapsulation improvement of authentication payloads, enabling future integration of new authentication modes.

  • Improvements to feature advertisement and negotiation.

Ceph clients or other Ceph daemons connecting to the Ceph Monitor daemon uses the v2 protocol first, if possible, but if not, then the legacy v1 protocol is used. By default, both messenger protocols, v1 and v2, are enabled. The new v2 port is 3300, and the legacy v1 port is 6789, by default.

The messenger v2 protocol has two configuration options that control whether the v1 or the v2 protocol is used:

  • ms_bind_msgr1 - This option controls whether a daemon binds to a port speaking the v1 protocol; it is true by default.

  • ms_bind_msgr2 - This option controls whether a daemon binds to a port speaking the v2 protocol; it is true by default.

Similarly, two options control based on IPv4 and IPv6 addresses used:

  • ms_bind_ipv4 - This option controls whether a daemon binds to an IPv4 address; it is true by default.

  • ms_bind_ipv6 - This option controls whether a daemon binds to an IPv6 address; it is true by default.

Note: The ability to bind to multiple ports has paved the way for dual-stack IPv4 and IPv6 support.

The msgr2 protocol supports two connection modes:

  • crc

    • Provides strong initial authentication when a connection is established with cephx.

    • Provides a crc32c integrity check to protect against bit flips.

    • Does not provide protection against a malicious man-in-the-middle attack.

    • Does not prevent an eavesdropper from seeing all post-authentication traffic.

  • secure

    • Provides strong initial authentication when a connection is established with cephx.

    • Provides full encryption of all post-authentication traffic.

    • Provides a cryptographic integrity check.

The default mode is crc.

Ceph Object Gateway Encryption

Also, the Ceph Object Gateway supports encryption with customer-provided keys using its S3 API.

Important: To comply with regulatory compliance standards requiring strict encryption in transit, administrators MUST deploy the Ceph Object Gateway with client-side encryption.
Ceph Block Device Encryption

System administrators integrating Ceph as a backend for OpenStack Platform 13 MUST encrypt Ceph Block Device volumes using dm_crypt for RBD Cinder to ensure on-wire encryption within the Ceph storage cluster.

Important: To comply with regulatory compliance standards requiring strict encryption in transit, system administrators MUST use dmcrypt for RBD Cinder to ensure on-wire encryption within the Ceph storage cluster.