Understanding OSD usage stats

Edit online

Use the ceph osd df command to view OSD utilization stats.

Example 

[ceph: root@host01 /]# ceph osd df
ID CLASS WEIGHT  REWEIGHT SIZE    USE     DATA    OMAP    META    AVAIL   %USE VAR  PGS
 3   hdd 0.90959  1.00000  931GiB 70.1GiB 69.1GiB      0B    1GiB  861GiB 7.53 2.93  66
 4   hdd 0.90959  1.00000  931GiB 1.30GiB  308MiB      0B    1GiB  930GiB 0.14 0.05  59
 0   hdd 0.90959  1.00000  931GiB 18.1GiB 17.1GiB      0B    1GiB  913GiB 1.94 0.76  57
MIN/MAX VAR: 0.02/2.98  STDDEV: 2.91

  • ID: The name of the OSD.

  • CLASS: The type of devices the OSD uses.

  • WEIGHT: The weight of the OSD in the CRUSH map.

  • REWEIGHT: The default reweight value.

  • SIZE: The overall storage capacity of the OSD.

  • USE: The OSD capacity.

  • DATA: The amount of OSD capacity that is used by user data.

  • OMAP: An estimate value of the bluefs storage that is being used to store object map (omap) data (key value pairs stored in rocksdb).

  • META: The bluefs space allocated, or the value set in the bluestore_bluefs_min parameter, whichever is larger, for internal metadata which is calculated as the total space allocated in bluefs minus the estimated omap data size.

  • AVAIL: The amount of free space available on the OSD.

  • %USE: The notional percentage of storage used by the OSD

  • VAR: The variation above or below average utilization.

  • PGS: The number of placement groups in the OSD.

  • MIN/MAX VAR: The minimum and maximum variation across all OSDs.