Enabling monitoring stack

Alerts for the monitoring stack of the IBM Storage Ceph are viewed on the dashboard. However, enabling the monitoring stack is done through the command-line interface. The monitoring stack includes Prometheus, Alertmanager, and Grafana. You can use the Prometheus and Alertmanager API to manage alerts and silences.

Prerequisites

  • A running IBM Storage Ceph cluster.

  • Root-level access to all the hosts.

Procedure

  1. Log into the cephadm shell:

    Example

    [root@host01 ~]# cephadm shell
  2. Set the APIs for the monitoring stack:

    • Specify the host and port of the Alertmanager server:

      Syntax

      ceph dashboard set-alertmanager-api-host 'ALERTMANAGER_API_HOST:PORT'

      Example

      [ceph: root@host01 /]# ceph dashboard set-alertmanager-api-host 'http://10.0.0.101:9093'
      Option ALERTMANAGER_API_HOST updated
    • To see the configured alerts, configure the URL to the Prometheus API. Using this API, the Ceph Dashboard UI verifies that a new silence matches a corresponding alert.

      Syntax

      ceph dashboard set-prometheus-api-host 'PROMETHEUS_API_HOST:PORT'

      Example

      [ceph: root@host01 /]# ceph dashboard set-prometheus-api-host 'http://10.0.0.101:9095'
      Option PROMETHEUS_API_HOST updated

      After setting up the hosts, refresh your browser’s dashboard window.

    • Specify the host and port of the Grafana server:

      Syntax

      ceph dashboard set-grafana-api-url 'GRAFANA_API_URL:PORT'

      Example

      [ceph: root@host01 /]# ceph dashboard set-grafana-api-url 'http://10.0.0.101:3000'
      Option GRAFANA_API_URL updated
  3. Get the Prometheus, Alertmanager, and Grafana API host details:

    Example

    [ceph: root@host01 /]# ceph dashboard get-alertmanager-api-host
    http://10.0.0.101:9093
    [ceph: root@host01 /]# ceph dashboard get-prometheus-api-host
    http://10.0.0.101:9095
    [ceph: root@host01 /]# ceph dashboard get-grafana-api-url
    http://10.0.0.101:3000
  4. Optional: If you are using a self-signed certificate in your Prometheus, Alertmanager, or Grafana setup, disable the certificate verification in the dashboard This avoids refused connections caused by certificates signed by an unknown Certificate Authority (CA) or that do not match the hostname.

    • For Prometheus:

      Example

      [ceph: root@host01 /]# ceph dashboard set-prometheus-api-ssl-verify False
    • For Alertmanager:

      Example

      [ceph: root@host01 /]# ceph dashboard set-alertmanager-api-ssl-verify False
    • For Grafana:

      Example

      [ceph: root@host01 /]# ceph dashboard set-grafana-api-ssl-verify False
  5. Get the details of the self-signed certificate verification setting for Prometheus, Alertmanager, and Grafana:

    Example

    [ceph: root@host01 /]# ceph dashboard get-prometheus-api-ssl-verify
    [ceph: root@host01 /]# ceph dashboard get-alertmanager-api-ssl-verify
    [ceph: root@host01 /]# ceph dashboard get-grafana-api-ssl-verify
  6. Optional: If the dashboard does not reflect the changes, you have to disable and then enable the dashboard:

    Example

    [ceph: root@host01 /]# ceph mgr module disable dashboard
    [ceph: root@host01 /]# ceph mgr module enable dashboard