Monitoring resources not cleaned up

IBM Cloud Pak foundational services Prometheus as a data source of Grafana is removed from IBM Cloud Pak foundational services version 3.8.x and later.

Up to foundational services version 3.7.x, the monitoring service supported two modes. For more information, see Red Hat OpenShift Container Platform monitoring mode and IBM Cloud Pak foundational services monitoring mode.

From IBM Cloud Pak foundational services version 3.8.x onwards, IBM Cloud Pak foundational services monitoring mode is removed.

The monitoring resources are automatically removed from your cluster. However, some resources might remain in your cluster.

Symptom

Some of the ibm-monitoring-prometheus-operator and ibm-monitoring-prometheus-operator-ext operator resources are not properly cleaned up.

Resolving the problem

Following instructions are for manually removing the resources. Complete these steps only if you see foundational services Prometheus-stack-related pods in version 3.8.x or later.

1. Delete the ClusterServiceVersions (CSVs) that are installed in your cluster.

   1. Get a list of the CSVs.

      oc get csv -n ibm-common-services
      

   2. If you see the ibm-monitoring-exporters-operator or ibm-monitoring-prometheus-operator CSV, delete it.

      oc delete csv ibm-monitoring-exporters-operator.v1.x.x -n ibm-common-services
      

      oc delete csv ibm-monitoring-prometheus-operator-ext.v1.x.x -n ibm-common-services
      

2. Delete the operandrequests that are installed in your cluster.

   1. Get a list of the operandrequests.

      oc get operandrequest -n ibm-common-services
      

   2. If you see the monitoring-exporters-operator-request or monitoring-prometheus-ext-operator-request operandrequest, delete it.

      oc delete operandrequest monitoring-exporters-operator-request -n ibm-common-services
      

      oc delete operandrequest monitoring-prometheus-ext-operator-request -n ibm-common-services
      

3. Delete the daemonsets that are installed in your cluster.

   1. Get a list of the daemonsets.

      oc get ds -n ibm-common-services
      

   2. If you see the ibm-monitoring-nodeexporter daemonset, delete it.

      oc delete ds ibm-monitoring-nodeexporter -n ibm-common-services
      

4. Delete the statefulsets that are installed in your cluster.

   1. Get a list of the statefulsets.

      oc get statefulset -n ibm-common-services
      

   2. If you see the prometheus-ibm-monitoring-prometheus or alertmanager-ibm-monitoring-alertmanager statefulset, delete it.

      oc delete statefuleset prometheus-ibm-monitoring-prometheus -n ibm-common-services
      

      oc delete statefuleset alertmanager-ibm-monitoring-alertmanager -n ibm-common-services
      

5. Delete the deployments that are installed in your cluster.

   1. Get a list of the deployments.

      oc get deployment -n ibm-common-services
      

   2. If you see the ibm-monitoring-prometheus-operator, ibm-monitoring-collectd, ibm-monitoring-kube-state, or ibm-monitoring-mcm-ctl deployment, delete it.

      oc delete deployment ibm-monitoring-prometheus-operator -n ibm-common-services
      

      oc delete deployment ibm-monitoring-collectd -n ibm-common-services
      

      oc delete deployment ibm-monitoring-kube-state -n ibm-common-services
      

      oc delete deployment ibm-monitoring-mcm-ctl -n ibm-common-services