Secret watcher pod shows CrashLoopBackOff status

The secret-watcher pod keeps crashing and shows a CrashLoopBackOff status.

Cause

There might be many Kubernetes secrets that are configured in the cluster. The current memory limit that is configured in the secret-watcher pod is not sufficient to manage the Kubernetes secrets. Kubernetes killed the pod due to less memory.

Resolving the problem

To resolve the issue, increase the resource memory limit to a value that is sufficient for the pod to manage all the Kubernetes secrets.

1. Log in to your cluster node.

2. Check the number of Kubernetes secrets that are in the cluster.

   kubectl get secrets --all-namespaces | wc -l
   

3. Edit the secret-watcher deployment.

   kubectl -n kube-system edit deployment secret-watcher
   

4. Locate the resources section. Add the required memory limit in the resources.limits.memory section.

   resources:
     limits:
       cpu: 200m
       memory: 128Mi
     requests:
       cpu: 10m
       memory: 10Mi
   

5. Save and exit the deployment.

6. Check the secret-watcher pod status. The status must show as 1/1 Running.

   kubectl -n kube-system get pods | grep secret-watcher