platform-auth-service pod restarts multiple times

The platform-auth-service pod restarts multiple times.

Symptoms

The platform-auth-service pod restarts multiple times. The platform-auth-service shows the following logs:

oc logs platform-auth-service -n <your-foundational-services-namespace> -c platform-auth-service
2019-02-12 18:18:12,170 WARN received SIGTERM indicating exit request
2019-02-12 18:18:12,170 INFO waiting for 01-wlp-platform-auth-service, 02-directory-service to die
2019-02-12 18:18:12,183 INFO stopped: 02-directory-service (terminated by SIGTERM)
[AUDIT   ] CWWKE0085I: The server defaultServer is stopping because the JVM is exiting.
[AUDIT   ] CWWKE1100I: Waiting for up to 30 seconds for the server to quiesce.
[AUDIT   ] CWWKT0017I: Web application removed (default_host): http://platform-auth-service-v9h8q:9080/oidc/
[AUDIT   ] CWWKT0017I: Web application removed (default_host): http://platform-auth-service-v9h8q:9080/oauth2/
2019-02-12 18:18:15,385 INFO waiting for 01-wlp-platform-auth-service to die
2019-02-12 18:18:18,387 INFO waiting for 01-wlp-platform-auth-service to die
2019-02-12 18:18:21,390 INFO waiting for 01-wlp-platform-auth-service to die
2019-02-12 18:18:22,391 WARN killing '01-wlp-platform-auth-service' (9) with SIGKILL
2019-02-12 18:18:22,514 INFO stopped: 01-wlp-platform-auth-service (terminated by SIGKILL)

Causes

The platform-auth-service is using more resources than the configured limit. This issue is seen more often on Linux on IBM Z and LinuxONE platforms.

Resolving the problem

To resolve the issue, remove the resource limits.

  1. Install the Kubernetes CLI (kubectl).

  2. Edit the platform-auth-service daemonset.

    kubectl edit ds platform-auth-service -n <your-foundational-services-namespace>
    
  3. Locate the following section:

         name: platform-auth-service
         ports:
         - containerPort: 8443
           hostPort: 8443
           name: http
           protocol: TCP
         readinessProbe:
           failureThreshold: 3
           httpGet:
             path: /
             port: 8443
             scheme: HTTPS
           periodSeconds: 10
           successThreshold: 1
           timeoutSeconds: 1
         resources:
           limits:
             cpu: "1"
             memory: 1Gi
           requests:
             cpu: 100m
             memory: 256Mi
    
  4. Remove the limits section under the resources section. The resources section must look like the following piece of code after the change:

           resources:
             requests:
               cpu: 100m
               memory: 256Mi
    
  5. Save the file and wait until all the platform-auth-service pods restart. The pods might take a few minutes to restart.