Configuring Kubernetes Observer jobs

Enabling access to the URL routes

To access the URL routes for the topology Swagger documentation, see the Enabling access to URL routes topic.

Kubernetes Observer Load job prerequisites

Before defining a Kubernetes Observer Load job, you must create a service account in the Kubernetes environment and obtain its token. The following example uses the default namespace.

1. Create either cluster role access or role access.

   For clusterole access, create a configuration file called asm-k8s-observer.yaml with the custom cluster role asm:kubernetes-observer.

   Use the following sample content:

   kind: ClusterRole
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     annotations:
       rbac.authorization.kubernetes.io/autoupdate: "true"
     name: asm:kubernetes-observer
   rules:
   - apiGroups: [""]
     resources: [ "pods", "namespaces", "nodes", "services", "endpoints", "persistentvolumes", "persistentvolumeclaims"]
     verbs: ["get", "list"]    
   - apiGroups: ["apps"]
     resources: ["replicasets", "deployments", "statefulsets", "daemonsets"]
     verbs: ["get", "list"]
   

   For role access:

   Use the following sample content:

   kind: Role
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     namespace: {namespace} <-- Specify which namespace to create the Role access
     name: asm:kubernetes-observer
   rules:
   - apiGroups: [""]
     resources: [ "pods", "services", "endpoints"]
     verbs: ["get", "list"]
   - apiGroups: ["apps"]
     resources: ["replicasets", "deployments", "statefulsets", "daemonsets"]
     verbs: ["get", "list"]     
   

   Observing Custom Resources:

   To model custom resource definitions (CRDs) and custom resources (CRs) in the topology, identify the names of the CRDs that are present in the environment by running the oc get crds command. Then, update the job parameters to include the names of CRDs to be discovered. When a CRD is being included in the job, the respective CRs will also be discovered.

   Example (discovering ibm.com CRDs):

   oc get crds | grep ibm.com
   

   Sample output:

   asmformations.asm.ibm.com                                   2020-12-03T01:53:38Z
   asms.asm.ibm.com                                            2020-12-03T01:53:38Z
   cemformations.cem.ibm.com                                   2020-12-03T01:53:38Z
   cemserviceinstances.cem.ibm.com                             2020-12-03T01:53:38Z
   noiconnectionlayers.noi.ibm.com                             2020-12-03T01:47:19Z
   noiformations.noi.ibm.com                                   2020-12-03T01:47:19Z
   noihybrids.noi.ibm.com                                      2020-12-03T01:47:19Z
   nois.noi.ibm.com                                            2020-12-03T01:47:20Z
   

   Also, change the existing ClusterRole in the target environment, to include the following resources:

   - apiGroups: ["apiextensions.k8s.io"]
     resources: ["customresourcedefinitions"]
     verbs: ["get", "list"]
   - apiGroups: ["noi.ibm.com"]
     resources: ["nois", "noihybrids", "noiformations", "noiconnectionlayers"]
     verbs: ["get", "list"]
   - apiGroups: ["asm.ibm.com"]
     resources: ["asms", "asmformations"]
     verbs: ["get", "list"]
   - apiGroups: ["cem.ibm.com"]
     resources: ["cemformations"]
     verbs: ["get", "list"]
   

   The apiextensions.k8s.io segment of the cluster role provides your permission required to access all the custom resource definitions resources.

   - apiGroups: ["apiextensions.k8s.io"]
     resources: ["customresourcedefinitions"]
     verbs: ["get", "list"]
   

   The asm.ibm.com segment provides access to asms and asmformations custom resources. The names of the resources listed are the plural for each of the resources. They are usually derived from the first word of the CRD.

   - apiGroups: ["asm.ibm.com"]
     resources: ["asms", "asmformations"]
     verbs: ["get", "list"]
   

   For a fresh discovery:

   Create a configuration file called asm-k8s-observer.yaml in the target environment using the yaml resource provided in the sample content, then run the following command before moving to the next step:

   oc create -f asm-k8s-observer.yaml
   

   For an existing discovery:

   Create a configuration file called asm-k8s-observer.yaml using the yaml resource provided in the sample, then run the following command to patch the clusterole:

   oc patch clusterrole asm:kubernetes-observer --patch "$(cat asm-k8s-observer.yaml )"
   

2. Create the service account in the target environment.

   oc create serviceaccount asm-k8s-account -n <namespace>
   

   To verify that the service account exists:

   oc get serviceaccount
   

   Important: When running the following command, ensure you know the namespace in which it was created:

   oc create serviceaccount asm-k8s-account
   

3. Bind the asm:kubernetes-observer ClusterRole or role to the asm-k8s-account service account.

   Important: The service account and the cluster role binding need to be created in the same namespace.

   Example command for clusterole binding:

   oc create clusterrolebinding asm-k8s --clusterrole=asm:kubernetes-observer --serviceaccount={namespace}:asm-k8s-account
   

   Example command for role binding:

   oc create rolebinding asm-k8s --role=asm:kubernetes-observer --serviceaccount={namespace}:asm-k8s-account
   

   Example role binding output:

   rolebinding.rbac.authorization.k8s.io/asm-k8s created
   

4. Obtain the Kubernetes service account token. Complete the following substeps depending on your version of Kubernetes.

   For Kubernetes on OpenShift:

   1. Look for the mounted service account token secret from the secrets section:

      oc describe serviceaccount asm-k8s-account
      

      Example:

      oc describe serviceaccount asm-k8s-account
      Name: asm-k8s-account
      Namespace: default
      Labels: <none>
      Annotations: <none>
      Image pull secrets: asm-k8s-account-dockercfg-wwzx2
      Mountable secrets: asm-k8s-account-dockercfg-wwzx2
      Tokens: asm-k8s-account-token-tzl52
      Events: <none>      
      

   2. Describe the asm-k8s-account-token-******* (which in this example is tzl52) to obtain the token's value:

      oc describe secret asm-k8s-account-token-tzl52
      

      Alternatively, use the following method to obtain the token's value (applicable to both OpenShift and GKE environments):

      oc describe serviceaccount asm-k8s-account | grep Tokens | awk '{ print $2} ' | xargs -I{} oc get secret {} --template='{{.data.token}}' | base64 -d
      

      For Kubernetes load jobs, ensure you have the Kubernetes service details to hand, such as the Kubernetes host IP and SSL Certificate details.

   For Kubernetes on GKE:

   1. Verify that the token has been created using the kubectl describe serviceaccount <service-account> command.

      Example of a missing token:

      oc kubectl describe serviceaccount asm-k8s-account
      Name:                asm-k8s-account
      Namespace:           default
      Labels:              <none>
      Annotations:         <none>
      Image pull secrets:  <none>
      Mountable secrets:   <none>
      Tokens:              <none>
      Events:              <none>
      

   2. If missing, create the token manually.

      Example:

      kubectl apply -f - <<EOF
      apiVersion: v1
      kind: Secret
      metadata:
         name: asm-k8s-account-secret
         annotations:
            kubernetes.io/service-account.name: asm-k8s-account
      type: kubernetes.io/service-account-token
      EOF
      secret/asm-k8s-account-secret created
      

   3. Verify that the token exists.

      Example of a token that is present:

      oc kubectl describe serviceaccount asm-k8s-account
      Name:                asm-k8s-account
      Namespace:           default
      Labels:              <none>
      Annotations:         <none>
      Image pull secrets:  <none>
      Mountable secrets:   <none>
      Tokens:              asm-k8s-account-secret
      Events:              <none>
      

   4. Describe the secret to obtain the token's value.

      Example:

      oc kubectl describe secret asm-k8s-account-secret
      Name:         asm-k8s-account-secret
      Namespace:    default
      Labels:       <none>
      Annotations:  kubernetes.io/service-account.name: asm-k8s-account
                    kubernetes.io/service-account.uid: d82e86c1-2258-4072-8a53-f517f776a149
      
      Type:  kubernetes.io/service-account-token
      
      Data
      ====
      ca.crt:     1509 bytes
      namespace:  7 bytes
      token:      eyJhbGciO
      

Kubernetes Observer Listen job prerequisites

Important: Before defining a Kubernetes Observer Listen job, you must have configured and run a Load job.

Before defining a Listen job, update the ClusterRole or Role configuration.

1. Edit the asm-k8s-observer.yaml configuration file and add the following information:

   For clusterole access, add watch as in the following example.

   kind: ClusterRole
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     annotations:
       rbac.authorization.kubernetes.io/autoupdate: "true"
     name: asm:kubernetes-observer
   rules:
   - apiGroups: [""]
     resources: [ "pods", "namespaces", "nodes", "services", "endpoints", "persistentvolumes", "persistentvolumeclaims"]
     verbs: ["get", "list", "watch"]
   - apiGroups: ["apps"]
     resources: ["replicasets", "deployments", "statefulsets", "daemonsets"]
     verbs: ["get", "list", "watch"]
   

   For role access, add watch as in the following example.

   kind: Role
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     namespace: default
     name: asm:kubernetes-observer
   rules:
   - apiGroups: [""]
   resources: [ "pods", "services", "endpoints"]
   verbs: ["get", "list", "watch"]
   - apiGroups: ["apps"]
   resources: ["replicasets", "deployments", "statefulsets", "daemonsets"]
   verbs: ["get", "list", "watch"]
   

2. Run the following command:

   oc apply -f asm-k8s-observer.yaml
   

Kubernetes Observer Weave Scope listen job prerequisites

For Weave Scope listen jobs, do a Weave Scope installation, and then configure a job using the Weave Scope IP and port parameters. You must install Weave Scope and then use the Weave Scope Master IP and Node port parameters. For more information on Weave Scope, see the following location: https://www.weave.works/docs/scope/latest/introducing/

1. Create namespace 'weave' with 'ibm-privileged-psp'.

oc create namespace weave oc -n weave create rolebinding weave-clusterrole-rolebinding --clusterrole=ibm-privileged-clusterrole --group=system:serviceaccounts:weave

2. Install Weave Scope using the following command:

  ```
oc apply -f "https://cloud.weave.works/k8s/scope.yaml?k8s-service-type=NodePort&k8s-version=$(kubectl version | base64 | tr -d '\n')"

This will result in a port being opened that the observer can use.

3. You can discover the NodePort using the following command:

oc -n weave describe service weave-scope-app

4. Launch the Weave Scope UI using the following URL:

  ```
https://<master ip>:<NodePort>

5. Create a new route using the Red Hat OpenShift Container Platform console to expose the Red Hat OpenShift Container Platform Weave Scope service under the weave namespace. In the left panel of the Red Hat OpenShift Container Platform console, click Networking > Routes.

6. Click Create Route to create a new route.

7. Complete the required fields and then click Create.

8. A new route has been created. Launch the Weave Scope UI using the location link.

Kubernetes Observer kubeconfig load job prerequisites

For a kubeconfig load job, ensure your kubeconfig.yml file is in the correct format. If it is not, generate a new file.

Important: The kubeconfig should meet the following criteria:

• Single cluster
• User type of 'token'
• Base64 encoded JSON

View the kubeconfig.yml file:

oc config view --raw -o json

Important: Make sure the token and certificate are not omitted or redacted.

If it meets all required conditions, encode it using the following command:

oc config view --raw -o json | base64 -w 0

If it does not meet all conditions, generate a new kubeconfig.yml file using the following steps:

1. Create a cluster-role or role, service account, cluster-role/role binding. From the service account, get the secret, and from the secret get the token. Refer to "Kubernetes Observer Load job prerequisites" for detailed steps.

2. Get the current context details:

   oc config current-context
   

3. Get the current cluster, cluster ca (certificate-authority-data), and cluster server details:

   oc config view --raw -o json
   

4. Create a kubeconfig.yml using the obtained data. Sample kubeconfig.yml file:

   apiVersion: v1
   kind: Config
   current-context: <CURRENT_CONTEXT>
   contexts:
   - name: <CURRENT_CONTEXT>
     context:
       cluster: <CURRENT_CONTEXT>
       user: <USER>
       namespace: <NAMESPACE>
   clusters:
   - name: <CURRENT_CONTEXT>
     cluster:
       certificate-authority-data: <CLUSTER_CA>
       server: <CLUSTER_SERVER>
   users:
   - name: <USER>
     user:
       token: <TOKEN_VALUE>
   

   Where USER is the user-defined value, and NAMESPACE is the namespace of the service account.

5. Test the kubeconfig validity and access:

   oc --kubeconfig <config>.yml get all
   

6. Run the following command to convert the file to base64 encoded json:

   oc --kubeconfig <config>.yml config view --raw -o json | base64 -w 0
   

Load job parameters

Listen job parameters

Weave_scope job parameters

Kubeconfig load job parameters