IBM Support

Mustgather: Collecting data to diagnose issues with Business Automation Application

Product Documentation


Abstract

This document describes the general information and diagnostic data needed to start troubleshooting issues related to Business Automation Application, which is included in IBM Cloud Pak for Automation. Include the diagnostics retrieved from using this document when opening a case for Business Automation Application or Cloud Pak for Automation.

Content

Overview of Business Automation Application diagnostic information

General diagnostic information

As needed diagnostic information

Detailed diagnostic collection steps

These are the detailed steps to gather different types of data for Business Automation Application. When running diagnostic commands, run them from an empty collection directory to make it easy to package the files. Run the commands from the project or namespace containing Business Automation Application or use the -n flag with all kubectl commands.

1: Provide a detailed description of the problem and your environment

  • Provided a detailed description of your issue. Include screen captures and re-create steps if possible.
  • Is it an intermittent or re-creatable issue? Has this always been an issue or an issue that started only after a change occurred?
  • What is the business impact? Should we be aware of any deadlines impacted by the issue?
  • Provide a reference to the documentation being followed for the failing operation
  • Which platform are you using (OpenShift, managed OpenShift,  other Kubernetes platform)?

2: Gather the following configuration information

  •  If using OpenShift, provide the output of this command: 
    oc version > version.txt
  • Provide the output of this command: 
    kubectl version >> version.txt
    Note: Running the two commands above in order will produce a single versions.txt file containing both versions
  • Provide the Business Automation Application config maps: 
    kubectl get $(kubectl get configmap -o name | grep "\-ae\-") -o yaml > AEConfigMaps.yaml
  • Provide the Custom Resource(CR) .yaml file used by the operator to set up the environment 
    kubectl get icp4acluster -o yaml > config.yaml
  • Collection information about the nodes. 
    kubectl get nodes -o wide > nodes.txt
  • Collect information about the pod statuses 
    kubectl get pods > pods.txt
  • Collect information about the pod containers
    kubectl get pods -o jsonpath="{..image}" > containerInfo.txt
  • On OpenShift gather route configuration 
    kubectl get route > routes.txt
    Note: If needed, more detailed route config info can be gotten with -o yaml option
  • Collect the defined secrets 
    kubectl get secrets > secrets.txt
  • Collect the defined persistent volume claims 
    kubectl get pvc > pvcs.txt
  • Collect the description and log of any pod you are having issues with: 
    kubectl describe pod <pod-name> > describe-<podname>.txt
kubectl logs <podname> > log-<podname>.log
  • Copy the version.txt to your current directory of the host machine 
    kubectl cp <name_of_ae_running_pod>:/usr/home/solution-server/version.txt ./
  • Copy the build.txt to your current directory of the host machine 
    kubectl cp <name_of_ae_running_pod>:/usr/home/solution-server/build.txt ./
  • Obtain the Business Automation Application truststore certificate content Copy the truststore certificates to your current directory of the host machine 
    kubectl cp <name_of_AE_running_pod>:/shared/tls/truststore/pem/trusts.pem ./

3: Logs and Trace

Follow these steps to enable tracing for Business Automation Application
  1. Edit the CR .yaml file used by the operator to create the Business Automation Application pods. You can use the file gathered earlier in this document.
    Modify the following properties in the Business Automation Application log_level section of the yaml file and set the following trace string. 
    spec:
  ...
  application_engine_configuration:
    ...
    log_level:
      node: trace
      browser: 4

  2. Update the operator with the new configuration.

    kubectl apply -f my_icp4a_cr.yaml --overwrite=true

    Note: It can take the operator up to 30 minutes to recognize the change and update the configuration.
    You can check the logs to see the update take effect using:

    oc logs <ae-pod-name>
  3. Collect the trace.  In addition to using the oc logs command, if persistent storage has been setup (available starting in version 21.0.1), you may obtain the logs and trace from the persistent volume specified in the application_engine_configuration[*].log_storage.pvc_name property of your configuration, under the AE directory.  Please see IBM Business Automation Application Engine parameters for more details.
    The trace may be collected by copying using the oc command as well as long as the pod is available (where <pod-name> refers to any pod with persistent storage enabled): 
    oc cp <pod-name>:/logs/application/AE ./
  4. Disable the trace by setting the values back to their original settings and applying the operator changes again.

4:  Navigator issues

  • If using Navigator to manage and run the application
    • Provide the Navigator version
    • Collect Navigator log. Please see Navigator Log Files related link for details
  • Enable trace for Solution Discovery Plugin (trace required if you are using Navigator to manage and run the Application)
    1. Login to Navigator Admin Desktop
    2. Click “Settings” on the left-hand panel, then select the “Logging” tab
    3. Under the “Debug” section, make sure “All classes” is selected for the “Include” section and “None” is selected for the “Exclude” section

5: Collect Browser data

If experiencing client-side (browser) related issues:

6: Collect Operator Logs

If you are having issues during the deployment by the operator then collect the operator logs described in the installation troubleshooting page.

What to do next

  1. Review the log files and traces at the time of the problem to try to determine the source of the problem.
  2. Check these locations for known issues:
  3. Once you completed gathering all the needed info and diagnostics, you can add them to your case. Alternatively, you can upload files to ECURep. For more information, see Enhanced Customer Data Repository (ECuRep) - Overview.

Document Location

Worldwide

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SS7JTW","label":"IBM Digital Business Automation"},"Component":"Application Engine","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Product Synonym

App Engine

Document Information

Modified date:
16 April 2021

UID

ibm11103553

Page Feedback