IBM Support

Mustgather: Collecting data to diagnose issues with Enterprise Records Container

Troubleshooting


Problem

This document describes the general information and diagnostic data needed to start troubleshooting issues related to IBM Enterprise Record container components. This technote provides a description of the background information IBM® Support will need when reporting problems related to IBM Enterprise Records.

Resolving The Problem

Collect the following troubleshooting data when contacting support for assistance with a IBM Enterprise Record container issue.


Overview of IBM Enterprise Record container diagnostic information

General diagnostic information


As needed diagnostic information

Enterprise Records Tooling diagnostic information
 

Detailed diagnostic collection steps

These are the detailed steps to gather different types of data for IBM Enterprise Record container environments.  When running diagnostic commands, run them from an empty collection directory to make it easy to package the files. Run the commands from the project or namespace containing the Enterprise Record container or use the -n <namespace> flag with all kubectl commands.

1: Provide a detailed description of the problem and your environment
  •  Provided a detailed description of your issue. Include screenshots and re-create steps if possible.
    Is it an intermittent or recreatable issue? Has this always been an issue or an issue that started only after a change occurred?
    What is the business impact? Should we be aware of any deadlines impacted by the issue?
  • Provide a reference to the documentation being followed for the failing operation
  • For new deployments or upgrades, were images downloaded from Entitled Registry or Passport Advantage?
  • Is this a development, test or production environment?
  • Which platform setup are you using (OpenShift, OpenShift on IBM Cloud Public, other Kubernetes platform)?
  • What is the database type and version?
 
2: Gather configuration information:
Provide the following logs and configuration data to assist with resolving problems:
  •  If using OpenShift, provide the output of this command:
    oc version > version.txt
  • Provide the output of this command:
    kubectl version >> version.txt
  • Provide the Custom Resource(CR) .yaml file used by the operator to configure the environment
    kubectl get icp4acluster -o yaml > config.yaml
  • Collection information about the nodes.
    kubectl get nodes -o wide > nodes.txt
  • Collect information about the pod statuses
    kubectl get pods > pods.txt
  • Collect information about the pod containers
    kubectl get pods -o jsonpath="{..image}" > containerInfo.txt
  • On OpenShift gather route configuration
    oc get route > routes.txt
    Note: If needed, more detailed route config info can be gotten with -o yaml option
  • Collect the defined secrets
    kubectl get secrets > secrets.txt
  • Collect the defined persistent volume claims
    kubectl get pvc > pvcs.txt
  • Collect the description and log of any pod you are having issues with:
    kubectl describe pod <pod-name> > describe-<podname>.txt
    kubectl logs <podname> > log-<podname>.log
3: Collect Enterprise Records log files
Collect the following logs for the IBM Enterprise Record (IER) container located in the <IER Persistent Volume>. The Persistent Volume (PV) path can be found by reviewing the persistent volume claim info you have gathered.
  • messages.log

4: Collect Operator logs

If you are having issues during the deployment by the operator then collect the operator logs described in the installation troubleshooting page.
5: CPE logs
If the issue suggests a problem with the content platform engine(CPE), then provide the CPE logs.
See the FileNet Content Manager mustgather for more details.
If the issue suggests a problem with in Business Automation Navigator, then provide the navigator logs.
See the Business Automation Navigator mustgather for more details.
Enterprise Records Tools
In addition to the must gather items for the general product, the following information should be gathered when using an IBM Enterprise Records tool.

1: IBM Enterprise Records Configuration Manager

  • IBM Enterprise Records Configuration Manager logs.
  • Go to the <IER_install_location>/configure/configmgr.ini file, located in the same directory as <IER_install_location>/configure/configmgr.exe.
    Add the parameter ”-debug” above the “-vmargs” parameter before running configuration manager.
    Run Configuration Manager and reproduce the problem.
    Shut down Configuration Manager before collecting the log file.
  • What profile are you running?
  • Which task operation are you running?
  • Provide information about Content Engine version and Operating System information.
  • Provide any error messages shown on the UI or command-line console. Please include the entire error to the degree possible.
  • Provide the following files:
  •  If Linux: runtime packages installed on Linux.
     If Command Line: IER Configuration Manager command-line version and task files found under the <IER_install_location>/configure/profiles folder.

2: Basic Disposition Sweep

You can configure tracing for IBM Enterprise Records basic disposition sweep by using the log4j configuration sample file that is provided under the IBM Enterprise Records installation directory. For more information, see Configure tracing for IBM FileNet Content Engine using log4j.

This log4j.xml file is available at IBM/EnterpriseRecords/Workflow/configureRMworkflow/lib.

Use the basic sweep command-line utility to configure disposition settings and to run disposition sweep on categories defined in a basic schedule file plan
3: File Plan Import/Export
  • Verify Content Engine Client version matches the Content Engine to which it is connecting. What is the Content Engine version?
  • What operation are you running with the tool (configure, import, export or update)?
  • Add a new JVA_ARGS call as follows to your file – “set JVA_ARGS=%JVA_ARGS% -DtraceSettingsFile=TraceSettings.properties”
    Create a new file named TraceSettings.properties with in the same directory as FilePlanTool.jar. Insert the below content:
    traceFileName=<output file path and name>.log
    com.ibm.ejs.ras.*=all=enabled
    SASRas=all=enabled
    com.ibm.*=all=enabled
    *=all=enabled
Provide the following files:
  • FilePlanTool.bat/.sh
  • FilePlanConfiguration.xml – located in the lib/config directory
  • FilePlanToolActivity.log – same directory as .bat/.sh
  • FilePlanToolException.log – same directory as .bat/.sh
  • File Plan XML file (import and update only)
  • Log file generated – FPTWASTrace.log in the above code
  • (WebSphere only) sas.client.props. By default, this file is located in the profile_root/properties directory of your WebSphere Application Server installation
4: RM Sweep
  • Verify Content Engine Client version matches the Content Engine to which it is connecting. What is the Content Engine version?
  • Verify Process Engine Client version matches the Process Engine to which it is connecting. What is the Process Engine version?
  • Which protocol is used (http, https)?
  • Which operation are you running (configure, Disposition Sweep, Hold Sweep or Autodelete)?
  • (Websphere login and connection issues) Provide the application server logs associated with WebSphere. You will need to modify your .bat or .sh file to enable the correct tracing information.

What to do next
  1. Review the log files and traces at the time of the problem to try to determine the source of the problem.
  2. Once you completed gathering all the needed info and diagnostics, you can add them to your case. Alternatively, you can upload files to ECURep. For more information, see Enhanced Customer Data Repository (ECuRep) - Overview.
 

Document Location

Worldwide

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSNVVQ","label":"IBM Enterprise Records"},"ARM Category":[{"code":"a8m50000000Kz3fAAC","label":"IBM Enterprise Records"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"5.2.1","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
22 July 2020

UID

ibm16242974