IBM DOORS Next Generation Performance MustGather

Steps

1. Detailed Description: We need a good Detailed description of the  performance issue including the specific functionality that is affected, Statements like "DNG is unusable" or "Nothing works" is not enough information, we need to see examples of the issues including Detailed descriptions, screen prints or MP4s showing the problem.
   
    
2. Environment Details:  If not already known, obtain the Environment details:
   • Is it a Standalone or Distributed environment
   • Is it hosted on a Vapp?
   • Is the Environment Windows or Linux?
   • What Webserver is used Tomcat, Liberty or Websphere (Include Version)?
   • Include the DB vendor and version.
   • Confirm when the last DBStats was run on the Database.
   • Number of CPUs?
   • How Much RAM is allocated to RM server?
   • What is the JVM heap set for RM server?
   • Is Configuration Management enabled?
   • Approx Number of Project areas and Number of artifacts per project?
   • What is the Size of RM indices on disk?
     
      

1. Gather Browser Logging (HAR file)
   •     a. In Firefox navigate to the "Open Menu" (Hamburger icon on the top right hand side of the browser)
   •     b. Click On the Developer Option.
   •     c. Click on the Network Option, a separate window will open.

   •     d. Enter “?debug=true“ after web in the following URL:https://<server>:<port>rm/web?debug=true replacing <server> and <port> with your server and port details.

   •     e. Click on the Clear option.

   •     f. Perform the steps that result in the performance issue
   •     g. From the Network window right click and select "Save all As Har"
   •     h. Send on the file to the Case for review.
      
2. Active Services page:

   •    a. Navigate to https://<JazzServer>/rm/admin#action=com.ibm.team.repository.admin.activeServices

   •    b. Take a screen print and send it to us for review.
      
3. Run the Performance Health Check Widget:
   •    a. Follow the steps here: Jazz.Net Wiki: Performance health check widget   
   •    b. Send on a screen print when complete
4. Gather Foundation Counters:

   •     a. Navigate to https://<JazzServer>/rm/service/com.ibm.team.repository.service.internal.counters.ICounterContentService/

   •     b. Save and send on the results
      
5. Gather Additional RM counters:

   •    a. Enable JVM param: -Dcom.ibm.team.repository.service.internal.vvc.enableCounters=true and restart (Enable Counters service only needs to be done once)

   •    b. Run through steps which exhibit slow performance (Module View Hang Etc)

   •    c. Navigate to web page https://[JazzServer]/rm/service/com.ibm.team.repository.service.internal.counters.ICounterContentService/

   •    d. When it completes loading right click save as HTML or take a screenshot of the entire page.
      
6. Gather Java Cores:

   •    a. Navigate to https://<JazzServer>/rm/admin#action=com.ibm.rdm.fronting.server.web.logging

   •    b. Scroll to the end and Trigger an IBM JVM JavaDump (run this 5 times, 1 minute apart)
   •    c. Provide the javacore.YYYYMMDD.HHMMSS.XXXXX.XXXX.txt files for review
        or use the following process 
        e. Generating a core file and preparation of core file for analysis on Linux systems
      
7. Gather Query Stats:

   •     a. Go to rm/admin > Advanced Properties, set  "Enable repodebug service" to 'true' and then Save.

   •     b. Open https://<JazzServer>/rm/repodebug/database/queryStats and click on stop, then on reset, then on start

   •     c. Allow this to run for 15-20 minutes while reproducing the slow performing scenario(s) in a separate tab

   •     d. On the /rm/repodebug/database/queryStats tab, click on Stop. The page should refresh and show tables with SQL query statistics. Save the entire page and send to the case.

8. Gather the Verbose GC logs:
   •      a. See  Enabling verbose garbage collection (verboseGC) in WebSphere Application Server 

   •      b. For Liberty its automatically enables and the files are located here:  ....server/liberty/servers/clm/

9. Gather Full set of DNG ISADC Logs:
   •     a. See  Collecting diagnostics with IBM Support Assistant Data Collector
   •     b. Upload the resulting zip file to the Case