Mustgather: How to collect and review the ACTIVATIONHEAPS macro output from system service tools

Resolving The Problem

1. STRSST <enter>
2. Specify credentials <enter>
3. 1. Start a service tool <enter>
4. 4. Display/Alter/Dump <enter>
5. 2. Dump to printer  <enter>
6. 2. Licensed Internal Code (LIC) data  <enter>
7. 14. Advanced analysis  <enter>
8. Specify an option of 1 and the command ACTIVATIONHEAPS <enter>
9. Specify options of -s <enter> 
10. Specify a dump title <enter>
11. Exit system service tools by pressing F3 <exit> and watch for one of these messages:  
    If the following message appears, then take option 7. Display dump status  and wait until the dump completes.
    Dumps still incomplete.  Press F3=Exit or F12=Cancel to end.
    
    If the following message appears, then continue to press F3 <exit> and <enter> to exit System Service Tools
    Dump completed normally          -SAMPLE 1

1. WRKJOB OPTION(*SPLF)  <enter> 
2. Locate the spooled file named QPCSMPRT and take option 5=Display <enter>

• The output lists the heaps and the owning jobs in descending order by CUR SIZE (current size).
• A typical failure when a specific heap fills is MCH6903 in the given job owning the heap.  Restarting the job is typically all that is required to recover. 
• This example shows two heaps, the first one owned by job 178213/QTMHHTTP/ADMIN  and the second one owned by job 178442/QTMHHTTP/WQLIB85
• The first heap owned by job 178213 has a current size of x'210' out of the maximum size of x'FFFFF'.  The second heap owned by job 178442 has a current size of x'210' out of maximum size of x'FFFFF' 
• The CUR SIZE (current size) should typically reach some peak value and remain.  If the CUR SIZE continues to increase until the job eventually fails with an error, like MCH6903 F/#hmalcmi , then this implies a leak.
• Multiple ACTIVATIONHEAPS -s can be taken over time to determine if a job's activation group heap continues to grow. 
• 'C3359B519A' and 'D26D3D336F' are examples of the address that uniquely identify the heap.  Some advanced PEX traces use a filter and this value will be required to uniquely identify the heap.    
• 'Kernal Heap' is a special type that would be pursued using the SAGHEAPINFO macro instead of the ACTIVATIONHEAPS macro.