Answer

Answer

You can collect My TraceRoute (MTR) data on individual endpoints. To collect MTR data, you must install and run the MTR tool on the Bare Metal Servers or Virtual Server Instances (VSI) that you want to troubleshoot.

How do you install the MTR tooling?

To install MTR or WinMTR, complete one of the following processes

• CentOS or Red Hat Linux operating systems
  From the command line, run the following command:
  yum install mtr
  
  
• Debian or Ubuntu Linux operating system
  From the command line, run the following command:
  apt-get update
  apt-get -y install mtr
  
  
• Windows Server
  Download the latest version of WinMTR from SoundForge and install the application on your server.

How do you collect bi-directional MTRs?

Use the following commands to collect the required information for IBM Cloud Support.
Note: You must run these commands on both endpoints.

• Linux Distribution (CentOS, Red Hat, Debian, and Ubuntu operating systems)
  mtr -wrc 100 x.x.x.x
  - or -
  mtr -wrc 100 x.x.x.x > filename.txt
  
  Sample Output:
  
  root@debianCJ:~# mtr -wrc 100 169.60.192.212
  Start: Tue Sep 25 12:29:35 2018
  HOST: debianCJ Loss% Snt Last Avg Best Wrst StDev
  1.|-- c9.04.7a9f.ip4.static.sl-reverse.com 99.0% 100 4.9 4.9 4.9 4.9 0.0
  2.|-- fd.28.9905.ip4.static.sl-reverse.com 0.0% 100 0.6 2.7 0.5 47.4 6.4
  3.|-- 96.9e.fd9f.ip4.static.sl-reverse.com 0.0% 100 11.0 1.1 0.5 16.2 2.1
  4.|-- ae9.bbr01.eq01.ams02.networklayer.com 42.0% 100 1.7 2.0 1.6 4.6 0.6
  5.|-- ae3.cbs02.eq01.wdc02.networklayer.com 0.0% 100 78.9 79.6 78.8 82.0 0.5
  6.|-- ae7.cbs01.eq01.wdc02.networklayer.com 26.0% 100 79.0 79.3 78.7 84.8 0.7
  7.|-- ae1.cbs01.tl01.atl01.networklayer.com 9.0% 100 92.1 92.0 91.6 96.4 0.5
  8.|-- ae0.cbs02.eq01.dal03.networklayer.com 61.0% 100 108.6 109.2 108.4 120.7 2.0
  9.|-- ae1.dar02.dal10.networklayer.com 0.0% 100 108.5 109.9 108.2 135.6 4.9
  10.|-- ae1.ppr02.dal10.networklayer.com 0.0% 100 106.4 106.8 105.9 141.7 3.7
  11.|-- 73.76.2ea9.ip4.static.sl-reverse.com 0.0% 100 106.0 106.1 105.9 107.5 0.2
  12.|-- 169.48.168.108-static.reverse.networklayer.com 0.0% 100 105.5 105.5 105.3 106.3 0.0
  13.|-- d4.c0.3ca9.ip4.static.sl-reverse.com 0.0% 100 106.1 106.1 105.9 106.5 0.≈0
  
  
• Windows Server
  You can double-click the WinMTR icon or run the command from the command line. Enter the IP address into the Host field and click Start as shown in the following screen shot:
  
  
  
  To collect an MTR for your IBM Cloud account, allow the MTR tooling to continue for at least 100 packets per hop. When the process has completed, you should see something like the following screen shot:
  
  
  
  Click Stop to end the capture. You can either copy the text to the clipboard or to a file. Attach this file or text to your ticket.
  
  

How do you interpret the MTR output?

Generally, it is recommended that you provide the MTR data to IBM Cloud Support so that we can determine the root cause of your connection issue. By bi-directionally creating an MTR, it is possible to understand the path your data is traveling and pinpoint issues, such as sub-optimal routing and packet loss. The MTR is not a definitive check for packet loss as the ICMP protocol might hash differently on certain links.

Because routers might de-prioritize ICMP (Ping) traffic, you might run into a scenario where you can see packet loss in the MTR and it is not an actual case of packet loss. Here is the sample output that was provided previously:
HOST: debianCJ Loss% Snt Last Avg Best Wrst StDev
1.|-- c9.04.7a9f.ip4.static.sl-reverse.com 99.0% 100 4.9 4.9 4.9 4.9 0.0
2.|-- fd.28.9905.ip4.static.sl-reverse.com 0.0% 100 0.6 2.7 0.5 47.4 6.4
3.|-- 96.9e.fd9f.ip4.static.sl-reverse.com 0.0% 100 11.0 1.1 0.5 16.2 2.1
4.|-- ae9.bbr01.eq01.ams02.networklayer.com 42.0% 100 1.7 2.0 1.6 4.6 0.6
5.|-- ae3.cbs02.eq01.wdc02.networklayer.com 0.0% 100 78.9 79.6 78.8 82.0 0.5
6.|-- ae7.cbs01.eq01.wdc02.networklayer.com 26.0% 100 79.0 79.3 78.7 84.8 0.7
7.|-- ae1.cbs01.tl01.atl01.networklayer.com 9.0% 100 92.1 92.0 91.6 96.4 0.5
8.|-- ae0.cbs02.eq01.dal03.networklayer.com 61.0% 100 108.6 109.2 108.4 120.7 2.0
9.|-- ae1.dar02.dal10.networklayer.com 0.0% 100 108.5 109.9 108.2 135.6 4.9
10.|-- ae1.ppr02.dal10.networklayer.com 0.0% 100 106.4 106.8 105.9 141.7 3.7
11.|-- 73.76.2ea9.ip4.static.sl-reverse.com 0.0% 100 106.0 106.1 105.9 107.5 0.2
12.|-- 169.48.168.108-static.reverse.networklayer.com 0.0% 100 105.5 105.5 105.3 106.3 0.0
13.|-- d4.c0.3ca9.ip4.static.sl-reverse.com 0.0% 100 106.1 106.1 105.9 106.5 0.≈0

In this sample output, note that multiple hops report a severe packet loss of up to 99%. However, also note that this packet loss never continues to the next hop. Support engineers will generally consider this trivial packet loss. It can be assumed that the intervening hops are due to de-prioritized ICMP because the loss does not continue to the next hop.

If real packet loss was occurring, it would look similar to the following example:

Host Loss% Snt Last Avg Best Wrst StDev
1. 117.28.242.129 0.0% 372 0.8 0.8 0.3 60.8 3.5
2. 10.1.0.53 0.0% 372 0.8 1.0 0.4 34.7 2.3
3. 222.77.78.50 0.0% 372 1.2 1.2 0.9 4.1 0.2
4. 59.43.74.61 0.0% 372 1.3 1.2 0.7 7.7 0.7
5. 59.43.116.53 0.0% 372 10.7 4.9 2.0 26.9 3.8
6. 59.43.98.209 0.0% 372 11.1 13.5 11.0 114.0 11.4
7. 59.43.247.102 9.9% 372 11.4 13.2 11.3 121.2 10.2
8. 59.43.246.54 1.3% 372 11.7 14.6 11.3 96.4 13.0
9. 59.43.183.110 3.2% 371 23.6 27.5 23.3 116.0 13.4
10. 59.43.249.5 9.2% 371 70.2 70.5 50.7 157.3 12.4
11. 203.129.8.2 72.8% 371 64.3 60.3 48.0 69.8 4.3
12. ae5.dar01.hkg02.networklayer.com 65.9% 371 65.3 68.5 59.2 113.0 5.7
13. po1.fcr01.hkg02.networklayer.com 80.3% 371 58.0 64.9 51.8 147.1 13.5
14. hk.datacenter 73.8% 371 65.5 66.1 45.8 73.5 4.6

In this example, packet loss begins at hop 203.129.8.2 and you can see it continue with each subsequent hop. This example might indicate an actual packet loss.

Note: It is important that you provide MTR data to IBM Cloud Support so that it is reviewed along with other diagnostic data. Although it is a good tool, MTR does not conclusively prove the reason for issues. MTR data is best compared with other diagnostic data to determine what might be going on in your situation.