IBM Security QRadar SOAR App Host: App Host is paired but not running

Troubleshooting

Problem

Customer set the proxy configuration at App Host and is getting the following error on App Host:

"The App Host is paired but communication has not yet been successful."

Symptom

The appHostCoreDNS.txt shows:

  Warning  FailedCreatePodSandBox  30m (x922 over 3h50m)  kubelet  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox image "rancher/mirrored-pause:3.6": failed to pull image "rancher/mirrored-pause:3.6": failed to pull and unpack image "docker.io/rancher/mirrored-pause:3.6": failed to resolve reference "docker.io/rancher/mirrored-pause:3.6": failed to do request: Head "https://registry-1.docker.io/v2/rancher/mirrored-pause/manifests/3.6": proxyconnect tcp: EOF
  Warning  FailedCreatePodSandBox  4m59s (x116 over 30m)  kubelet  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox image "rancher/mirrored-pause:3.6": failed to pull image "rancher/mirrored-pause:3.6": failed to pull and unpack image "docker.io/rancher/mirrored-pause:3.6": failed to resolve reference "docker.io/rancher/mirrored-pause:3.6": failed to do request: Head "https://registry-1.docker.io/v2/rancher/mirrored-pause/manifests/3.6": proxyconnect tcp: EOF
  Warning  FailedCreatePodSandBox  4s (x2 over 17s)       kubelet  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox image "rancher/mirrored-pause:3.6": failed to pull image "rancher/mirrored-pause:3.6": failed to pull and unpack image "docker.io/rancher/mirrored-pause:3.6": failed to resolve reference "docker.io/rancher/mirrored-pause:3.6": failed to do request: Head "https://registry-1.docker.io/v2/rancher/mirrored-pause/manifests/3.6": proxyconnect tcp: EOF

Cause

The App host can't reach out to the IBM official repository due to a local network issue.

Environment

Any versions

Diagnosing The Problem

Follow MustGather to collect App host logs and status.

Check the proxy settings via the commands:

manageAppHost showconfig

And

manageAppHost proxy --environment

It shows the following that is wrong

HTTP_PROXY=http://proxy.xx.com:8080
HTTPS_PROXY=https://proxy.xx.com:8080

Resolving The Problem

The proxy setting needs to be:

HTTP_PROXY=http://proxy.xx.com:8080
HTTPS_PROXY=http://proxy.xx.com:8080

The steps are:

1. Clear the current wrong proxy setting:

manageAppHost proxy --clear

2. Reconfigure with the right settings in /etc/systemd/system/k3s.service.env file:

Here is an example:

HTTP_PROXY=http://proxy.xxx.com:8080
HTTPS_PROXY=http://proxy.xxx.com:8080
NO_PROXY=localhost,127.0.0.1,0.0.0.0,localaddress,10.0.0.0/8

3. Only applies if proxy server requires authentication. Update the system's cacerts to trust the CA certificate from the proxy itself by adding the certificate in PEM encoding to /etc/pki/ca-trust/source/anchors then run the following command:

sudo update-ca-trust

4. Restart the Kubernetes K3s.

sudo systemctl restart k3s

5. List all system containers to verify:

sudo kubectl get pods -n kube-system

Related Information

Configure proxy authentication

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEGM63","label":"IBM Security QRadar SOAR on Cloud"},"ARM Category":[{"code":"a8m0z0000001jTpAAI","label":"Integrations-\u003EAppHost"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Tips