Troubleshooting
Problem
EDR linux agent 0.80.1 fails to start on some endpoints due to eBPF probe loading issue.
Symptom
The log traces will show a long eBPF probe dump similar to following:
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: from 1992 to 1993: R0=inv(id=0,umin_value=18446744071562067968,var_off=(0xffffffff80000000; 0x7fffffff)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=197,imm=0) R8=inv(id=0,umin_value=77826,umax_value=4295098342,var_off=(0x0; 0x1ffffffff)) R9=inv(id=0,umin_value=18446744071562067968,var_off=(0xffffffff80000000; 0x7fffffff)) R10=fp0,call_-1 fp-88=map_value fp-96=map_value fp-104=map_value fp-112=ctx
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: 1993: (bf) r9 = r0
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: 1994: (67) r9 <<= 32
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: 1995: (c7) r9 s>>= 32
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: 1996: (b7) r1 = 2
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: 1997: (6d) if r1 s> r9 goto pc-1526
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: R0=inv(id=0,umin_value=18446744071562067968,var_off=(0xffffffff80000000; 0x7fffffff)) R1=inv2 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=197,imm=0) R8=inv(id=0,umin_value=77826,umax_value=4295098342,var_off=(0x0; 0x1ffffffff)) R9=inv(id=0,umin_value=18446744071562067968,var_off=(0xffffffff80000000; 0x7fffffff)) R10=fp0,call_-1 fp-88=map_value fp-96=map_value fp-104=map_value fp-112=ctx
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: 1998: (bf) r2 = r8
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: 1999: (07) r2 += -1
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: 2000: (57) r2 &= 131071
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: 2001: (79) r1 = *(u64 *)(r10 -88)
Apr 10 15:33:36 debian10.localdomain keeperx-loader.sh[1736]: 2002: (0f) r1 += r2
Cause
This issue is caused by the Falco eBPF probe used by QRadar EDR Linux Agent v0.80.1, which has a known issue with Debian 10 kernels.
Environment
- QRadar EDR Linux Agent 0.80.1
- Issue is currently reported in Debian kernel 4.19.0-26 and Ubuntu 5.4.0-1106 kernel.
Diagnosing The Problem
Verify Kernel version using following command:
uname -a
Resolving The Problem
This issue is planned to be fixed in future releases. Temporary workaround is provided below. There are 2 possible scenarios for this workaround:
- Scenario 1: New Installation
-
Install prerequisite packages as described in our installation document.
-
sudo apt-get install --no-install-recommends curl dkms gcc linux-headers-$(uname -r) make
-
-
Use the following command to force usage of Falco kernel module (skipping eBPF probe):
-
sudo sh -c "echo FORCE_KMOD=1 >> /etc/reaqtahive.d/keeperx.env"
-
-
Use the following command to load unsigned module ignoring kernel taint state:
-
sudo sh -c "echo KMOD_IGNORE_TAINT=1 >> /etc/reaqtahive.d/keeperx.env"
-
- Restart the agent service:
-
sudo systemctl reset-failed keeperx
-
sudo systemctl restart keeperx
-
-
- Scenario 2: Upgrade from previous version of QRadar EDR agent.
- Install dkms as additional pre-requisite package:
-
sudo apt-get install --no-install-recommends dkms
-
-
Use the following command to force usage of Falco kernel module (skipping eBPF probe):
-
sudo sh -c "echo FORCE_KMOD=1 >> /etc/reaqtahive.d/keeperx.env"
-
- Assuming KMOD_IGNORE_TAINT flag is already set for the installed agent, proceed with upgrade command:
-
sudo dpkg -i hive-installer-0.80.1-x86_64.deb
-
For more information on agent upgrade refer this.
-
- Install dkms as additional pre-requisite package:
Related Information
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSOO77","label":"IBM Security QRadar EDR"},"ARM Category":[{"code":"a8m3p000000PCPsAAO","label":"Support"},{"code":"a8m3p0000000rbnAAA","label":"Support-\u003EAdministration Task"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
30 April 2024
UID
ibm17148175