Installing the host agent on Linux

Checking that you have a supported Linux distribution

The one-liner supports the following Linux distributions:

• Debian derivatives (apt package manager)
• Red Hat derivatives (yum package manager)
• SUSE derivatives (zypp package manager)

Installing the agent by using the one-liner

You need root user privileges to install the Instana host agent.

To use the one-liner to install the host agent, complete the following steps:

1. Sign in to the Instana UI, and then select an option to display the agent catalog; for example, on the home page, click Deploy Agent located at the top right corner.

   If you are starting a new trial instance of Instana, the agent catalog is displayed with a prompt to select a host agent to install.

2. Click the tile Linux - Automatic Installation (One-liner)

3. Select the packaging option to install a dynamic or static agent, and the runtime options that you want to use. If you change options, then the script parameters are updated.

   The script is pre-filled with your agent key and host agent endpoint.

   For example, the UI provides the following script to install the agent with the Dynamic packaging option, and with the Azul Zulu 1.8 and Interactive runtime options:

   curl -o setup_agent.sh https://setup.instana.io/agent && chmod 700 ./setup_agent.sh && sudo ./setup_agent.sh -a <your_agent_key> -d <your_agent_key> -t dynamic -e <host-agent-endpoint>
   

4. (Optional) Select the option to install and start the host agent as a service (only supported for systemd-based systems). This option adds the -s parameter to the script.

5. Copy and then run the script to install the host agent.

   Before running the script, you can add other parameters for more options. For a description of available parameters, see One-liner script parameters.

Result: You have installed the host agent.

• If you selected the option to install and start the host agent as a service, the agent starts automatically.
• The public systems package repository of Instana is added to your system by creating a repository file. This file adds Instana authenticated repositories to the machine's installation sources.

What to do next:

Check the status of the agent in the Instana UI or on the host. For more information, see Checking the status of the host agent.

If the host agent is not started, start the host agent by running one of the following commands:

• If your operating system uses systemd as the init system (and you have not already started the agent as a service), run the systemctl start instana-agent.service command.
• If your operating system uses SysVinit as the init system, run the service instana-agent start command.

For more information about starting the agent and enabling it as a service, see Starting the agent.

If you have problems installing the agent, see Potential problems during installation.

You can perform other installation-related activities on the agent to customize the agent configuration, update the agent, uninstall the agent, and manage the agent in the Instana UI.

Potential problems during installation

  Setting up Instana APT repository
  Importing Instana GPG key
  Updating apt metadata ...
  E: The method driver /usr/lib/apt/methods/https could not be found.
  APT repository metadata update failed

  Setting up Instana agent for GNU/Linux
  Setting up Instana zypper repository
  Updating zypper metadata ...
  Installing Instana agent ...
  Error building the cache:
  [instana-agent|https://_@packages.instana.io/agent/generic/x86_64] Valid metadata not found at specified URL
  Some of the repositories have not been refreshed because of an error.
  No provider of 'instana-agent-static' found.
  Instana agent package install failed

Setting up Instana agent for GNU/Linux
Setting up Instana YUM repository
Updating YUM metadata ...
YUM repository metadata update failed

Error unpacking rpm package instana-agent-dynamic-20210630-0948.x86_64
Error: Transaction failed

sudo rpm -ivh instana-agent-dynamic-20210713-1352.x86_64.rpm
...
error: unpacking of archive failed on file /etc/init.d/instana-agent;60eda3b3: cpio: Digest mismatch
error: instana-agent-dynamic-20210713-1352.x86_64: install failed

sudo rpm -ivh --nodigest --nofiledigest instana-agent-dynamic-20210713-1352.x86_64.rpm

Downloading the agent package

To download an Instana agent package, complete the following steps:

1. Sign in to the Instana UI, and then select an option to display the agent catalog; for example, on the home page, click Deploy Agent.

   If you are starting a new trial instance of Instana, the agent catalog is displayed with a prompt to select a host agent to install.

2. Click the tile Linux - Packages(DEB, RPM).

3. To configure the agent, select one of the options in the following areas:

   • Packaging: Click Dynamic or Static.
   • Runtime: Click Azul Zulu 1.8 or Eclipse OpenJ9 11.
   • Mode: Click RPM or DEB.

4. From the Platform architecture menu, select your platform architecture.

5. Click the Download icon to download the agent package.

In an offline (air-gapped) environment that is not connected to the internet, you can download the agent package for DEB or RPM directly from the Artifactory. Make sure that you download the latest package.

Installing the agent package

Instruct your package manager to install the agent package for your Linux distribution (or install the agent package manually):

• Debian derivatives (apt package manager)
• RPM-based distributions (yum package manager)
• openSUSE/SLES (zypp package manager)

# /etc/apt/sources.list.d/instana-agent.list
deb [arch=amd64] https://packages.instana.io/agent generic main

# /etc/apt/auth.conf.d/instana-packages.conf
machine packages.instana.io
login _
password <your_agent_key>

wget -qO - https://packages.instana.io/Instana.gpg | sudo apt-key add -

[instana-agent]
name=Instana
baseurl=https://_:<your_agent_key>@packages.instana.io/agent/generic/x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.instana.io/Instana.gpg
priority=5
sslverify=1

rpm --import https://packages.instana.io/Instana.gpg

Preparing the agent

After you refresh the package manager's sources, configure the host agent endpoint and the agent key.

If the following environment variables are set before you run the installation, the agent is automatically configured.

export INSTANA_AGENT_KEY=$agentkey
export INSTANA_AGENT_HOST=$endpoint
export INSTANA_AGENT_PORT=$endpoint_port

The endpoint, port, and agent key values are shown on Agent deployment screens in the Instana UI; for example, on a Linux - Automatic Installation (One-liner) screen, the deployment code includes ... -a aGeNTKEY0vaLuO0Eu1ABc ... -e ingress-green-saas.instana.io:443.

Summary of options for setting environment variables:

• Git-based configuration management: Placing this information in a file is a good approach.
• Systemd: A drop-in is the easiest approach to overwriting environment specifics for services.
• SysVinit: Place the files in either /etc/sysconfig (Red Hat derivatives) or /etc/default (Debian derivatives).

For more information about the options for environment variables, see Setting and overriding environment variables.

To continue, install the agent on your chosen architecture.

Installing the agent

After you refresh the package manager's sources, run the command to install the agent on your chosen architecture:

Debian derivatives

• Static agent: apt-get install instana-agent-static
• Dynamic agent: apt-get install instana-agent-dynamic

Red Hat derivatives

• Static agent: yum install instana-agent-static
• Dynamic agent: yum install instana-agent-dynamic

Eclipse OpenJ9 11 bundled packages

Install the agent with one of the following package names: instana-agent-static-j9 or instana-agent-dynamic-j9.

Result: You have installed the host agent.

What to do next:

Check the status of the agent in the Instana UI or on the host. For more information, see Checking the status of the host agent.

If you have problems installing the agent, see Potential problems during installation.

You can perform other installation-related activities on the agent to customize the agent configuration, update the agent, uninstall the agent, and manage the agent in the Instana UI.

Potential problems during installation

You might meet the following problems during installation:

• Debian-based Derivatives might present a connection error message. This bug is found in GnuTLS-depending programs like Curl, git, or apt:

  GnuTLS recv error (-9): A TLS packet with unexpected length was received
  

• With Debian-based Derivatives, you might experience the following output when you issue the installation. In this case, install the package apt-transport-https because Apt doesn't currently pull fonts from HTTPS servers.

  E: The method driver /usr/lib/apt/methods/https could not be found.` `APT repository metadata update failed
  

• In case SUSE Linux Enterprise Server 12 gives you the following output, you need to update your GnuTLS, OpenSSL, and NSS libraries and its dependent Programs (for example Curl). This update is important to properly receive artifacts through modern HTTPS connections:

  Error building the cache:
  [instana-agent|https://_@packages.instana.io/agent/generic/x86_64] Valid metadata not found at specified URL
  Some of the repositories have not been refreshed because of an error.
  No provider of 'instana-agent-static' found.
  Instana agent package install failed
  

• If you try to enable the instana-agent service by using systemd, you can see the following error message in SUSE Enterprise Linux:

  systemctl enable instana-agent
  
  Synchronizing state of instana-agent.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
  Executing: /usr/lib/systemd/systemd-sysv-install enable instana-agent
  ln -sf ../instana-agent /etc/init.d/rc2.d/S50instana-agent
  ln: failed to create symbolic link '/etc/init.d/rc2.d/S50instana-agent': No such file or directory
  

  To solve the error, delete the file /etc/init.d/instana-agent, and then run the systemctl enable instana-agent command.

  Then, you can see that the symlink is created correctly:

  systemctl enable instana-agent
  
  Created symlink /etc/systemd/system/multi-user.target.wants/instana-agent.service → /lib/systemd/system/instana-agent.service.`
  

• On CentOS 6 based systems, outdated libcurl versions can cause this log output:

  Updating YUM metadata ...
  YUM repository metadata update failed
  

To solve the issue, update the libcurl versions.

Customizing the agent configuration

After you install the host agent, you can customize the configuration of the agent as you need. For example, you can change the TCP port and host name of the Instana backend/service that your host agent connects to, configure the agent to report to multiple backends, or configure use of a proxy for indirect communication from the host agent to the backend.

For more information about customizing the agent configuration, see Configuring the host agent.

Starting the agent

To start the Instana agent, use one of the following commands:

• systemctl start instana-agent.service, if your operating system uses systemd as init system.

• service instana-agent start, if your operating system uses SysVinit as init system.

When you install the agent, it is enabled to start automatically at system boot. You can turn off this behavior by setting the environment variable INSTANA_AGENT_AUTOSTART=false when installing the agent.

Updating the agent

You can use the package manager of your operating system to upgrade an agent package on your machine. To update and manage the packages and patches of your system, run the following command:

yum update // apt-get upgrade

Then the instana-agent-static or instana-agent-dynamic packages are upgraded.

Uninstalling the agent

To uninstall the Instana agent, the first step is to get the name of the package. Then you can run a command with the package name.

• Uninstall on Debian Derivatives
• Uninstall on Red Hat Derivatives

apt list --installed | grep instana-agent

instana-agent-static-j9/generic,now 20221024-1455 amd64 [installed]

sudo apt-get purge <package_name>

yum list installed | grep instana-agent

instana-agent-static-j9.x86_64       20221024-1455                 @instana-agent

sudo yum remove <package_name>

rm -rf /opt/instana/agent

Prerequisites

A Java Development Kit (JDK) needs to be available for the agent. You can see the list of supported JDKs as follows. You have two options:

• Place or link the JDK into <instana-agent-install-dir>/jvm (so that <instana-agent-install-dir>/jvm/bin/java exists).

• The customizable way is to export an environment variable that is called JAVA_HOME to point to that JDK (this environment variable can be also set via instana-agent-install-dir>/bin/setenv).

The following JVMs are supported for running the agent:

• IBM OpenJ9 JDK 11 (Preferred)
• Azul Zulu JDK 11
• OpenJDK JDK 11
• Oracle Hotspot JDK 11
• Amazon Corretto JDK 11
• IBM J9 11

The JVM must be executable for all users on the system. You are recommended to use the latest available patch release of the Java distribution of your choice. The agent requires a JDK that supports TLSv1.3 (available in all current JDK 8 and JDK 11 builds). Depending on your OS distribution, the packages that are provided by the OS distributor might not contain strong encryption support due to export control. If you use such a package, you might meet the errors like "java.lang.RuntimeException: Could not generate DH key pair."

To reduce the resources used by the agent when detect file system changes, you need to install the inotify-tools package on your Linux distribution.

Agent communication

The agent downloads updates and sensors from the following host:

DNS Name: artifact-public.instana.io Destination Port: tcp/80 and tcp/443

Currently, the Instana Service is provided in two different regions. Your individual instance is located geographically closest to most of your agents and users. Agents are preconfigured upon download, but some installation methods that require Instana Backend configuration. Consult the Agent Management section inside the product or your technical contact at Instana to learn about the region where your instance is located.

If you are using the self-hosted (on-premises) Instana, use the endpoints that are described at Self-Hosted Classic Edition (Docker).

If you are using Instana SaaS, use the endpoints that are described at host agent endpoints.

Downloading the agent

To download the agent, complete the following steps:

1. Sign in to the Instana UI, and then select an option to display the agent catalog; for example, on the home page, click Deploy Agent.

   If you are starting a new trial instance of Instana, the agent catalog is displayed with a prompt to select a host agent to install.

2. Click the tile Linux - Archive (tar.gz)

3. Select the packaging option to install a dynamic or static agent.

4. In the dropdown menu, select the platform architecture (operating system).

   The archive file is automatically preconfigured with your Instana account settings so that you only need to extract and start the agent.

5. Click the download link, and then extract the agent archive.

wget --save-cookies {{agent_folder_name}}/instana-cookies.txt --post-data 'email={{instana_username}}&password={{instana_password}}' https://{{instana_tenant_unit}}-{{instana_tenant}}.instana.io/auth/signIn

wget --content-disposition --load-cookies {{agent_folder_name}}/instana-cookies.txt --post-data 'type=linux64' https://instana.io/ump/{{instana_tenant}}/{{instana_tenant_unit}}/agent/download -O {{opt_folder}}/{{name_of_agent_archive.tar.gz}}

Starting the agent

To start the agent, run the following command:

INSTANA_AGENT_FOLDER/bin/start

Stopping the agent

To stop the agent, run the following command:

INSTANA_AGENT_FOLDER/bin/stop

Status of the agent

To see the status of the agent, run the following command:

INSTANA_AGENT_FOLDER/bin/status

.tar file installation (RHEL 9)

To install the Instana agent, complete the following steps:

1. Create a normal user for the Instana agent:

   sudo adduser instana
   

2. Install Java.

3. Download the Instana agent .tar file from the agent download page, and then extract the .tar files. For extraction, use GNU tar. After you extract the .tar files, the instana-agent-* directory is displayed.

   tar -xvf <instana-agent-*>.gz
   

4. Change the owner of the instana-agent directory:

   chown -R instana:instana /instana-agent
   

5. Go to the INSTANA_AGENT directory and start the Instana agent:

   #To start agent
   ./bin/start
   

6. Optional: Check the status of the agent:

   ./bin/status
   

Supported features

The Instana agent that is run as a non-root user supports the following features:

Troubleshooting a non-root agent

The following issues might be encountered when the Instana agent is started:

• Issue: Access denied for /var/lib/instana

  java.nio.file.AccessDeniedException: /var/lib/instana
  

  Solution: Create a directory with root privilege and change ownership to an Instana user

  sudo mkdir /var/lib/instana
  sudo chown instana: /var/lib/instana
  sudo chmod 700 /var/lib/instana
  

• Issue: Memory calculcator error

  Skipping the memory calculator: cannot retrieve the maximum amount of memory to be used by the agent from /sys/fs/cgroup/memory/memory.limit
  The memory calculator has not been used, and neither the 'JAVA_MAX_MEM' nor 'JAVA_OPTS' environment variables specify a maximum heap setting
  

  Solution: This error must not impact the agent itself. The memory calculator is a small binary that is used for systemd or containers to determine how much Java heap to assign to the agent, based on limits set. You can fix it by entering the JAVA_OPTS environment variable.

• Issue: Password prompting issue that is caused by the agent trying to read the symlink of the process, which is owned by the root.

  Solution:

  • Add following entry in the sudoers file:

    instana ALL=(root)    NOPASSWD:/usr/bin/readlink
    

    This entry allows the agent to access the readlink command for root process.

  • Optional: Debug the issue by enabling sudo logs:

    Defaults logfile=/var/log/sudo.log
    

    The following snippet displays a log where the agent script tries to read the readlink command:

    Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
        PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
        /proc/21025/exe
    Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
        PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
        /proc/21025/cwd
    Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
        PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
        /proc/12152/exe
    Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
        PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
        /proc/12152/cwd
    

Setting and overriding environment variables

When the agent is installed as a package and is started by SystemV init or systemd, a place is needed to set specific Instana environment variables. Also, it might be necessary to override specific environment variables. For example, for LXC container support, the agent needs to have the LXC command-line utils on its path.

Check the following sections about setting and overriding environment variables:

• SystemV Init
• Systemd
  • Drop-in units
  • EnvironmentFile directives
  • Global environment settings

INSTANA_AGENT_KEY=$agentkey
INSTANA_AGENT_HOST=$endpoint
INSTANA_AGENT_PORT=$endpoint_port
PATH=/usr/local/bin:${PATH}

[Service]
Environment=INSTANA_AGENT_KEY=$agentkey
Environment=INSTANA_AGENT_HOST=$endpoint
Environment=INSTANA_AGENT_PORT=$endpoint_port
Environment=PATH=/usr/local/bin:${PATH}

INSTANA_AGENT_KEY=$agentkey
INSTANA_AGENT_HOST=$endpoint
INSTANA_AGENT_PORT=$endpoint_port
PATH=/usr/local/bin:${PATH}

[Service]
EnvironmentFile=/etc/instana/environment.conf

[Manager]
DefaultEnvironment="INSTANA_AGENT_KEY=<key>" "INSTANA_AGENT_HOST=<host>" "INSTANA_AGENT_PORT=<port>"

Configuration files

The following files are created once during the installation:

• /opt/instana/agent/etc/mvn-settings.xml
• /opt/instana/agent/etc/org.ops4j.pax.url.mvn.cfg
• /opt/instana/agent/etc/instana/com.instana.agent.bootstrap.AgentBootstrap.cfg
• /opt/instana/agent/etc/instana/com.instana.agent.main.config.Agent.cfg
• /opt/instana/agent/etc/instana/com.instana.agent.main.config.UpdateManager.cfg
• /opt/instana/agent/etc/instana/com.instana.agent.main.sender.Backend.cfg
• /opt/instana/agent/etc/instana/configuration.yaml

These files are not overwritten when a user edits them or the package manager upgrades the agent package.

Temporary files

The Instana host agent installs some files that it needs in /tmp/.instana/ directory, if those files are deleted over the uptime of the host, it can cause issues during updates.

# Instana files in /tmp cannot be deleted when the agent is running,
# or it may lead to issues on agent update
R! /tmp/.java_pid*
R! /tmp/.instana/*

#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

# See tmpfiles.d(5) for details
# Clear tmp directories separately, to make them easier to override
v /tmp 1777 root root 10d
v /var/tmp 1777 root root 30d

...

WARNING: The tmpfiles.d policies of this system may cause the Instana agent to malfunction by deleting the /tmp/.instana directory; refer https://www.instana.com/docs/setup_and_manage/host_agent/on/linux#tmpfiles.d for more information.

# Deal with tmpwatch, found in RHEL 6 and CentOS 6.
# It is a cronjob, installed in /etc/cron.d/daily, with pretty much hardcoded
# configurations to wipe data the Instana agent needs.
readonly TMPWATCH_CRONJOB='/etc/cron.d/daily/tmpwatch'
if [ -f "${TMPWATCH_CRONJOB}" ]; then
  # Let's check if we already touched the file and, if not, add our excludes
  if ! grep "${TMPWATCH_CRONJOB}" Instana 2&>1 > /dev/null; then
    echo "tmpwatch discovered in '${TMPWATCH_CRONJOB}': adding the exclude flags for Instana"
    # Edit in place with `sed`
    sed -i '/^flags=/a flags="${flags} -x /tmp/instana -x /tmp/.javapid* # Added by Instana"' "${TMPWATCH_CRONJOB}"
  fi
fi