Administering the host agent on Linux
After you install the host agent, configure and manage the agent as you require. See the following list for all the available agent configuration options on Linux.
Checking the status of the host agent
After you install the host agent, check the status of the host agent on the Instana UI or on the host. For more information, see Checking the status of the host agent.
Customizing the agent configuration
After you install the host agent, the following configuration files are created:
/opt/instana/agent/etc/mvn-settings.xml
/opt/instana/agent/etc/org.ops4j.pax.url.mvn.cfg
/opt/instana/agent/etc/instana/com.instana.agent.bootstrap.AgentBootstrap.cfg
/opt/instana/agent/etc/instana/com.instana.agent.main.config.Agent.cfg
/opt/instana/agent/etc/instana/com.instana.agent.main.config.UpdateManager.cfg
/opt/instana/agent/etc/instana/com.instana.agent.main.sender.Backend.cfg
/opt/instana/agent/etc/instana/configuration.yaml
You can customize the configuration of the agent as you need. For example, you can change the TCP port and hostname of the Instana backend and services that your host agent connects to, configure the agent to report to multiple backends, or configure a proxy for indirect communication from the host agent to the Instana backend.
For more information about customizing the agent configuration, see the following topics:
- Agent logging
- Configuring Instana backend
- Configuring host agent mode
- Configuring host agents by using the agent configuration file
- Converting a static agent to a dynamic agent
- Limiting host agent CPU and memory
- Retrieving current version of dynamic agents
- Setting up an agent proxy
- Setting up TLS encryption for agent endpoint
- Updating interval of dynamic agents
- Version pinning of dynamic agents
Starting, stopping, and updating the agent
Starting the agent
When you install the host agent, it is enabled to start automatically at system boot. You can turn off this behavior by setting the environment variable INSTANA_AGENT_AUTOSTART=false when you install the agent.
-
To start the host agent, run the following command:
/opt/instana/agent/bin/start -
If your operating system uses
systemdas the init system, run the following command:systemctl start instana-agent.service -
If your operating system uses
SysVinitas the init system, run the following command:service instana-agent start
Stopping the agent
To stop the agent, run the following command:
/opt/instana/agent/bin/stop
-
If your operating system uses
systemdas the init system, run the following command:systemctl stop instana-agent.service -
If your operating system uses
SysVinitas the init system, run the following command:service instana-agent stop
Updating the agent
You can use the package manager of your operating system to upgrade an agent package on your machine. To update and manage the packages and patches of your system, run the following command:
yum update // apt-get upgrade
The instana-agent-static or instana-agent-dynamic packages are upgraded.
Managing agents by using the Agent Management dashboard
You can complete the following operations by using the Agent Management dashboard in the Instana UI:
- Changing agent modes
- Changing log level
- Updating agents
- Resetting agents
- Rebooting agents
- Viewing sensors
- [Configuration management]
- Self-monitoring
To open the Agent Management dashboard, complete the following steps:
- In the sidebar of the Instana UI, select Infrastructure. Alternatively, in the host overview, under Instana Agent, click the agent.
- Click a specific monitored host.
- On the host dashboard, click Open Agent Management.
Setting and overriding environment variables
When the host agent is installed as a package and is started by SystemV init or systemd, you must set specific Instana environment variables. In addition, you might need to override specific environment variables. For example, for LXC container support, the host agent must have the LXC command-line utils on its path.
-
If your operating system uses
SysVinitas the init system, to set and override environment variables, go to the SystemV Init section. -
If your operating system uses
Systemdas the init system, to set and override environment variables, go to the Systemd section.
SystemV Init
If your operating system uses SysVinit as the init system, the host agent sources the corresponding shell scripts in the /etc/default/ directory (Debian and derivatives) or /etc/sysconfig/ directory (Red Hat and derivatives).
If the service name is not changed, the SysVinit script sources the /etc/default/instana-agent file or the /etc/sysconfig/instana-agent file.
For example, in Debian, you can update the Instana variables and PATH by using /etc/default/instana-agent (when this file is found readable):
INSTANA_AGENT_KEY=$agentkey
INSTANA_AGENT_ENDPOINT=$endpoint
INSTANA_AGENT_ENDPOINT_PORT=$endpoint_port
PATH=/usr/local/bin:${PATH}
INSTANA_AGENT_HOST and INSTANA_AGENT_PORT for agent configuration is deprecated. These environment variables are reserved for configuring Instana tracers to connect to the agent. If they are set globally, Instana tracers cannot connect to the agent. Support for using these environment variables to configure the agent will be removed in future versions of Instana.To find the endpoint, port, and agent key values, go to the home page of the Instana UI and click Agent deployment. On the agent deployment catalog page, click the tile Linux - Automatic Installation (One-liner). Then, you can see endpoint, port, and agent key values in the deployment script.
Systemd
If your operating system uses Systemd as the init system, you can set and override environment variables by using the following available options. The drop-in units method is the recommended approach because you need not modify the service unit file whereas the other methods need you to make changes to the file. These changes might also be removed during the updates. The drop-in units method ensures that the changes and environment variables set are retained even after updates.
sudo systemctl daemon-reload command, and then run the sudo systemctl restart instana-agent.service command to reload the agent with the variable changes.Drop-in units
By using drop-in units, you can override the specific settings without changing the original systemd unit file. You must place the drop-in file to /etc/systemd/system/<unit>.d/ with a name <name>.conf.
For example, to set the Instana and PATH environment variables, create the /etc/systemd/system/instana-agent.service.d/10-environment.conf file with the following contents:
[Service]
Environment=INSTANA_AGENT_KEY=$agentkey
Environment=INSTANA_AGENT_ENDPOINT=$endpoint
Environment=INSTANA_AGENT_ENDPOINT_PORT=$endpoint_port
Environment=PATH=/usr/local/bin:${PATH}
INSTANA_AGENT_HOST and INSTANA_AGENT_PORT for agent configuration is deprecated. These environment variables are reserved for configuring Instana tracers to connect to the agent. If they are set globally, Instana tracers cannot connect to the agent. Support for using these environment variables to configure the agent will be removed in future versions of Instana.EnvironmentFile directives
This method uses the drop-in units except the environment directives that are read from a separate file. For example, create a /etc/instana/environment.conf file with the following contents:
INSTANA_AGENT_KEY=$agentkey
INSTANA_AGENT_ENDPOINT=$endpoint
INSTANA_AGENT_ENDPOINT_PORT=$endpoint_port
PATH=/usr/local/bin:${PATH}
Then, in the systemd drop-in unit file, use the following configuration:
[Service]
EnvironmentFile=/etc/instana/environment.conf
INSTANA_AGENT_HOST and INSTANA_AGENT_PORT for agent configuration is deprecated. These environment variables are reserved for configuring Instana tracers to connect to the agent. If they are set globally, Instana tracers cannot connect to the agent. Support for using these environment variables to configure the agent will be removed in future versions of Instana.Global environment settings
By default, systemd reads global configuration from various paths such as /etc/systemd/system.conf or /etc/systemd/system.conf.d/10-default-env.conf. By placing DefaultEnvironment configuration in these files, they become available to all systemd units. For more information, see systemd config.
For example, the following configuration creates three Instana environment variables when you set in the/etc/systemd/system.conf.d/10-default-env.conf file:
[Manager]
DefaultEnvironment="INSTANA_AGENT_KEY=<key>" "INSTANA_AGENT_ENDPOINT=<endpoint>" "INSTANA_AGENT_ENDPOINT_PORT=<endpoint-port>"
INSTANA_AGENT_HOST and INSTANA_AGENT_PORT for agent configuration is deprecated. These environment variables are reserved for configuring Instana tracers to connect to the agent. If they are set globally, Instana tracers cannot connect to the agent. Support for using these environment variables to configure the agent will be removed in future versions of Instana.Deleting temporary files
The Instana host agent installs some files that it needs in the /tmp/.instana/ directory. If those files are deleted over the uptime of the host, it can cause issues during updates.
Tmpfiles.d
On system.d-based systems, the systemd-tmpfiles daemon can delete files that are needed by the host agent in the /tmp/.instana directory, which causes update failures that are hard to troubleshoot.
Since version 181, DEB and RPM agent packages automatically install the following configuration script in /usr/lib/tmpfiles.d that adjusts the behavior to not delete the required files and impact the functions of the Instana host agent.
# Instana files in /tmp cannot be deleted when the agent is running,
# or it may lead to issues on agent update
R! /tmp/.java_pid*
R! /tmp/.instana/*
This configuration allows tmpdfiles.d to delete the files that are needed by the Instana agent only after a host restart. The newly running Instana host agent re-creates the files on startup.
Red Hat Enterprise Linux and CentOS, in versions 6 and 7, include default tmpdfiles.d policies that delete the entire /tmp directory every 10 days regardless of which policies a package can specify. It means that a package like Instana can define custom policies to control the behavior of the system or the package itself.
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.
# See tmpfiles.d(5) for details
# Clear tmp directories separately, to make them easier to override
v /tmp 1777 root root 10d
v /var/tmp 1777 root root 30d
...
Short of editing the default tmpfiles.d policies, no known workaround is available. The offending tmpfiles.d policy is located in /usr/lib/tmpfiles.d/tmp.conf.
Many Debian derivatives, including Ubuntu, include default tmpdfiles.d policies that delete the entire /tmp directory regardless of which policies a package like Instana's can specify. Upon installation, the DEB and RPM packages run a test to verify the behavior of the temporary files tmpfiles.d, running the command systemd-tmpfiles --remove.
You might see the following warning on systems with policies that override the required behavior that is specified by the Instana agent packages:
WARNING: The tmpfiles.d policies of this system may cause the Instana agent to malfunction by deleting the /tmp/.instana directory; refer https://www.instana.com/docs/setup_and_manage/host_agent/on/linux#tmpfiles.d for more information.
You can address this issue by editing the default tmpfiles.d policies. The offending tmpfiles.d policy is located in /usr/lib/tmpfiles.d/tmp.conf.
Tmpwatch
Red Hat Enterprise Linux 6 and CentOS 6 include the tool tmpwatch, which is known to delete unnecessary files in the /tmp/.instana. However, the use of tmpwatch in these distributions can occasionally cause update problems with the host agent. For more information, see Deleting temporary files.
# Deal with tmpwatch, found in RHEL 6 and CentOS 6.
# It is a cronjob, installed in /etc/cron.d/daily, with pretty much hardcoded
# configurations to wipe data the Instana agent needs.
readonly TMPWATCH_CRONJOB='/etc/cron.d/daily/tmpwatch'
if [ -f "${TMPWATCH_CRONJOB}" ]; then
# Let's check if we already touched the file and, if not, add our excludes
if ! grep "${TMPWATCH_CRONJOB}" Instana 2&>1 > /dev/null; then
echo "tmpwatch discovered in '${TMPWATCH_CRONJOB}': adding the exclude flags for Instana"
# Edit in place with `sed`
sed -i '/^flags=/a flags="${flags} -x /tmp/instana -x /tmp/.javapid* # Added by Instana"' "${TMPWATCH_CRONJOB}"
fi
fi
Cleaning agent data directory
You can use the environment variable AGENT_CLEAN_DATA_DIR to clean the agent's data directory.
When you set AGENT_CLEAN_DATA_DIR to true before the agent startup, the contents from the tmp, cache, and logs directories are cleared. For dynamic agents, this process also removes outdated artifacts and fetches the latest artifacts and JAR files. For both dynamic and static agents, the process continues by rebuilding the cache and generating a fresh agent log.
To enable the environment variable, use the following configuration:
export AGENT_CLEAN_DATA_DIR=true
After enabling the environment variable, start the agent to run the cleaning process.
If you use dynamic agents, you are suggested to remove the AGENT_CLEAN_DATA_DIR environment variable when no longer needed. If it remains enabled, the agent cleans its data directory and downloads artifacts including logs during every startup, which increase the agent startup time.