Monitoring Synthetic PoP
After you install the Instana host agent in a Kubernetes cluster, the Synthetic PoP sensor is automatically installed to monitor Instana Synthetic PoP. You can view the Instana Synthetic PoP metrics in the Instana UI after you configure Synthetic PoP sensor as outlined in this topic.
Supported information
Supported platforms
Instana supports the stable version of Kubernetes. According to the Kubernetes version compatibility, two versions earlier than the stable version are considered as soft-deprecated. If the Kubernetes stable version is 1.26, the Kubernetes sensor is pinned to version 1.24. The minimum-supported version for the Kubernetes distribution is 1.10.
Supported Synthetic PoP versions
Synthetic PoP sensor supports metrics and configuration data for Synthetic PoP 1.0.9 and later.
Installing
The Instana Synthetic PoP sensor can work only with the Instana host agent installed on a Kubernetes cluster. By default, the Instana Synthetic PoP sensor is automatically installed if the Instana host agent is installed on the Kubernetes cluster.
By default, poll_rate
is set to 60 seconds. To change poll_rate
, update the Configuration file as shown in the following example:
# Synthetic PoP
com.instana.plugin.syntheticpop:
enabled: true
poll_rate: 60 # seconds, minimum 60 seconds by default
Enabling TLS on the Synthetic PoP sensor
If Synthetic PoP is deployed with TLS encryption, you must enable TLS encryption on the Synthetic PoP sensor to call the Synthetic health check service by HTTPS.
Configure the key or certificate files for the Synthetic PoP sensor. Use the same TLS key and certificate files that are used for deploying Synthetic PoP with TLS encryption.
To install the Instana host agent, complete the following steps:
- Find the TLS key and certificate files when you deploy Synthetic PoP with TLS encryption.
- Create a Kubernetes secret for all TLS key files and certificate files in the Instana agent namespace:
kubectl create secret generic pop-cert-secret -n instana-agent-namespace \ --type='kubernetes.io/tls' \ --from-file=ca.crt=/path/to/ca.crt \ --from-file=tls.crt=/path/to/tls.crt \ --from-file=tls.key=/path/to/tls.key
You can use the Helm chart, the YAML file, or operator to install Synthetic PoP sensor with TLS encryption.
Installing the Synthetic PoP sensor with Helm chart
To install and configure the Synthetic PoP sensor by using the Helm chart, complete the following steps:
- Download and decompress the Instana agent Helm chart:
helm repo add instana-agent-repo https://agents.instana.io/helm helm repo update helm pull instana-agent-repo/instana-agent tar xzvf instana-agent-*.tgz
- Add a property for all key and certificate files in both the
volumeMounts
andvolumes
sections in theinstana-agent/templates/agent-daemonset.yaml
file as shown in the following example:volumeMounts: - name: pop-cert mountPath: /opt/instana/agent/etc/pop volumes: - name: pop-cert secret: secretName: pop-cert-secret
- Set the
caCertificate
,clientCertificate
, andclientKey
fields as TLS encryption configuration. Create theagent_with_pop_sensor.yaml
configuration file as shown in the following example:com.instana.plugin.syntheticpop: enabled: true poll_rate: 60 # seconds, minimum 60 seconds by default caCertificate: '/opt/instana/agent/etc/pop/ca.crt' # path to X.509 CA certificate file if HTTPS is enabled for PoP Controller clientCertificate: '/opt/instana/agent/etc/pop/tls.crt' # path to X.509 client certificate file if HTTPS is enabled for PoP Controller clientKey: '/opt/instana/agent/etc/pop/tls.key' # path to client private key file if HTTPS is enabled for PoP Controller
- Install Instana host agent with the Synthetic PoP sensor:
helm install instana-agent \ ./path/to/instana-agent \ --namespace instana-agent \ --set agent.key='<your_agent_key>' \ --set agent.endpointHost='<your_host_agent_endpoint>' \ --set agent.endpointPort=443 \ --set cluster.name='<your_cluster_name>' \ --set zone.name='<your_zone_name>' \ --set-file agent.configuration_yaml=agent_with_pop_sensor.yaml
To verify the TLS configuration, enter the Instana host agent container, and check whether the ca.crt
, tls.key
, and tls.crt
files exist in the /opt/instana/agent/etc/pop/
directory.
If the Instana host agent is installed but Synthetic PoP is deployed with TLS encryption, you need to upgrade your Instana host agent by using TLS configuration to monitor Synthetic PoP.
Upgrading the Instana host agent
To upgrade your Instana host agent, complete the following steps:
- Follow step 1 to 3 in Installing the Synthetic PoP sensor with Helm chart.
- Run helm upgrade:
helm pull --repo https://agents.instana.io/helm --untar instana-agent && kubectl apply -f instana-agent/crds && helm upgrade instana-agent \
./path/to/instana-agent \
--namespace instana-agent \
--set agent.key='<your_agent_key>' \
--set agent.endpointHost='<your_host_agent_endpoint>' \
--set agent.endpointPort=443 \
--set cluster.name='<your_cluster_name>' \
--set zone.name='<your_zone_name>' \
--set-file agent.configuration_yaml=agent_with_pop_sensor.yaml
The Instana Helm chart deploys the instana-agent-operator internally, which then deploys the agent resources. This deployment is based on the agent custom resource (CR), which is created based on the Helm values. The operator pattern requires a custom resource definition (CRD) in the cluster before defining any custom resources. Therefore, the Helm chart includes the required CRD, which is deployed during the initial installations before submitting other artifacts. Currently, Helm can handle only the initial installations of CRDs and not their upgrade, deletion, and uninstallation. Because of this limitation, CRD updates are automatically submitted to the cluster only during the initial installation, and not during the upgrades. For more information on this limitation, see Custom Resource Definitions.
Installing the Synthetic PoP sensor with the YAML file
To install and configure the Synthetic PoP sensor by using a yaml
file within your Kubernetes cluster, see Installing the Host Agent on Kubernetes.
Customize the deployment.yaml
to add TLS certificates.
-
In the Instana UI, go to Installing Instana Agents, select your platform, usually is Kubernetes - Yaml or OpenShift - Yaml.
-
Enter a cluster name and download the
deployment.yaml
file. -
Add a property for all key and certificate files in both the
volumeMounts
andvolumes
sections in thedeployment.yaml
file as shown in the following example:volumeMounts: - name: pop-cert mountPath: /opt/instana/agent/etc/pop volumes: - name: pop-cert secret: secretName: pop-cert-secret
-
Update the key or certificate file path in the
com.instana.plugin.syntheticpop
section of thedeployment.yaml
file as in the following content:com.instana.plugin.syntheticpop: enabled: true poll_rate: 60 # seconds, minimum 60 seconds by default caCertificate: '/opt/instana/agent/etc/pop/ca.crt' # path to X.509 CA certificate file if HTTPS is enabled for PoP Controller clientCertificate: '/opt/instana/agent/etc/pop/tls.crt' # path to X.509 client certificate file if HTTPS is enabled for PoP Controller clientKey: '/opt/instana/agent/etc/pop/tls.key' # path to client private key file if HTTPS is enabled for PoP Controller
-
Install the Instana agent by applying the updated deployment file:
kubectl apply -f deployment.yaml
Installing the Synthetic PoP sensor with Operator
To install and configure the Synthetic PoP sensor by using the Operator, see Installing the Host Agent on Kubernetes. You must customize the
instana-agent.customresource.yaml
and edit daemonset to add TLS certificates.
-
Log on to the Instana UI.
-
In the navigation menu, click More > Agents.
-
Click Instana Agents, and select your platform. For example, Kubernetes operator or OpenShift operator.
-
In the Cluster name field, enter a cluster name, and download the
instana-agent.customresource.yaml
file. -
Update the key or certificate file path in the
configuration_yaml
section of theinstana-agent.customresource.yaml
file as shown in the following example:configuration_yaml: | com.instana.plugin.syntheticpop: enabled: true poll_rate: 60 # seconds, minimum 60 seconds by default caCertificate: '/opt/instana/agent/etc/pop/ca.crt' # path to X.509 CA certificate file if HTTPS is enabled for PoP Controller clientCertificate: '/opt/instana/agent/etc/pop/tls.crt' # path to X.509 client certificate file if HTTPS is enabled for PoP Controller clientKey: '/opt/instana/agent/etc/pop/tls.key' # path to client private key file if HTTPS is enabled for PoP Controller
-
Install the Instana agent by applying the updated customer resource file as shown in the following example:
kubectl apply -f instana-agent.customresource.yaml
-
Add
pop-cert-secret
to Instana agent DaemonSet by running the following command:kubectl patch daemonset.apps/instana-agent -n instana-agent --type='json' \ -p='[{"op":"add","path":"/spec/template/spec/containers/0/volumeMounts/0","value":{"mountPath": "/opt/instana/agent/etc/pop","name": "pop-cert"}},{"op":"add","path":"/spec/template/spec/volumes/0","value":{"name": "pop-cert","secret":{"defaultMode": 420,"secretName": "pop-cert-secret"}}}]'
When the Instana agent is reinstalled by using the Operator, the updates of
daemonset.apps
are lost. You must run this command again whenever you install the Instana host agent by using the Operator.
To verify the TLS configuration, enter the Instana host agent container, and check whether the ca.crt
, tls.key
, and tls.crt
files exist in the /opt/instana/agent/etc/pop/
directory.
Viewing metrics
To view the Synthetic PoP metrics, complete the following steps:
- In the navigation menu, select Infrastructure.
- Click the Comparison Table tab, and select Synthetic PoP in the Table content dropdown list.
- Click the name of Synthetic PoP to open the dashboard.
The Synthetic PoP dashboard is displayed with all the collected performance metrics and configurations.
Configuration data
The configuration data that is collected from the Instana Synthetic PoP is listed in the following tables:
Instana Synthetic PoP
Instana Synthetic PoP | Description |
---|---|
Location Name | Identified name of the Synthetic PoP. |
Location Display Name | Displayed name of the Synthetic PoP. |
Location Description | Location details of the Synthetic PoP. |
Version | Version of the Synthetic PoP. |
Started At | Start time of the Synthetic PoP. |
Playback Capabilities | Supported Synthetic type of the Synthetic PoP. |
Instana Synthetic Endpoint | Backend endpoint of the Synthetic PoP. |
Country | Deployed country of the Synthetic PoP. |
City | Deployed city of the Synthetic PoP. |
Redis TLS | Status of Redis TLS (Enabled or Disabled ). |
Tenant Type | Type of tenant (Single or Multi ). |
Instana Synthetic PoP Kubernetes
Synthetic PoP is deployed in a Kubernetes cluster with a namespace. You can go to the Kubernetes namespace and cluster from the Synthetic PoP snapshot or the metric dashboard.
Kubernetes | Description |
---|---|
Namespace | The namespace of the Kubernetes cluster where the PoP is deployed. |
Redis and Playback Engine Namespace | The namespace of the common Redis and playback engines for multi-tenant PoP. |
Cluster | The cluster name of the Kubernetes cluster where the PoP is deployed. |
Supported Synthetic types
Synthetic PoP supports the following Synthetic types:
- API Simple: API Simple test (type: HTTPAction)
- API Script: API Script test (type: HTTPScript)
- Browser Test: Browser Script test (type: BrowserScript), Webpage Script test (type: WebpageScript), and Webpage Simple test (type: WebpageAction)
- ISM (Internet Service Monitoring) Test: SSL Certificate Check (type: SSL Certificate)
Performance metrics
The performance metrics that are collected from Instana Synthetic PoP are listed in the following tables:
Synthetic PoP Metric | Description |
---|---|
Status | The status of the Synthetic PoP. |
Scheduled/Completed | The number of total scheduled and completed tests for enabled Synthetic types. |
Active Test Count | The number of running Synthetic tests for all Synthetic types. |
API Simple Workload | The workload for API Simple tests that is running in the HTTP playback engine. |
API Script Workload | The workload for API Script tests that is running in the Javascript playback engine. |
Browser Test Workload | The workload for Browser script, Webpage script, and Webpage simple tests that are running on the Browserscript playback engine. |
ISM Workload | The workload for SSL Certificate tests that are running on the ISM playback engine. |
Queued Tasks | The number of tasks in the task queue that are waiting to be executed for each type. |
Result Queue Depth | The number of all Synthetic test results in the result queue that are waiting to be sent to backend. |
Built-in events
Built-in events trigger issues or incidents based on failing health signatures on entities.
For more information about built-in events for the Synthetic PoP sensor, see Built-in events reference.
Custom events
Custom events trigger issues or incidents based on the thresholds of an individual metric of any specific entity.
The "Synthetic pop status" built-in event can trigger an offline event if the status becomes Offline
in some conditions, for example, communication error with Instana backend. However, after a PoP is uninstalled, the PoP
sensor is deactivated, and no default events are triggered. This behavior can be considered as normal.
To trigger an event for the PoP uninstallation case, you can create a customized event. Use the Offline event detection
system rule, and choose the entity.type:syntheticpop
scope. Select a period in the Grace Period list to keep the event open.
Monitoring issues
TLS not configured
Monitoring issue type: tls_not_configured
This monitoring issue indicates that TLS encryption is enabled in Synthetic PoP but TLS key and certificate files are not configured in PoP sensor. To configure TLS key and certificate files, see Enabling TLS on the Synthetic PoP sensor.
Redis TLS not enabled
Monitoring issue type: redis_tls_not_enabled
This monitoring issue indicates that TLS encryption is not enabled in Synthetic PoP. To ensure secure communication, it is recommended to enable TLS in the Synthetic PoP production environment. For more information, see Synthetic PoP TLS encryption.
Call PoP health failed
Monitoring issue type: call_pop_health_api_failed
This monitoring issue indicates that the Synthetic PoP sensor cannot get the health data of Synthetic PoP. Check Instana agent logs to see more error messages. To resolve this issue, try the following steps:
- Check whether the Instana agent is installed on a Kubernetes cluster by using Helm chart, YAML, or Operator
- If Synthetic PoP is installed with TLS enabled, check whether TLS is configured on the Synthetic PoP sensor correctly, see Enabling TLS on the Synthetic PoP sensor.