Configuring the Python collector

After the Instana Python collector is installed, you won't need to manually configure it for monitoring. It automatically starts collecting key metrics and distributed traces that are related to your Python processes. However, you can configure the individual components according to your specific requirements.

General configuration

The Instana Python package aims to be a fully no-touch automatic solution for Python monitoring but still be fully configurable when needed. The following options are available to configure this package.

Enabling AutoProfile™

AutoProfile generates and reports process profiles to Instana automatically and continuously. Learn more about profiles in the Analyze Profiles section.

To enable AutoProfile set the environment variable INSTANA_AUTOPROFILE=true. AutoProfile is supported for manual installation only. Make sure that the Instana sensor is initialized in the main thread.

Establishing host agent communication

The Instana Python package tries to communicate with the Instana agent through IP 127.0.0.1 and as a fallback through the host's default gateway for containerized environments. If the agent is not available at either of these locations, you can use environment variables to configure where to look for the Instana host agent.

The environment variables must be set in the environment of the running process.

export INSTANA_AGENT_HOST = '127.0.0.1'
export INSTANA_AGENT_PORT = '42699'

See also:

Setting the service name

By default, the Instana makes the best effort in naming your services appropriately. If for any reason you want to customize how services are named, you can do so by setting an environment variable:

export INSTANA_SERVICE_NAME=myservice

See also the General Reference: Environment Variables for Language Sensors

Setting the process name

Use INSTANA_PROCESS_NAME to set a custom label for infrastructure entity that represents the Python process.

Package configuration

The Instana package includes a runtime configuration module that manages the configuration of various components.

As the package evolves, more options are added.

from instana.configurator import config

# To enable tracing context propagation across Asyncio ensure_future and create_task calls
# Default is false
config['asyncio_task_context_propagation']['enabled'] = True

Debugging and more verbosity

Setting INSTANA_DEBUG to a non-nil value enables extra logging output generally useful for development and troubleshooting.

export INSTANA_DEBUG="true"

See also the General Reference: Environment Variables for Language Sensors

Disabling automatic instrumentation

This Instana package includes automatic instrumentation that is initialized on package load. This instrumentation provides distributed tracing information to your Instana dashboard. To see the complete list of automatic instrumentation, see the Supported versions document.

You can disable automatic instrumentation (tracing) by setting the environment variable INSTANA_DISABLE_AUTO_INSTR, which suppresses the loading of instrumentation that is built into the tracer.

export INSTANA_DISABLE_AUTO_INSTR="true"

Kubernetes

In certain scenarios on this platform, the Python sensor might not be able to automatically locate and contact the Instana host agent. To resolve this issue, see the Configuring Agent Network Access for Kubernetes section in the documentation.

See also:

Frameworks

You can configure the Instana Python collector to monitor and collect data from the following frameworks:

Django (Manual)

When the AUTOWRAPT_BOOTSTRAP=instana environment variable is set, the Django framework must be automatically detected and instrumented. If for some reason, you prefer to or need to manually instrument Django, you can instead add instana.instrumentation.django.middleware.InstanaMiddleware to your MIDDLEWARE list in settings.py:

import os
import instana

# ... <snip> ...

MIDDLEWARE = [
    'instana.instrumentation.django.middleware.InstanaMiddleware',
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

Pyramid

Starting from Instana Python 3.0.0, the Pyramid instrumentation is automatic.

Starting from version 1.22.0 to 2.5.3, the Instana Python package includes manual support for Pyramid. To add visibility to your Pyramid-based application, complete the following steps:

  1. Make sure that the instana package is added to the requirements.txt and installed in the virtual environment or container.
  2. Add import instana to the beginning of your __init__.py file for your Pyramid application.
  3. Add the Instana instrumentation Tween to your configuration.
import instana

with Configurator(settings=settings) as config:
    # ...
    config.include('instana.instrumentation.pyramid.tweens')
    # ...

The following image displays an example of the Instana instrumentation in your Pyramid configuration:

pyramid

In case you have the pyramid.tweens option set in your production.ini config, make sure that instana.instrumentation.pyramid.tweens.InstanaTweenFactory is the first entry in this list:

pyramid.tweens =
    instana.instrumentation.pyramid.tweens.InstanaTweenFactory
    # other tweens

WSGI and ASGI stacks

The Instana Python package includes Web Server Gateway Interface (WSGI) and Asynchronous Server Gateway Interface (ASGI) middleware that can be added to any compliant stack. Automation is available for various stacks, but manual addition is also possible for those stacks that don't have automatic support yet from Instana.

After you install the Instana Python package, use the following commands:

import instana

from instana.middleware import InstanaWSGIMiddleware
# or
from instana.middleware import InstanaASGIMiddleware

# Wrap the wsgi app in Instana middleware (InstanaWSGIMiddleware)
wsgiapp = InstanaWSGIMiddleware(MyWSGIApplication())

Instana is working to automate instrumentation for all major frameworks but in the meantime, check some specific quick starts for those stacks that don't have automatic support yet from Instana.

Bottle WSGI

Use the following commands for instrumenting Bottle WSGI:

# Import Instana and the Instana WSGI middleware wrapper
import instana
from instana.middleware import InstanaWSGIMiddleware

from bottle import Bottle, run

app = Bottle()

@app.route('/hello')
def hello():
    return "Hello World!"

# Wrap the application with the Instana WSGI Middleware
app = InstanaWSGIMiddleware(app)

# Alternative method for reference
# app = InstanaWSGIMiddleware(bottle.default_app())

run(app, host='localhost', port=8080)

CherryPy WSGI

Use the following commands for instrumenting CherryPy WSGI:

import cherrypy

# Import Instana and the Instana WSGI middleware wrapper
import instana
from instana.middleware import InstanaWSGIMiddleware

# My CherryPy application
class Root(object):
    @cherrypy.expose
    def index(self):
        return "hello world"

cherrypy.config.update({'engine.autoreload.on': False})
cherrypy.server.unsubscribe()
cherrypy.engine.start()

# Wrap the application with the Instana WSGI Middleware
wsgiapp = InstanaWSGIMiddleware(cherrypy.tree.mount(Root()))

In this example, we use uwsgi as the web server and booted with:

uwsgi --socket 127.0.0.1:8080 --enable-threads --protocol=http --wsgi-file mycherry.py --callable wsgiapp -H ~/.local/share/virtualenvs/cherrypyapp-C1BUba0z

Where ~/.local/share/virtualenvs/cherrypyapp-C1BUba0z is the path to my local virtualenv from pipenv

Falcon WSGI

The Falcon framework can also be instrumented through the WSGI wrapper as such:

import falcon

# Import Instana and the Instana WSGI middleware wrapper
import instana
from instana.middleware import InstanaWSGIMiddleware

app = falcon.API()

# ...

# Wrap the application with the Instana WSGI Middleware
app = InstanaWSGIMiddleware(app)

Then, booting your stack with uwsgi --http :9000 --enable-threads --module=myfalcon.app as an example

Gevent-based applications

Instana supports applications based on gevent 1.4 and later.

If you are manually importing the Instana Python package, make sure that the gevent import and monkey patching happen first.

    from gevent import monkey
    monkey.patch_all()
    import instana # <--- after the gevent monkey patching of stdlib

Before Instana Python Tracer 2.5.0, the gevent-based applications must not use the Activating without code changes method of package activation (that uses the AUTOWRAPT_BOOTSTRAP environment variable). This method doesn't work due to gevent's first-order monkey patching requirements as described earlier. In this case, use the Activating with code changes method.

Starting with Instana Python Tracer 2.5.0, the tracer automatically performs monkey.patch_all() when the AutoTrace webhook or the Activating without code changes method is used. You can fine-tune this monkey patching by setting the INSTANA_GEVENT_MONKEY_OPTIONS environment variable. With this comma-separated list, you can specify the modules to include or exclude from monkey patching as MONKEY OPTIONS to gevent's gevent.monkey.main function.

The following examples show options that are available for customizing modules for monkey patching:

export INSTANA_GEVENT_MONKEY_OPTIONS='--no-socket, --dns, --no-time, --select, --no-ssl'
export INSTANA_GEVENT_MONKEY_OPTIONS='no-socket, dns, no-time, select, no-ssl'
export INSTANA_GEVENT_MONKEY_OPTIONS='no-socket,dns,no-time,select,no-ssl'

If you use Django together with gevent and Instana autotracing, then ensure that you set the DJANGO_SETTINGS_MODULE environment variable before the autotracing starts. For more information about the DJANGO_SETTINGS_MODULE environment variable, see the Django documentation.

If this level of customization is still insufficient, then use the Activating with code changes method of package activation.

Tools

You can configure the Instana Python collector to monitor and collect data from different tools.

Web servers

The following configurations can be used to monitor different web servers:

uWSGI web server

Make sure enable-threads is enabled for uwsgi.

Threads

This Python instrumentation creates a lightweight background thread to periodically collect and report process metrics. By default, the GIL and threading are disabled under uWSGI. If you want to instrument your application that runs under uWSGI, make sure that you enable threads by passing the --enable-threads command (or enable-threads = true in INI style). For more information, see the uWSGI documentation.

uWSGI example: Command line

uwsgi --socket 0.0.0.0:5000 --protocol=http -w wsgi -p 4 --enable-threads

uWSGI example: INI file

[uwsgi]
http = :5000
master = true
processes = 4
enable-threads = true # required

Gunicorn web server

To instrument your application that runs under Gunicorn, make sure that you give --preload as an argument.

Preload

This Python instrumentation creates a lightweight background thread to periodically collect and report process metrics. By default, the application code processes after workers are forked in Gunicorn. If you want to instrument your application that runs under Gunicorn, make sure that you enable preloading by passing the --preload command. For more information, see the Gunicorn documentation.

Gunicorn example: Command line

To run Gunicorn with preloading, use the following command that is shown in this example:

gunicorn -w 4 --preload "file:app"

Gunicorn example: Config file

To use Gunicorn with a config file, use a Python file with the following variables. Add -c file_name.py to the Gunicorn command.

bind = "0.0.0.0:8000"
workers = 4
preload_app = true  # required

End-user monitoring (EUM)

Instana provides deep end-user monitoring that links server-side traces with browser events to give you a complete view from server to browser.

See the end-user monitoring page for more details.

Ignoring endpoints

Starting from Python Tracer 3.3.0, you can use this feature to filter out unnecessary traces or calls to reduce the overall data ingestion. For example, you can exclude the tracing of specific endpoints such as redis.get.

This feature is supported for the redis package only.

You can enable the endpoint exclusion by using any one of the following approaches:

If more than one approach is used at the same time, Python Tracer prioritizes in the following order: environment variable, in-code configuration, and agent configuration.

Setting environment variables

To specify the endpoints that you want to exclude, set the INSTANA_IGNORE_ENDPOINTS environment variable as shown in the following example:

INSTANA_IGNORE_ENDPOINTS=redis:get,type

This configuration excludes the GET and TYPE commands in the redis package from tracing.

To filter redis completely, you can use the following configuration:

INSTANA_IGNORE_ENDPOINTS=redis

To filter multiple services, you can use the following configuration:

INSTANA_IGNORE_ENDPOINTS=redis:get,type;dynamodb:query,scan

This configuration excludes the GET and TYPE commands for the redis service and the QUERY and SCAN commands for the dynamodb service.

Modifying in-code configuration

To ignore the endpoints by using the in-code configuration approach, pass the following configuration to the src/instana/configurator.py file:

config["tracing"]["ignore_endpoints"] = {"redis": ["get", "type"]}

This configuration excludes the GET and TYPE commands in the redis package from tracing.

To ignore all endpoints in the redis package, pass the following configuration to the src/instana/configurator.py file:

config["tracing"]["ignore_endpoints"] = {"redis": []}

To filter multiple services, you can use the following configuration:

config["tracing"]["ignore_endpoints"] = {"redis": ["get", "type"], "dynamodb": ["query", "scan"]}

This configuration excludes the GET and TYPE commands for the redis service and the QUERY and SCAN commands for the dynamodb service.

Modifying agent configuration

To exclude the endpoints by using the agent configuration approach, add the ignore-endpoints configuration to the agent configuration.yaml file. For more information, see the Ignoring endpoints section.

Additional information