Monitoring ClickHouse

ClickHouse is an open source column-oriented database management system. Instana can automatically discover and monitor ClickHouse server processes.

The ClickHouse sensor is automatically deployed and installed after you install the Instana agent.

Supported versions

Instana now supports Clickhouse 24.6 and earlier versions.

Sensor (Data Collection)

Tracked Configuration

The Instana agent automatically finds the ClickHouse configuration file (config.xml), either in its default location /etc/clickhouse-server/config.xml or by inspecting the --config-file argument that has been passed when starting the clickhouse-server process. However, the agent currently does not support (that is, does not automatically parse) settings overrides specified in auxiliary configuration files in conf.d/config.d directories next to the main config file (see the Configuration files section of the ClickHouse documentation). ZooKeeper configuration is currently also not automatically taken into account.

The following configuration values are automatically tracked by parsing the ClickHouse configuration file.

  • http_port
  • tcp_port
  • https_port
  • tcp_ssl_port
  • version
  • log
  • error_log
  • interserver_http_port
  • listen_hosts
  • max_connections
  • keep_alive_timeout
  • max_concurrent_queries
  • uncompressed_cache_size
  • mark_cache_size
  • path
  • tmp_path
  • user_config_file_name
  • default_profile
  • default_database
  • timezone
  • umask
  • builtin_dictionaries_reload_interval
  • max_session_timeout
  • default_session_timeout

Metrics

The following metrics are collected:

  • Merge
  • ReplicatedFetch
  • ReplicatedSend
  • ReplicatedChecks
  • BackgroundPoolTask
  • DiskSpaceReservedForMerge
  • DistributedSend
  • QueryPreempted
  • TCPConnection
  • HTTPConnection
  • InterserverConnection
  • OpenFileForRead
  • OpenFileForWrite
  • Read
  • Write
  • SendExternalTables
  • QueryThread
  • ReadonlyReplica
  • LeaderReplica
  • MemoryTracking
  • MemoryTrackingInBackgroundProcessingPool
  • MemoryTrackingForMerges
  • LeaderElection
  • EphemeralNode
  • ZooKeeperWatch
  • DelayedInserts
  • ContextLockWait
  • StorageBufferRows
  • StorageBufferBytes
  • DictCacheRequests
  • Revision
  • RWLockWaitingReaders
  • RWLockWaitingWriters
  • RWLockActiveReaders
  • RWLockActiveWriters

Configuration

The Instana agent automatically finds the ClickHouse config file (config.xml), as described above. The agent can figure out most of the configuration values it needs to connect to ClickHouse by parsing this config file.

The Instana agent accesses ClickHouse via its HTTP interface. In a vanilla ClickHouse installation, this interface is accessible without credentials. In this case, the Instana ClickHouse sensor does not need any special configuration.

If, on the other hand, you have configured the ClickHouse HTTP access to require credentials, you need to provide said credentials to the Instana agent. This can be done by configuring them in the agent's configuration file <agent_install_dir>/etc/instana/configuration.yaml.

Here are all possible configuration values for ClickHouse monitoring:

# Clickhouse
com.instana.plugin.clickhouse:
  enabled: false
  user: default    # the username used to access ClickHouse's HTTP interface
  password: root   # the password used to access ClickHouse's HTTP interface
  protocol: http   # 'http' or 'https'
  host: localhost  # the host name or IP of the ClickHouse server
  httpPort: 8123   # the port of the ClickHouse server
  tracingEnabled: false
  profilingEnabled: false
  clusterName: saas
  monitorTables:    # more in-depth metrics for these tables
    - 'database.table1'
    - 'database.table2'
    - 'database.table3'
  • enabled: The variable enables or disables the sensor for ClickHouse monitoring. Enter true to enable the sensor or false to disable it.
  • user: the username for accessing ClickHouse's HTTP interface,
  • password: the password for accessing ClickHouse's HTTP interface,
  • protocol: either 'http' or 'https',
  • host: the host name or IP of the ClickHouse server
  • httpPort: the port of the ClickHouse server
  • tracingEnabled: The variable enables or disables tracing. Enter true to enable tracing for gathering tracing metrics or false to disable it.
  • profilingEnabled: The variable enables or disables profiling. Enter true to enable profiling for gathering profile data or false to disable it.
  • clusterName: The name of the cluster to monitor.
  • monitorTables: The list of tables for which in-depth metrics are required.

The configuration value for httpPort needs to be an integer value, otherwise it will be ignored, that is, if it can't be parsed into a number, the standard port will be used.