Example infrastructure

CPU distribution per Pod

This example shows the 99th percentile of Docker CPU usage for each Kubernetes pod in the namespace robot-shop grouped by top 5 Kubernetes Pod label.

CPU distribution per Pod

In this example, you can further narrow it down by applying a filter like: Kubernetes Pod > Label: starts with "app".

Compare CPU usage with Number of Calls

It's possible to track correlations between two different sources of metrics. You can compare the number of calls to the CPU usage.

CPU over calls

As CPU goes up, fewer calls are handled.

Custom metric

Custom metrics show up in the metric catalog:

Custom metric config

They can be rendered in any type of data widget. Use the formatting options to match the format of the custom metric.

Custom metric chart

Counts of technology instances

For each type of technology, a Count metric can be used to visualize the number of reporting instances for that technology type. This metric is rendered directly by using the metadata that Instana collects about the reporting entities. The minimum supported granularity for count metrics is 1 minute, as opposed to the normal 10 second resolution for infrastructure custom dashboard metrics. This is because the metadata changes are stored only at a 1 minute resolution.

Multi-dimensional metric visualization

Improved multi-dimensional metric visualization unlocks new visualization capabilities in custom dashboards.

In previous Instana releases, custom dashboards can visualize only a single metric per datasource. Metrics with multiple dimensions, such as metrics with different "tags", are considered distinct metrics and can only be individually visualized. For example, a Prometheus metric like http_request_duration with multiple status_code label values can not be easily visualized. You need to select each unique http_request_duration and status_code combination as a separate datasource.

By using multi-dimensional visualization, each metric dimension is split into a distinct Instana tag, which can be used in the filter and grouping criteria. Improved multi-dimensional metric visualization is available for the following metrics:

  • Prometheus and JVM custom metrics. Tags for these metrics are located in the tag catalog at metric > tag on the Instana UI.
  • Elasticsearch Index metrics. These metrics are split out from the Elasticsearch Node metrics. The Elasticsearch index tag is now available in the tag catalog at elasticsearch > index on the Instana UI.

More support will be provided for other multi-dimensional metrics in the future.

For multi-dimensional metrics, each time series is identified by a unique ID and associated tags. As with other types of tags, when you filter or group by these new metric tags, you can find time-series that match the specified filtering criteria, and then you can group them by using the grouping criteria.

Example

Suppose you have a JVM application, which is tracking the count of vehicles sold for each region, category and type and exposing these metrics by using Prometheus. When looking at the JVM dashboard for the application on the Instana UI, you can see all the metrics that are exposed by the application.

Vehicles prometheus dashboard

The sold_total metric has many dimensions. In custom dashboards, it is exposed just as a Prometheus metric sold_total without any of its dimensions.

Vehicles metric catalog

When multi-dimensional visualization is enabled, these dimensions are available for filtering or grouping under metric > tag.

Vehicles metric tag

Then, you can visualize the sold_total metric across any dimension as follows:

Vehicles visualization

Using a regex to select metrics

At times, you might have multiple metrics with similar names that you want to visualize together. Instead of selecting multiple data sets, one for each metric, use a regular expression (Regex) to select which metrics to include in a data set.

To select metrics by using a regex, complete the following steps:

  1. Select one of the metrics that you want to visualize from the List tab.

Select metric

  1. Click the Regex tab in the metric selector overlay and customize your expression.

Regex tab

Customize expression

Now, you see a preview of the matched metric names under the Regex editor.

When you create the widget, the regex data set includes all time-series that match the regex.

To see each time-series individually on a chart, group the result by metricId.

Regex syntax

Instana supports the following operators from Apache Lucene flavored regex:

The expression must match the entire metric name.

Any character: .

. matches any character. For example:

ab.     # matches 'aba', 'abb', 'abz', etc.

Repetition characters: ? + *

Repeats the preceding character of the expression a specific number of times. For example:

abc?     # repeat 'c' one or zero times. matches 'ab' and 'abc'
ab+      # repeat 'b' one or more times. matches 'ab', 'abb', 'abbb', etc.
ab*      # repeat 'b' zero or more times. matches 'a', 'ab', 'abb', 'abbb', etc.

OR operator: |

The match succeeds if the longest pattern on either the left side or the right side of the expression matches. For example:

abc|xyz  # matches 'abc' and 'xyz'

Grouping: ( … )

You can use a group to treat a part of the expression as a single character. For example:

abc(def)?  # matches 'abc' and 'abcdef' but not 'abcd'

Character sets: [ … ]

The match succeeds if one of the characters in the brackets matches the expression. For example:

[abc]   # matches 'a', 'b', 'c'

Inside the brackets, - indicates a range unless - is the first character or escaped. For example:

[a-c]   # matches 'a', 'b', or 'c'
[-abc]  # '-' is first character. Matches '-', 'a', 'b', or 'c'
[abc\-] # Escapes '-'. Matches 'a', 'b', 'c', or '-'

A ^ before a character in the brackets negates the character or range. For example:

[^abc]      # matches any character except 'a', 'b', or 'c'
[^a-c]      # matches any character except 'a', 'b', or 'c'
[^-abc]     # matches any character except '-', 'a', 'b', or 'c'
[^abc\-]    # matches any character except 'a', 'b', 'c', or '-'

Unsupported syntax

Perl Compatible Regular Expressions (PCRE) character classes like \w or \S are not supported. Anchor operators like ^ or $ are not supported. The expression must match the entire string.

Limitations

  • Custom dashboards for infrastructure are restricted to aggregating at most 10k time-series or 10k groups of time-series in a single query.
  • No more than 100k unique multi-dimensional time-series per license at a given time is supported.
  • Currently, custom dashboards for infrastructure does not support RBAC.