How to use LogDNA provider for Terraform to define custom views.

IBM Log Analysis and IBM Cloud Activity Tracker are two critical components under the Observability category in IBM Cloud

They can be used to collect, filter, search and tail log data, define alerts and design custom views to monitor application and system logs. Both are based on the same underlying technology.

As more services and applications start sending logs to IBM Log Analysis and IBM Cloud Activity Tracker, it will quickly become very important to isolate logs between different teams and design custom views to make it easier for users to search and find relevant log statements.

LogDNA provider for Terraform

Of course, you can use the IBM Cloud console to configure these views. This works well for one Log Analysis instance and a few views. As your project grows and more applications are deployed (or more teams working on the project), you may want to (or I should say you should) automate the configuration. Fortunately, IBM Log Analysis supports the LogDNA provider for Terraform for its configuration. 

As an example, let’s say I want to create a view in Log Analysis to only show the logs produced by IBM Cloud Kubernetes Service. Using the LogDNA provider for Terraform, it’s pretty straightforward. It would be similar to something like this:

provider "logdna" {
  servicekey = var.logging_api_key
  url        = var.logging_api_url
}

resource "logdna_view" "containers" {
 name     = "Compute ~ Containers"
 query    = "host:containers-kubernetes"
}

In this Terraform snippet:

• logging_api_key is the service key to access the LogDNA API (this is different from the Ingestion API key). It can be obtained from the Service Credentials of the Log Analysis instance.
• logging_api_url is the API endpoint. It is specific to the region where the Log Analysis instance was created (e.g, https://api.us-south.logging.cloud.ibm.com).

Sample set of custom views

But that’s just one view; what about other IBM Cloud services? To help in this matter, I compiled a list of simple views to populate a Log Analysis or Activity Tracker instance in a Terraform module, released to the Terraform Registry. The module can be found here.

The example shows how to use the module:

module "views" {
  source          = "we-work-in-the-cloud/logging-default-views/ibm"
  logging_api_key = ibm_resource_key.logdna_ingestion_key.credentials.service_key
  logging_api_url = "https://api.${var.region}.logging.cloud.ibm.com"
}

Once applied, a set of views defined here is created in your Log Analysis or Activity Tracker instance. The current set of views covers Compute, Databases, Security and VPC logs:

This module aims to be a starting point and an example of what is possible with the LogDNA provider. The provider has even more options to configure your Log Analysis instances. You may want to clone the repo and customize the views or simply be inspired and start capturing your existing view configuration in Terraform. By doing so, it becomes easier to replicate a configuration from one instance to the next — across different regions or accounts, for example.

Feedback, questions and suggestions

If you have feedback, suggestions or questions about this post, please reach out to me on Twitter (@L2FProd).