Sample filters for customizing Analytics data

Adding a new field

You can add custom data to API event data by adding a field to the logging document. For example, if you want to search or visualize data that is stored in request and response headers, you can add fields to a filter to include that information in the top level of your analytics' data.

To avoid naming conflicts with future or current analytics fields, include a prefix that ensures your new field name is unique. For example, instead of naming the field employee_num you might name it x_mycompany_employee_num.

The following example copies the contents of the X-Employee-Num field from the request header and adds it to the x_mycompany_employee_num field in the event data. Adding the field to the event data enables you to use the information in visualizations.

# Here you should specify any filters that you want to take affect after
# the second stage of APIC Analytics filter processing.
#
# At this point the data has been modified by APIC Analytics, and is in the
# expected documented format. This is where you should feel free to modify
# the data prior to it being stored in APIC Analytics Storage.
#
# In message queue deployments, this will only affect data that will
# ultimately be stored in APIC Analytics Storage, and will not affect data
# that will be offloaded.
filter {
  if "apicapievent" in [tags] {
    if [request_http_headers] {
      ruby {
        code => "event.get('[request_http_headers]').collect {|i| event.set('[x_mycompany_employee_Num]', i['X-Employee-Num']) if i.has_key?('X-Employee-Num')}"
      }
    }
  }
}

Modifying an existing field

Sometimes you don't want to remove a field entirely from your data, you just want to redact sensitive information such as IDs to prevent them from being exposed in visualizations. You can modify the contents of a field and replace information with a symbols or a message.

The following example replaces sensitive information in the "Employee-Name" and "Employee-ID" header fields with the string "********sanitized********".

# Here you should specify any filters that you want to take effect after
# the second stage of APIC Analytics filter processing.
#
# At this point the data has been modified by APIC Analytics, and is in the
# expected documented format. This is where you should feel free to modify
# the data prior to it being stored in APIC Analytics Storage.
#
# In message queue deployments, this will only affect data that will
# ultimately be stored in APIC Analytics Storage, and will not affect data
# that will be offloaded.
filter {
  if "apicapievent" in [tags] {
    if [request_http_headers] {
      ruby {
        code => "headers=['X-Employee-Name','X-Employee-ID']; newHeaders = event.get('[request_http_headers]').collect {|i| headers.each {|header| i[header] = '********sanitized********' if i.has_key?(header)}; i}; event.set('[request_http_headers]', newHeaders)"
      }
    }
  }
}

Removing an existing field

Use the mutate remove_field operation to delete a field from the Analytics data. To remove multiple fields, delimit the field names with commas.

The following example removes the following fields from the Analytics data: request_http_headers, response_http_headers, request_body, response_body, and query_string.

# Here you should specify any filters that you want to take effect after
# the second stage of APIC Analytics filter processing.
#
# At this point the data has been modified by APIC Analytics, and is in the
# expected documented format. This is where you should feel free to modify
# the data prior to it being stored in APIC Analytics Storage.
#
# In message queue deployments, this will only affect data that will
# ultimately be stored in APIC Analytics Storage, and will not affect data
# that will be offloaded.
filter {
  if "apicapievent" in [tags] {
     mutate {
            remove_field => ["request_http_headers", "response_http_headers", "request_body", "response_body", "query_string"]
        }
  }
}

Filter and output plugin for splunk

When you create a filter for use with splunk, you must also create an output plugin for it.

# Here you should specify any filters that you want to take effect after
# the second stage of APIC Analytics filter processing.
#
# At this point the data has been modified by APIC Analytics, and is in the
# expected documented format. This is where you should feel free to modify
# the data prior to it being stored in APIC Analytics Storage.
#
# In message queue deployments, this will only affect data that will
# ultimately be stored in APIC Analytics Storage, and will not affect data
# that will be offloaded.
filter {
  if "apicapievent" in [tags] {
    ruby {
      code => "event.set('[@metadata][newevent]', event.to_json)"
    }
  }
}

apicapievent" in [tags] {
    http {
    url => "http://your-domain/services/collector/event"
      http_method => "post"
      codec => "json"
      content_type => "application/json"
      id => "offload_http"
      format => "message"
      message => '{"event": "%{[@metadata][newevent]}","index": "applications-int-ms","sourcetype": "json:apimlogs"}'
      headers => ["Authorization", "Splunk XXXX]
    }
  }
}