What problem are we solving with VPC Flow Logs?

Deploying distributed solutions comes with a set of requirements at all levels, from ensuring the security of the data to providing great availability and response time to end-users. There are several observability techniques you can put in place to ensure the requirements are met, such as collecting all application logs to a central logging system, instrumenting your compute resources to gather metrics, and so on. When an incident or degradation happens, having access to more than workload or user-centric data is key to determining what caused the anomaly, thereby enabling swift (if not automatic) restoration of the system. 

The network infrastructure is a critical foundational layer of your distributed systems. In the cloud age, it has, mistakenly, turned into an obscure, distributed, and often abstracted layer. When it comes to the network, you will want to not only collect information about the way the network flows in your environments to hone in on these anomalies, but also detect issues that could go completely unnoticed from a workload or user standpoint.

In the IBM Cloud Virtual Private Cloud (VPC), Flow Logs enable the collection, storage, and presentation of information about the IP traffic going to and from network interfaces within your VPC. Flow Logs for VPC are built into the IBM Cloud network fabric, and they are readily available to help with a number of tasks, including the following:

• Troubleshoot why specific traffic isn’t reaching an instance, which helps to diagnose potentially restrictive security policies.
• Analyze source and destination traffic from the network interfaces.
• Record the network traffic metadata that is reaching your instance, including for historical or regulatory purposes.
• Complement other available data to accelerate root0cause analysis and correlate incident data.
• Troubleshoot performance problems and the optimization of connectivity for development, testing, and IT Ops teams.

From a security standpoint, using Flow Logs enables security teams to do the following:

• Create a historical activity baseline, which can in turn be used to identify anomalies that could signal an attempted or planned attack.
• Identify potential botnet activity on a network by comparing the time-stamps of certain traffic or looking for connections to hosts associated with known botnets.
• Detect and block vulnerability scans against their network by checking for ping sweeps, port scans, and other malicious activity.

What is Kentik?

Kentik is the network observability company. Kentik’s platform is used daily by the network front line — whether digital business, corporate IT, or service provider. Network professionals turn to the Kentik Network Observability Cloud to plan, run, and fix any network, relying on our infinite granularity, AI-driven insights, and insanely fast queries. Kentik makes sense of network, cloud, host, and container flow, internet routing, performance tests, and network metrics, and is thrilled to be partnering with IBM Cloud, a leader in providing and managing hybrid cloud infrastructure for enterprises worldwide.

Kentik allows customers to visualize your entire network: hybrid cloud, multicloud, and on-premises — all in one place:

Solve problems fast in your IBM Cloud VPC environments by using Kentik’s rich visualizations and taking advantage of easy analysis of your network data:

Ask any question and get instant answers using Kentik’s Data Explorer for Network Observability:

Integrating Flow Logs with Kentik

Kentik makes it easy to ingest IBM Flow Logs into the Kentik Network Observability Cloud via Kentik’s Blueflow agent, which processes the logs from IBM Cloud buckets. Blueflow converts the logged data to kflow (Kentik’s flow record format), enriches it with other Kentik-collected network data (GeoIP, BGP, etc.), and stores it as flow records in Kentik. These records exist alongside flow data from your data center infrastructure and non-IBM cloud resources so you can see and analyze all of your network traffic data in a single comprehensive environment:

VPC Flow analytics example: What’s behind that spike?

Is it a misconfiguration? An attack? When network traffic rockets skyward, you need to find the root cause quickly so that your service is protected and your teams can resolve it fast. Using Kentik with IBM Flow Logs, you can automatically find these events, learn what’s causing them, and ask any question you want so you can articulate the problem and get it resolved, fast.

Configure a Kentik Insight to alert you when traffic spikes exceed thresholds in your IBM Cloud environment:

Use Kentik’s Pivot Dashboard to pivot the spike over 14 helpful visualizations, instantly. Modify the Pivot Dashboard to your preference, and instantly go from any dashboard pane to Data Explorer to ask any question you can think of:

VPC Flow analytics example: Finding infected hosts on your network

At some point, it’s inevitable. Through some slip or mishap, you’ve got a few uninvited guests doing  undesirable things on your network. Use Kentik to find the infected hosts and understand the impact of the intrusion so you can get back to work with confidence. 

Kentik’s Insights engine consistently monitors your IBM Cloud network activity to find any traffic to known botnets or internet threats and warn you:

Kentik also comes loaded with out-of-the-box dashboards to help you analyze these insights and discover which threats demand attention:

Once notified (or if otherwise investigating), use Kentik’s Data Explorer to unearth valuable details like which hosts sent traffic to malicious actors, what IPs and networks were communicated with, what protocols were used, how much data was sent, and when:  

Getting started with IBM VPC Flow Logs and Kentik

If you’re not already a customer, it’s easy to get started with IBM Cloud and Kentik:

• Enable Flow Logs on your VPC
• Start your Kentik trial
• Follow the instructions to install and docker run blueflow  
• See your network

Documentation

• About Flow Logs for VPC
• See demos and learn more at Kentik‘s web site
• Contact Kentik at sales@kentik.com
• Learn more about IBM VPC Flow Logs and Blueflow setup at the Kentik Knowledgebase