Memory leaks are common in almost any language—including garbage-collected languages—and Go is no exception. A reference to an object (if not properly managed) may be left assigned, even if unused. This usually happens on an application-logic level but can also be an issue inside an imported package.

Unfortunately, it’s hard to detect and fix memory leaks in development or staging environments—both because the production environment has different and more complex behavior and because many memory leaks take hours or even days to manifest themselves.

How to find memory leaks in production

Golang has a powerful profiling toolset called pprof that includes a heap allocation profiler. The heap profiler gives you the size of the allocated heap and the number of objects per stack trace (i.e., the source code location where the memory was allocated). This is critical information but is not sufficient as a single profile. To detect if there is an actual leak over a period of time, you need to record and compare regular allocation profiles.

Some issues when using pprof against production environments include the following:

• The profiler’s HTTP handler, which accepts profiling requests, must attach itself to the application’s HTTP server or have one running. This means you should take extra security measures to protect the listening port.
• Locating and accessing the application node’s host to run the go tool pprof may be tricky in container environments like Kubernetes.
• If the application has crashed or can’t respond to the pprof request, no profiling is possible.
• Obtaining the historical stack trace view of heap allocations—a regular manual pprof execution—requires interactive result analysis and comparison.

Using IBM Instana^™ for automatic memory leak detection and profiling

The IBM Instana platform automates the collection of heap allocation profiles, solving the above-mentioned issues. The IBM Instana Go Profiler, initialized in the application, continuously records and reports allocation profiles to the dashboard.

See the IBM Instana Profiling documentation for detailed setup instructions. After the application is restarted or deployed, the profiles will be available in the dashboard in a historically comparable form.

Similar profile history is automatically available for CPU usage, blocking calls and HTTP handlers. CPU, memory and GC metrics from Go runtime are also automatically available in the dashboard.