DEV Community

Cover image for gotracer: Turn Go Execution Traces into Actionable Findings
Haleh
Haleh

Posted on

gotracer: Turn Go Execution Traces into Actionable Findings

#go #cli #tooling

go tool trace provides a browser UI for exploring Go execution traces. It works well for manual
investigation, but it doesn't fit into scripts or CI pipelines — you can't grep the output, set
thresholds, or fail a build based on what the trace contains.

The problem

Go execution traces contain detailed runtime events: GC pauses, goroutine scheduling delays, mutex
waits, syscall blocks, heap growth, and more. These don't appear in application logs or metrics —
they're only visible inside the trace.

Without a way to set thresholds or run checks automatically, there's no practical path to catching
these issues in CI. You can't fail a build because a GC pause exceeded 20ms or flag a goroutine
count that keeps growing across releases.

Teams often end up investigating traces only when something is already slow in production.

Introducing gotracer

gotracer reads a Go execution trace, applies a set of
rules, and outputs findings — to the terminal, as JSON, or as an HTML report.

go install github.com/bright98/gotracer/cmd/gotracer@latest

curl -o trace.out "http://localhost:6060/debug/pprof/trace?seconds=5"
gotracer analyze trace.out
Enter fullscreen mode Exit fullscreen mode 
SEVERITY  RULE                   TIMESTAMP  MESSAGE
--------  ----                   ---------  -------
ERROR     GCPauseSpike           1.204s     GC pause of 45ms exceeds Error threshold (20ms)
WARN      HighSchedulingLatency  300ms      goroutine 42 waited 18ms to be scheduled
Enter fullscreen mode Exit fullscreen mode

Eight rules are included: GC pauses, scheduling latency, mutex contention, syscall blocks, goroutine
leak growth, heap spikes, processor starvation, and GC mark assist waits. Each finding includes a
timestamp and a suggested fix; repeated occurrences are grouped with a count.

CI integration

gotracer exits 1 on any Warn or Error finding, so it can be used as a CI gate:

- run: gotracer analyze trace.out
Enter fullscreen mode Exit fullscreen mode

If one code path triggers the same rule many times, --top N limits findings per rule:

gotracer analyze trace.out --top 3
Enter fullscreen mode Exit fullscreen mode

Configuration

When used as a Go library, each rule's thresholds are configurable. This is useful when different
services have different latency requirements — a batch job and a low-latency API don't share the
same acceptable GC pause.

Implementation

gotracer is built on golang.org/x/exp/trace, the structured trace reader introduced in Go 1.22.
Each rule processes the event stream once, so memory usage stays low even on large traces.

Source and docs: github.com/bright98/gotracer

Top comments (0)