DEV Community: A.D.

klag just got a bunch better — here’s what’s new.

A.D. — Sun, 07 Jun 2026 20:28:45 +0000

Over half year ago I’ve open sourced klag, a lightweight Kafka consumer lag exporter. I’ve been working in data streaming systems and data infrastructure for over a decade and it’s an ultimate tool, I wish I had sooner.

Klag (pronounced “kay-lag”) docker image has been downloaded over 2,000 times, repo has passed 70 ⭐️, contributions started flowing in. So what has been added recently?

🤖 Talk to your lag

klag now ships an MCP server. Point your AI agent at it and just ask: "which consumer groups are falling behind right now?"
No PromQL. No squinting at dashboards. Debugging a backlog at 2am is now a conversation instead of a query-writing exercise. This is the feature I'm most excited about — lag monitoring goes from read this to ask this.

⚡ Native builds — no JVM, instant startup

There's now a native image build. Same exporter, compiled ahead-of-time, near-instant startup, smaller footprint.
If klag is a tiny always-on sidecar whose whole job is to scrape offsets and emit numbers — and it should be — this is the build you want. No JRE dragging along.

🎯 Smarter group filtering

You can now exclude groups, not just include them. Want myapp-* but not the noisy canary groups? Done. Comma-separated globs, includes and excludes, exactly where you'd expect them.

📦 One-command install

The Helm chart is on ArtifactHub and the Grafana dashboard is public and ready to drop in. No git clone, no hardcoded datasource UIDs. helm install and you've got lag metrics flowing into Grafana in minutes.

🙌 ...and it's not just me anymore

People are showing up with PRs that fix real edges in real clusters — full KAFKA_* passthrough, older-broker compatibility, more accurate time-based lag, and more. That's exactly the feedback loop I wanted.

Introducing klag: The Kafka Lag Exporter I Always Wanted.

A.D. — Tue, 27 Jan 2026 20:32:13 +0000

TL;DR: please welcome github.com/themoah/klag — a new Kafka lag exporter with multiple sink options and even more visibility into your Kafka consumers.

Kafka lag exporter was very cool. I couldn't imagine running serious kafka-based workload without it. And then the repo got archived. So I had no other choice: I had to build my own kafka lag exporter. Better one. With all the features that I was missing.

It's fast, lightweight and very extendable, built atop Java 21, vert.x library and micrometer. It's already supporting otlp, datadog and prometheus.

So what is a kafka lag?

Basically, it’s the number of records (messages) that haven’t been processed by a consumer group — the downstream service. In general, you expect it to be zero or close to it. If it’s growing fast — something is wrong. Decreasing lag is the best indicator that you’ve successfully identified and fixed the issue.

Simple, right?

But let’s not forget: scale and distributed systems can make even the simplest problem incredibly hard. Processing metrics in a timely manner on huge clusters with hundreds of brokers, thousands of consumer groups, and millions of partitions is no joke.

The missing features.

Lag velocity.

Measuring the speed of lag change (growth or decline) is the key. Finding the right threshold, based on amount of records is a long story of trial and error. Lag has been growing without control for last period of time? That's really requires attention.

Hot partitions.

One partition can generate lots of headache, skew in processing, late data and other bad impacts. 100+ partitions topic might have an average lag under the threshold, but a single outlier partition can break your data pipeline.

Stale groups.

Reduce the noise. Inactive groups require cleanup, which can be very tricky.

Splitting the requests.

Huge cluster with thousands of topic-partitions and tens (or hundreds) of consumer groups create some enormous amount of metrics, which will overload your Kafka (and Zookeeper if you are still using it) cluster. Chunking request into mini sub-groups reduces the load on the cluster.

What's next ?

Planned next features:

Advanced filtering: whitelist (or blacklist) topics and/or consumer groups.
Support running with multiple metrics reporters.
More metrics sinks - stasD, cloudwatch and others.

Feature requests and feedback is always welcome.
(If you are running klag in production - i'd love to share your story or add a link to the Readme).

Hi, I'm A.D.

A.D. — Sun, 15 Jan 2017 15:42:35 +0000

I have been coding for 5+ years.

You can find me on Twitter as @themoah

I live in Tel-Aviv.

I work for ClickTale

I mostly program in these languages: Scala, C#, Python, JS.

I am currently learning more about ML.

Nice to meet you.