Kafka Offset Harbor: a local-first desktop window into Apache Kafka

🌐 Visit Kafka Offset Harbor

Kafka is the backbone of event-driven systems, but its data is invisible by design: there's no built-in window onto what's actually flowing through a topic. Answering "did this message arrive?", "why is this consumer behind?", or "can we replay from Tuesday?" usually means assembling CLI flags from memory.

Kafka Offset Harbor is my answer: a free, MIT-licensed desktop app (macOS, Windows, Linux-from-source) for browsing topics and messages, watching consumer lag, resetting offsets, and administering topics — built local-first, so your broker credentials never leave your machine.

Repo: https://github.com/rajeshkumaravel/kafka-offset-harbor

Why not the existing options?

• CLI tooling is complete but hostile to exploration — right for automation, wrong for "show me what's in this topic."
• Self-hosted web UIs invert the trust model: you deploy a service, hand it broker credentials, put it behind auth, and patch it forever. Reasonable for a platform team; overkill and a new attack surface for one engineer poking at staging.
• Commercial desktop tools proved the form factor, but closed source is a hard sell in the security-reviewed environments where a Kafka GUI matters most.

The gap: a desktop tool whose security story is inspectable — where "credentials never leave your machine" is a property you verify in source, not a promise in a policy.

What you get

• Explore — brokers (controller flagged), topics with partition/replication counts, consumer groups, all in one filterable tree; multiple saved connections with dev/stage/prod colour badges.
• Read — fetch from newest/oldest/a specific offset/a timestamp, per partition or all; a virtualized grid smooth at thousands of rows; contains/regex/JSONPath filters across key, value, headers; payloads in the Monaco editor (pretty JSON/XML, raw, hex); tombstones, sizes and headers surfaced; JSON/CSV export.
• Fix consumers — colour-coded per-partition lag, group state/members/assignments; offset resets to earliest/latest/offset/timestamp, guarded to idle groups only.
• Produce & administer — keys, tombstones, custom headers, repeat-N test data, JSON-array import; topic create/delete, partition expansion, config editing.
• Demo cluster — a full in-memory cluster (topics, messages, lagging groups) built in. Every feature works with zero setup and zero network access, because it's the real UI on a fake backend.

Architecture: the process boundary is the security boundary

Electron gives you a privileged Node.js main process and a Chromium renderer. Harbor's rule: everything sensitive — Kafka networking, credential decryption, disk persistence — lives only in main. The React renderer is untrusted display logic: it never sees a password and never opens a socket.

The renderer and main talk through one typed invoke bridge, against a contract declared in src/shared/ipc.ts. Every call returns a structured result envelope — success or typed error — so broker failures surface as values, not cross-process exceptions. Because CI type-checks both processes against the same contract, the app's entire privileged surface is readable in a single file, and extending it without declaring shapes is a compile error.

The adapter seam

Main reaches Kafka only through a ClusterAdapter interface with two implementations: KafkaJsAdapter (real clusters via kafkajs — PLAINTEXT, SSL/TLS, SASL PLAIN/SCRAM, AWS MSK IAM) and DemoAdapter (the in-memory demo cluster). The UI can't tell them apart — which is why demo mode is a first-class feature, UI development needs no broker, and a future backend is a third implementation rather than a rewrite.

Fetches are bounded (a window relative to newest/oldest/offset/timestamp — never an unbounded tail), and rendering is virtualized (TanStack Virtual), so cost tracks the viewport, not the topic size. No formal benchmarks yet — that's stated honestly, and the posture is structural.

The trade-offs, defended

Electron costs a Chromium runtime's memory. It buys one codebase with native installers on two platforms, the Monaco editor as a payload inspector (half the tool's value), and safeStorage — direct OS-keychain integration (Keychain/DPAPI/libsecret) with no custom crypto.

kafkajs over librdkafka bindings: the C client is faster, but a GUI's bottleneck is the human. Pure JS means the Kafka path needs no compile toolchain anywhere, keeping the parallel macOS/Windows CI build simple and the contribution barrier low. The only native module is better-sqlite3 (the connection store), isolated via the standard unpacked-module config.

Deliberately absent: telemetry, analytics, crash reporting, auto-update pings, cloud sync. "No phoning home" is only credible if it's total. Even the welcome-screen greeting comes from the system clock, because the obvious implementation (a weather API) ships your IP to a third party.

Security posture, verifiably

• Credentials: encrypted via OS keychain, stored in local SQLite, decrypted only in main at connect time. The IPC contract has no call that returns a secret. MSK IAM stores no secret at all — short-lived tokens minted from your local AWS credential chain via aws-msk-iam-sasl-signer-js.
• Network: outbound connections go to your configured brokers, full stop (plus AWS STS iff you use MSK IAM). Three runtime dependencies; none of the app source makes HTTP requests.
• Supply chain: better-sqlite3's install-time binary downloader (prebuild-install + its HTTP/tar chain) is explicitly excluded from the packaged app — commented and auditable in electron-builder.yml.
• Distribution: public GitHub Actions build (.github/workflows/release.yml) → tagged Releases. Builds are currently unsigned — disclosed, not hidden; signing is on the roadmap.
• Guardrails: offset resets refuse non-idle groups with an explanation; full local reset requires typing CONFIRM.

Try it

Grab an installer from Releases and hit Explore the demo cluster — no broker needed. Against a real one:

git clone https://github.com/rajeshkumaravel/kafka-offset-harbor.git
cd kafka-offset-harbor && npm install && npm run dev

# optional local broker with seeded topics + a lagging consumer group
docker build -t koh-kafka:local docker/
docker run -d --name koh-kafka -p 9092:9092 koh-kafka:local
./docker/seed.sh

Limitations & roadmap

No Schema Registry/Avro decoding yet (binary payloads show raw/hex); Linux installers pending; builds unsigned pending certificates; MSK IAM uses the default AWS credential chain (no per-connection profile picker yet); explorer, not monitor — nothing pages you.

Roadmap, roughly in order: Schema Registry/Avro, Linux packaging, code signing, saved queries, measured performance baselines.

Get involved

It's MIT and contributions are genuinely wanted — ⭐ star the repo, open an issue, argue with a trade-off in discussions, or pick up a roadmap item (Schema Registry is the big one). What do you use to look inside Kafka today?

---

And that’s it!

I hope you found this article useful, and as always, thanks for reading!

Happy Coding!
RK