A Better Way to Test Kafka and RabbitMQ Workflows

SixSet — Wed, 22 Jul 2026 15:29:03 +0000

This isn't a Kafka tutorial. It's about how I stopped writing disposable producer scripts every time I needed to test an event-driven workflow.

I keep running into the same situation. I'm testing a feature in a distributed system where the actions are loosely connected through events a user makes a payment or a purchase, and that has to start propagating: to accounting, to inventory, to whatever downstream service is waiting for it.

The problem is that on test environments, I rarely have the full chain available. Either the other services aren't deployed yet, or the functionality that would normally trigger the event just doesn't exist yet. So I need to fake that one event myself.

Most of the time this ended up being one of three things:

kafka-console-producer with a hand-typed payload
a small producer script in Python or Go, written once and never cleaned up
a throwaway Spring Boot app whose only job was to fire one event

All of these work. But after a few months, your repo (or your ~/scripts folder) turns into a graveyard of one-off producers, each slightly different, each half-remembered when you need it again six weeks later.

I got tired of that pattern enough to build Devset, a tool for testing event-driven systems without writing throwaway producers and consumers. This is a walkthrough of the three specific problems that pushed me to build it.

The examples below use Kafka, but the same workflow applies to RabbitMQ exchanges and routing keys.

Sending a single event

Sometimes I just need to fire one message a specific payload, on a specific topic, with a specific key and see what happens downstream. kafka-console-producer technically solves this problem, but in practice it still means constructing JSON by hand, remembering CLI flags, and repeating the whole process every time you want to replay the same event.

Instead of writing a producer every time, I paste the payload, pick the topic and key, and publish it. It gets saved, so the next time I need the same message I'm not reconstructing it from memory.

Replaying complete workflows

This is the part that actually mattered more than sending single messages.

Sometimes one event isn't enough I need to simulate a whole sequence. A user adds something to a cart, enters an amount, pays, and a separate service should confirm the order:

OrderCreated
     │
     ▼
InventoryReserved
     │
     ▼
PaymentAuthorized
     │
     ▼
InvoiceGenerated

That's a simplified example, but real workflows usually branch here. Inventory may fail, payment may time out, an order may need to be retried, or a compensating event may need to be emitted. Those are the scenarios that quickly turn a handful of producer scripts into something nobody wants to maintain.

Normally, testing that chain means either standing up four real services, or writing four scripts and running them in the right order with the right delays between them and hoping nothing about the payload shapes changed since the last time you touched them.

In Devset, this is one saved workflow. I lay the steps out on a canvas, each step can carry a condition, a delay, or reference state from an earlier step (so PaymentAuthorized can carry the order ID that OrderCreated generated), and the whole thing runs as a single unit that I can re-trigger any time I need it — instead of four services or four scripts, it's one saved workflow.

The same workflows also let me observe responses coming back, so I'm not only publishing events but exercising complete request/reaction chains.

Working with Protobuf

A chunk of my day-to-day work is on services that speak Protobuf, and firing Protobuf messages from a script is a different level of annoying than JSON. You need the schema on hand, you're dealing with compiled classes or manual encoding, and the moment the schema changes across environments you're debugging a wire-format mismatch instead of the thing you actually meant to test.

I keep the schemas I use versioned in one place now, and messages can be built against JSON Schema or Protobuf the same way — the encoding and validation happen automatically instead of being something I have to get right by hand every time.

Where this is going

Devset CE is source-available and self-hosted, and runs locally against your own brokers with a single Docker command. If you'd like to try it, the repository and documentation are linked below.

devset-io / devset-ce

Local-first workflow engine for testing Kafka and RabbitMQ event-driven systems. Define scenarios, send real messages, inspect results — all on your machine.

Local message lab for Kafka and RabbitMQ — no cloud, no scripts.

Devset

Testing event-driven systems is painful. You end up writing throwaway scripts to publish a few messages, integration tests are brittle and slow, multi-step scenarios need glue code nobody wants to maintain, and reproducing production payloads locally is a guessing game. Devset is a self-hosted scenario engine that compiles a declarative DSL into runs against real Kafka or RabbitMQ brokers, with a visual builder on top.

Think Postman or Cypress, but for Kafka and RabbitMQ instead of HTTP.

No cloud. Runs entirely on your machine.
No accounts. Open the UI and start.
No telemetry. Your scenarios stay local.

Source-available under FSL-1.1 — free for internal use; each release converts to Apache 2.0 on its second anniversary.

Quick start

1. Run Devset

docker run -p 8082:8082 -v devset-data:/data ghcr.io/devset-io/devset-ce:latest

Open http://localhost:8082.

2. Add a broker connection

Go to…

View on GitHub

Documentation: https://docs.devset.io

A few things I'm currently working on:

deeper branching in the Flow Builder, so workflows can handle retries, timeouts, and compensating events instead of just linear chains
live topic inspection
offline workflow simulation (run the pipeline without touching a real broker)
a JSON DSL so the same workflows can run from CI, not just the UI

If you work with Kafka or RabbitMQ and have opinions, issues, or PRs, I'd genuinely like to hear them.

How are you testing event chains like this today real services, mocks, something else? Curious what other people have landed on.

DEV Community: SixSet