"⚡ Kafka Is a Better Agent Bus Than Your Chat Loop"

#kafka #agents #java #architecture

Most agent runtimes are a while-loop with hope.

Call a model. Call a tool. Call the model again. If the tool is slow, the loop waits. If the tool fails halfway, you retry and double-charge the side effect. If ten users arrive together, you discover that “async” meant “threads until OOM.”

What Kafka gives the agent

Treat tool work and model steps as messages, not nested awaits:

Backpressure — consumers lag instead of melting the process
Retries with isolation — poison messages go to a DLQ, not into an infinite model retry
Fan-out — one “task created” event can wake planner, tool worker, and auditor
Audit — the log is the trace

This is the same reason we put payment capture and email sends on a bus. Agents are just another worker with expensive CPU and sticky side effects.

A minimal shape

user.request        → agent.tasks
agent.tasks         → agent.model.calls
agent.model.calls   → agent.tool.requests
agent.tool.results  → agent.tasks   (continue or finish)
agent.deadletters   ← anything that exceeded retries

Each message carries task_id, attempt, idempotency_key, and a hard budget (tokens + wall clock).

Idempotency is not optional

Tool calls that create tickets, charge cards, or send mail must key on idempotency_key. Kafka at-least-once delivery will replay. If your tool layer is not idempotent, the agent will look “smart” while it double-books the world.

public CompletionStage<ToolResult> run(ToolRequest req) {
    return idempotencyStore
        .begin(req.idempotencyKey())
        .thenCompose(token -> {
            if (token.alreadyDone()) {
                return CompletableFuture.completedFuture(token.result());
            }
            return execute(req).whenComplete((r, err) ->
                idempotencyStore.complete(token, r, err));
        });
}

Backpressure beats bigger boxes

When model latency spikes, lag grows on agent.model.calls. That is visible. A chat loop hides the same overload as rising latency and silent queueing inside one JVM. Prefer lag alerts over “the agent feels slow.”

Tune max.poll.records, pause partitions when budgets trip, and keep tool workers separate from model workers so a stuck HTTP tool does not block planning.

Honest trade-off

Kafka is operational weight. For a single-user CLI agent, a loop is fine. The bus starts to pay rent when you have concurrent users, shared tools with side effects, or a need to replay incidents.

Do not put prompt text in cleartext topics if the cluster is a shared compliance boundary. Encrypt or store payloads by reference.

When to reach for it

Multiple agent workers must share one task stream
Tools have side effects that need idempotent retries
You already run Kafka for the product and know the ops cost

Would you debug last week’s agent incident from a topic, or from a screenshot of a chat transcript?

DEV Community