Event-driven architecture without the Kafka tax — lightweight patterns for small teams
Most EDA guides assume Kafka, EventBridge, and a platform team to run it. Small teams need the decoupling benefit without the operational weight. Three patterns that get you there with tools you already have.
We had eight engineers. Someone said we should 'add Kafka'. I asked why. The answer was: 'so services can talk to each other without coupling.' I said: 'that is a good goal. Kafka is one way to achieve it. It also needs a cluster, a schema registry, consumer group management, and someone who understands partition rebalancing at 2am.' We did not add Kafka.
What EDA actually is, stripped back
Event-driven architecture is a decoupling technique. Instead of service A calling service B directly, A emits an event -- 'order created', 'payment failed', 'user signed up' -- and B (or C, or D) reacts to it. The publisher does not know or care who is listening. That separation is the value. The message broker is just the delivery mechanism.
Kafka, EventBridge, and RabbitMQ are excellent delivery mechanisms. They are also significant operational commitments. For a team of four to twelve engineers, that commitment often costs more than the problem it solves.
Pattern 1: In-process typed event bus
If your services are modules in a monolith or a modular monorepo, you do not need a network hop to decouple them. An in-process event bus gives you publisher/subscriber separation with zero infrastructure.
Live sandbox — scroll up in the terminal to see both patterns run. To experiment: open any file in src/services/, add a new bus.on() listener, save (Ctrl+S), and watch your handler fire without touching any other file. That's the point.
```typescript [TypeScript]
type EventMap = {
'order.created': { orderId: string; customerId: string; total: number }
'order.cancelled': { orderId: string; reason: string }
'payment.failed': { orderId: string; errorCode: string }
}
class EventBus {
// Map of event name to Set of handlers.
// Set (not Array) so unsubscribing is O(1) -- just delete from the Set.
private handlers = new Map void>>()
on(event: K, handler: (payload: EventMap[K]) => void) {
if (!this.handlers.has(event)) this.handlers.set(event, new Set())
this.handlers.get(event)!.add(handler as (payload: unknown) => void)
// Return an unsubscribe function -- call it when the subscriber is torn down.
return () => this.handlers.get(event)?.delete(handler as (payload: unknown) => void)
}
emit(event: K, payload: EventMap[K]) {
this.handlers.get(event)?.forEach(h => h(payload))
}
}
export const bus = new EventBus()
// Publisher -- emits one event and stops thinking about it.
// No idea who is listening or how many listeners there are.
bus.emit('order.created', { orderId: 'ord_123', customerId: 'cus_456', total: 4999 })
// Subscriber -- notification module, registered independently.
// Never imports OrderService. Only knows about the bus and the event type.
bus.on('order.created', ({ orderId, total }) => {
sendConfirmationEmail(orderId, total)
})
// Another subscriber -- analytics module, also independent.
// Added after the fact. Zero changes to the publisher or the other subscriber.
bus.on('order.created', ({ customerId, total }) => {
trackRevenueEvent(customerId, total)
})
```csharp [C# / .NET 8]
// C# / .NET 8 -- same pattern, different syntax.
// C# has built-in events and delegates. I could have used those.
// But this shows the pattern clearly -- it's an idea, not a language feature.
public sealed class EventBus
{
// Dictionary maps event name to list of handlers.
// Same structure as the TypeScript Map -- just C# syntax.
private readonly Dictionary<string, List<Action<object>>> _handlers = new();
public Action Subscribe<TPayload>(string eventName, Action<TPayload> handler)
{
if (!_handlers.ContainsKey(eventName))
_handlers[eventName] = new List<Action<object>>();
// Wrap the typed handler so the dictionary can store it generically.
Action<object> wrapped = payload => handler((TPayload)payload);
_handlers[eventName].Add(wrapped);
// Return an unsubscribe action -- same as the TypeScript version.
return () => _handlers[eventName].Remove(wrapped);
}
public void Publish<TPayload>(string eventName, TPayload payload)
{
if (!_handlers.TryGetValue(eventName, out var handlers)) return;
foreach (var h in handlers) h(payload!);
}
}
// Records are perfect for event types -- immutable value objects.
// An event that already happened should not be mutable.
public record OrderCreated(string OrderId, string CustomerId, decimal Total);
public record PaymentFailed(string OrderId, string ErrorCode);
// Usage
var bus = new EventBus();
// Publisher -- knows nothing about who is listening
bus.Publish("order.created", new OrderCreated("ord_123", "cus_456", 49.99m));
// Subscribers -- registered independently, no cross-imports
bus.Subscribe<OrderCreated>("order.created", e => SendConfirmationEmail(e.OrderId, e.Total));
bus.Subscribe<OrderCreated>("order.created", e => TrackRevenueEvent(e.CustomerId, e.Total));
The EventMap type makes this refactor-safe -- TypeScript will error if a publisher emits the wrong shape or a subscriber reads a field that does not exist. The return value of on is an unsubscribe function, so there are no memory leaks in short-lived contexts.
The limitation: events do not survive a process restart, and they do not cross service boundaries. That is fine for most intra-module communication. When you need durability or cross-service delivery, reach for the next pattern.
Pattern 2: Postgres as an event store (the outbox pattern)
You already have a database. Use it. An append-only events table is a durable event log that costs you nothing new to operate.
-- One migration. No new infrastructure.
CREATE TABLE events (
id BIGSERIAL PRIMARY KEY,
type TEXT NOT NULL,
payload JSONB NOT NULL,
occurred_at TIMESTAMPTZ NOT NULL DEFAULT now(),
processed BOOLEAN NOT NULL DEFAULT false
);
CREATE INDEX ON events (processed, occurred_at) WHERE NOT processed;
The outbox pattern: write the event in the same database transaction as your domain change. This guarantees the event is emitted if and only if the domain change commits -- no dual-write inconsistency.
```typescript [TypeScript]
// Publisher: domain change + event in one transaction.
// If the transaction rolls back, the event never exists.
// That is the guarantee the outbox pattern gives you.
async function createOrder(db: Pool, order: NewOrder): Promise {
await db.transaction(async (trx) => {
await trx.query(
'INSERT INTO orders (id, customer_id, total) VALUES ($1, $2, $3)',
[order.id, order.customerId, order.total]
)
await trx.query(
'INSERT INTO events (type, payload) VALUES ($1, $2)',
['order.created', JSON.stringify({ orderId: order.id, total: order.total })]
)
})
}
// Consumer: poll for unprocessed events on a schedule (cron or setInterval).
// FOR UPDATE SKIP LOCKED lets you run multiple consumers safely --
// locked rows are skipped, so two workers never process the same event.
async function processEvents(db: Pool): Promise {
const { rows } = await db.query(
SELECT id, type, payload
FROM events
WHERE NOT processed
ORDER BY occurred_at
LIMIT 20
FOR UPDATE SKIP LOCKED
for (const event of rows) {
await handleEvent(event.type, event.payload)
await db.query('UPDATE events SET processed = true WHERE id = $1', [event.id])
}
}
```csharp [C# / .NET 8]
// C# / .NET 8 -- using Npgsql + Dapper.
// Same SQL, same guarantees. The pattern does not change between languages.
async Task CreateOrder(NpgsqlConnection conn, Order order)
{
using var trx = await conn.BeginTransactionAsync();
// Domain write and event write in the same transaction.
await conn.ExecuteAsync(
"INSERT INTO orders (id, customer_id, total) VALUES (@Id, @CustomerId, @Total)",
order, transaction: trx);
await conn.ExecuteAsync(
"INSERT INTO events (type, payload) VALUES (@Type, @Payload::jsonb)",
new {
Type = "order.created",
Payload = JsonSerializer.Serialize(new { order.Id, order.Total })
},
transaction: trx);
await trx.CommitAsync();
// CommitAsync throws on failure -- the event never exists if the order did not save.
}
// Consumer -- FOR UPDATE SKIP LOCKED is the same SQL in every language
async Task ProcessEvents(NpgsqlConnection conn)
{
var events = await conn.QueryAsync<StoredEvent>(@"
SELECT id, type, payload
FROM events
WHERE NOT processed
ORDER BY occurred_at
LIMIT 20
FOR UPDATE SKIP LOCKED");
foreach (var ev in events)
{
await HandleEvent(ev.Type, ev.Payload);
await conn.ExecuteAsync(
"UPDATE events SET processed = true WHERE id = @Id",
new { ev.Id });
}
}
FOR UPDATE SKIP LOCKED is the key. It lets multiple consumer processes poll the same table without stepping on each other -- locked rows are skipped rather than blocking. You get basic fan-out and parallelism with a single SQL clause.
Pattern 3: Combine them
In practice I use both. The in-process bus handles immediate, same-process reactions -- send a notification, update an in-memory cache. The Postgres event table handles durability and cross-service work. The domain service writes to the DB events table; a separate consumer process polls it and emits on the in-process bus for local handlers to pick up.
The two patterns layer cleanly: persistence at the boundary, speed in the core. You add a message broker later if and when throughput demands it -- and by then, your event contracts are already defined and tested.
When you actually need Kafka
Three signals: your event table is being polled so frequently that it is a meaningful fraction of your database load; you need guaranteed ordering across partitions that a single Postgres table cannot provide; or you have genuinely independent services deployed across teams who need a shared contract enforced by a schema registry. Those are real problems. They are also problems most teams do not have until well past the point where they can afford to solve them properly.
Start with the patterns that match your current scale. Migrate to a broker when the polling table actually hurts -- not when someone reads a blog post about Kafka. You will know when. Your database will tell you.
Top comments (0)