NATS is the messaging backbone I reach for before Kafka

#ai #softwareengineering #opinion #devops

Most teams do not choose a messaging system. They accumulate one: a queue for jobs, pub/sub for cache invalidation, request/reply for services, then a durable event log for analytics. Suddenly the diagram has SQS, SNS, RabbitMQ, Kafka, and three internal libraries called "messaging".

This is why NATS deserves attention.

Core NATS gives you fast pub/sub, queue groups, and request/reply. JetStream adds persistence, streams, consumers, acknowledgements, replay, key-value buckets, and object storage. One binary can cover three jobs that often become three products.

That does not make NATS a tiny Kafka. If your event log is the source of truth, if replay from months of history is normal, or if the data ecosystem matters more than latency and topology, Kafka is still the obvious answer.

The better claim is narrower: NATS is a strong default when you need a communication fabric before you need a data platform.

the shape

Core NATS routes messages by subject. A service publishes to orders.created, subscribers express interest in orders.* or orders.>, and the server moves bytes with little ceremony. Core delivery is at-most-once. If nobody is listening, the message is gone. If a subscriber is too slow and buffers overflow, messages can be dropped.

That sounds bad only if you expected durable messaging from the core protocol. For live signals, cache invalidations, service discovery, request/reply, and control-plane chatter, this is the point.

JetStream is where durability enters. A stream stores messages for one or more subjects. A consumer is a stateful view over that stream. The server tracks delivery, acknowledgements, redelivery, filters, starting position, and retention.

A small setup is not exotic:

jetstream {
  store_dir: "/var/lib/nats/jetstream"
  max_mem_store: 1G
  max_file_store: 100G
}

And the stream definition usually names the business thing:

nats stream add ORDERS \
  --subjects "orders.*" \
  --storage file \
  --replicas 3 \
  --retention limits \
  --max-age 168h

JetStream KV and Object Store are useful, but they are stream abstractions, not replacements for Postgres, S3, or search.

the comparison

Axis	NATS and JetStream	SQS	RabbitMQ	Kafka
Delivery	Core NATS is at-most-once. JetStream is at-least-once, with publisher deduplication inside a configured window.	Standard queues are at-least-once. FIFO queues add ordering and 5-minute deduplication.	Ack-driven queues. Quorum queues add replicated durability.	Durable log with consumer offsets. Exactly-once exists through transactions, but it shapes the design.
Ordering	Subject and consumer design matter. Queue groups scale work but remove total ordering.	FIFO ordering is per message group. Standard queues do not give strict ordering.	Queue order holds until concurrency, redelivery, priority, or multiple consumers complicate it.	Ordering is per partition. More partitions mean less global order.
Replay	JetStream replays retained messages, bounded by age, bytes, messages, or storage.	Retention is up to 14 days. Not an event log.	Queues are work queues. Streams add log-style replay.	Kafka's home ground: long retention, compaction, replay, and tiered storage.
Fanout	Subject wildcards and queue groups are simple and expressive.	Usually SNS plus SQS for durable fanout.	Exchanges and bindings are mature and flexible.	Consumer groups share work. Multiple groups each keep position.
Backpressure	Core NATS reports slow consumers. JetStream adds pull consumers, acks, redelivery, and flow control.	Visibility timeout and consumer scaling are the main levers.	Prefetch, acks, dead lettering, and queue depth are normal knobs.	Backpressure becomes lag, disk, and retention pressure.
Operations	One small server binary. Clustering and JetStream need care, but the footprint is light.	Fully managed. You trade control for not running anything.	Mature, but clustering, queue types, upgrades, and plugins are real work.	Serious platform. Modern Kafka removed ZooKeeper, not operational responsibility.
Multi-region	Leaf nodes and superclusters are a real differentiator, especially for edge.	Regional service. Cross-region behavior is yours.	Federation and shovel exist, but feel like plumbing.	Possible, but usually a replication architecture.

SQS is what I want when the job is simple and the team is on AWS. Receive a message, process it, delete it. If processing fails, visibility timeout makes it visible again. Add a dead-letter queue after exhausted retries. It is boring in the best way, but it is not a communication backbone.

RabbitMQ is what I want when routing is part of the application. Exchanges, bindings, routing keys, prefetch, acknowledgements, and AMQP contracts are useful. The current story is different from old folklore: in RabbitMQ 4.x, classic queues are non-replicated, while quorum queues and streams are the durable replicated structures for serious workloads.

Kafka is what I want when the log is the product. If consumers join later and replay history, if compacted topics rebuild state, if analytics hangs off the same stream, Kafka earns its cost. Tiered storage helps retention economics, but it does not make Kafka small.

NATS is what I want when service communication is the product. Request/reply, fanout, queue groups, and durable operational events can share one subject space. That is attractive for a small platform team.

the multi-region argument

The strongest NATS argument is topology.

Real systems are no longer one neat region. They have edge devices, private networks, compliance boundaries, and workloads that need local autonomy with selective sharing.

NATS leaf nodes fit that shape. A leaf can dial out from an edge site into a hub. Local clients connect to the local server. Interest is bridged over the leaf connection, so the larger subject space can reach the edge.

Superclusters connect clusters across gateways. Leaf nodes connect local domains into that topology. JetStream domains, mirrors, and sources decide where durable data lives.

That explicit locality is the point. Some messages are local. Some are mirrored centrally. Some are just live signals. Kafka can do multi-region, but now you are designing replication. SQS is regional. RabbitMQ has federation and shovel. NATS makes the network shape feel native.

when I would pick each one

Pick SQS when you are on AWS, need a simple work queue, and do not want to run anything. Remember the 14-day retention limit and FIFO group model.

Pick RabbitMQ when you need complex routing, AMQP contracts, per-message acknowledgements, or a broker that models workflow state directly.

Pick Kafka when the event log is the source of truth. If replay is normal, retention is long, and analytics matters, Kafka is the boring correct answer.

Pick NATS when you need microservice RPC plus events, low latency, multi-region or edge topology, small ops footprint, and bounded durable replay.

the fintech version

In fintech, the decision is mostly about idempotency, ordering, and audit.

Ordering should usually be scoped to an account, card, payment, or ledger. Global ordering is expensive and often fake. NATS subjects and JetStream consumers can model scoped ordering if keys and concurrency are disciplined.

Idempotency is non-negotiable. JetStream publisher deduplication helps, much like SQS FIFO deduplication helps, but only inside a window. The business operation still needs an idempotency key in the system of record.

The thing you give up by choosing NATS over Kafka is the comfort of a very large replayable log. If you need to rebuild projections from years of immutable account events, I still want Kafka. If you need fast service coordination, regional fanout, and durable operational events with bounded replay, NATS is a strong fit.

My default question is simple: can core NATS handle the live communication, and can JetStream handle the durable parts with honest retention limits? If yes, start there. Add Kafka when the log becomes a product.