Stateless vs Stateful

📺 Prefer to watch? 90-second YouTube Short · 💬 Telegram

Originally published on software-engineer-blog.com.

Same request, sent twice—does your server remember the first one? That single question splits every backend architecture into two opposing camps, and the answer determines whether you scale effortlessly or fight your infrastructure at every step.

• Mental model: Stateless = server answers and forgets; stateful = server remembers and pins you to itself.

---

The Core Question

When a client sends a request, it's asking your backend to do something: fetch data, update a cart, stream video. After the server answers, something critical happens: does the server keep a record of who you are and what state you're in? Or does it wipe its memory clean and force you to re-prove yourself next time?

That distinction cascades into nearly every decision you'll make about how to build the system.

---

Stateless: The Memoryless Responder

In a stateless architecture, every request is self-contained. The client doesn't just ask for something—it carries proof of who it is and the complete context needed to answer.

Think of it like a library where no patron has an account. Every time you visit, you must bring:

• Your ID
• Your borrowing history (so the librarian can see what you've already checked out)
• Your membership status
• Any notes about holds or restrictions

The librarian processes your request, hands you what you need, and the moment you leave, they discard everything about you. Next visit, you bring the same proof again.

How It Works

In practice, stateless often means:

// Client carries a JWT (JSON Web Token) that contains user info, permissions, timestamps
const token = eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOjEyMzQ1LCJyb2xlIjoiYWRtaW4iLCJleHAiOjE2OTk5OTk5OTl9.signature;

// Every request includes it
fetch('https://api.example.com/orders', {
  headers: {
    'Authorization': `Bearer ${token}`
  }
});

// Server validates the token (signature check, expiration, claims)
// Server answers the request using only what's in the token
// Server never stores session state

Any server in your fleet can answer because no state lives on the server—it all travels in the request.

The Advantage: Horizontal Scaling

If you have ten servers and a request lands on server #3, but #3 is overloaded, the load balancer can silently redirect the next request to server #7. Neither server has met you before. Neither cares. Both servers validate the same token and answer.

You add servers without needing to:

• Copy session data between nodes
• Pin requests to sticky sessions
• Worry about which server holds your state
• Rebuild state after a node crashes

This is why stateless dominates the edge, APIs, and microservices.

The Cost: Bigger Payloads and Repeated Work

But nothing is free.

Proof re-sent every time: If your token is 2KB and you make 100 requests per session, you're pushing 200KB of proof over the network that the server already validated once. Multiply that across millions of users.

State must live elsewhere: Real continuity—a shopping cart, a multi-step wizard, a live database transaction—can't live in a token. Tokens expire, get revoked, and are immutable once issued. So stateful data migrates to:

• A database (slower to query on every request)
• A cache like Redis (faster, but introduces a dependency)
• The client, encoded in the token (works only if state is small and doesn't need to change server-side)

Repeated computation: If your token says "user_id: 42", the server must re-fetch that user's permissions, preferences, and rights on every single request. Stateful systems might cache this in memory.

---

Stateful: The Remembering Server

Stateful is the opposite. The server remembers you. It stores your session in memory (or sometimes disk), opens a connection, and maintains continuity.

The library patron now has an account. The librarian recognizes you, pulls up your file, and knows exactly what you've borrowed, what holds you have, and whether you owe fines.

How It Works

// Client connects and gets a session ID
POST /login
body: { username, password }
response: { sessionId: "abc123xyz" }  // Stored in memory on server

// Client sends sessionId on future requests
GET /orders
headers: { 'Cookie': 'sessionId=abc123xyz' }

// Server looks up sessionId in its in-memory store
const session = sessionStore['abc123xyz'];
// Picks up where it left off—user already logged in, permissions cached, state known

The session object lives on a single server. That server becomes your "home" for this session.

The Advantage: Natural Continuity

For anything ongoing—a WebSocket connection, a long transaction, a multi-request workflow—stateful is simpler and faster:

• State is already in memory (no database lookup)
• The server knows your context without re-validating
• Connection persistence keeps overhead low
• Perfect for real-time: gaming, live chat, collaborative editing

The Cost: Immobility and Fragility

But the price is steep:

Sticky sessions: Once you're assigned to server #3, you're pinned there. If server #3 is slow or CPU-bound, tough luck—you can't move. Load balancing becomes uneven. One slow client can occupy one server while others sit idle.

No free horizontal scaling: You can't just add servers. You'd need to either:

• Replicate session state across servers (expensive, complex)
• Use a session store (Redis) to share state, which defeats the point and reintroduces the lookup cost

Failover is hard: If server #3 crashes, your session dies with it. You're logged out. You lose in-flight data. The user must reconnect and start over.

Cold start penalty: New servers don't have any session data cached, so they're slower until they warm up.

---

Comparison: Side by Side

Dimension	Stateless	Stateful
Server Memory	No user state stored	Session lives in memory
Per-Request Cost	Validate token, maybe fetch from DB/cache	Lookup session in memory (fast)
Scaling	Any server answers any request	Client pinned to one server (sticky)
Load Balancing	Round-robin, least-connections (free choice)	Must use affinity, uneven distribution
Payload Size	Larger (token included)	Smaller (just session ID)
Failover	Seamless (state not on server)	Hard (state dies with node)
Ideal For	APIs, microservices, public endpoints	Real-time, connections, complex workflows

---

The Hybrid Reality: Most Systems Use Both

You don't have to choose one or the other. In fact, the most robust systems use both.

Edge & APIs: Stateless
Public-facing REST endpoints are stateless. They validate JWTs, answer the request, forget you.

Internal Continuity: Stateful
A WebSocket connection for real-time notifications? Stateful. A multi-second database transaction? Stateful. A live collaboration session? Stateful.

Shared State: External Store
When stateful data needs to survive a restart or move across servers, it lives in Redis, memcached, or a database—not in a token, not in a single server's memory.

// Stateless edge accepts request with token
GET /user/42/orders
headers: { Authorization: 'Bearer eyJ...' }

// Token is validated (stateless)
// Permission check cached in Redis (external store)
// Order list fetched from database
// Response sent
// Server forgets you

// But if user opens a WebSocket for live order updates
WS wss://api.example.com/orders/stream
// Connect to a stateful server
// Server subscribes to order changes in Redis Pub/Sub
// Server pushes updates to client via WebSocket
// When server crashes, client reconnects to another server
// Redis holds the state, not the server

---

Reframing for LLM Inference

If you're running an LLM inference service, this distinction maps cleanly to request patterns:

Stateless inference:
Every request includes the full prompt and model context. The inference server processes it and returns an answer. No session or conversation history stored on the server. You can route requests to any available GPU. This is how most LLM APIs work (OpenAI, Anthropic, etc.).

Stateful inference:
A client opens a connection and runs a multi-turn conversation. The server keeps the conversation history in memory, enabling lower latency for follow-up requests (you don't re-encode the full conversation context each time). But now that conversation is pinned to one GPU. If that GPU fails, the conversation is lost. This model works for some chat applications but requires either session replication or external state storage (vector DB) to be production-safe.

The stateless model dominates because it simplifies scaling and fault tolerance—you can batch requests across many GPUs and lose a GPU without losing a conversation.

---

The Verdict

Reach for stateless when you're building APIs, public endpoints, or systems that must scale horizontally without complexity. Reach for stateful only when continuity demands it—real-time connections, complex transactions, or scenarios where re-validating on every request is prohibitively expensive. In production systems, combine both: stateless at the edge, stateful where needed, and push persistent state to an external store.

Watch the 90-second reel for a quick mental model: stateless = any barista can serve you, but you bring your order every time. Stateful = same barista remembers your order, but you're stuck with that barista.