Muhammad Hassan Obaid

Posted on Oct 31

𝑫𝒂𝒚 14: 𝑺𝒚𝒔𝒕𝒆𝒎 𝑫𝒆𝒔𝒊𝒈𝒏 — 𝑺𝒕𝒂𝒕𝒆𝒍𝒆𝒔𝒔 𝒔𝒚𝒔𝒕𝒆𝒎𝒔 - 𝑷𝒂𝒓𝒕 2

#architecture #backend #scalability #systemdesign

System Design Day — Stateless Systems

Series: System Design Day | Level: Beginner → Experienced | Reading time: 7–9 minutes

01. Understand the Problem

Context — What is a Stateless System?

A stateless system is a system in which a server does not retain client-session information between independent requests. Every request must carry the information needed to process it, or the server must retrieve that information from an external source.

In simple words: the server remembers how to process your request, not what you did in your previous request.

Problem Statement — Why Does It Exist?

Imagine an application running on a single server. When a user logs in, the server stores their session in its own memory.

Now imagine the application grows and starts running on five servers.

A user logs in through Server A, but their next request reaches Server C.

Server C does not know that the user has already logged in because Server A holds their session.

This creates a problem: how do we distribute requests across multiple servers without depending on which server handled the previous request?

Stateless architecture helps solve this problem by removing the dependency on locally retained client-session state.

Why It Matters

Without a suitable stateless design:

Requests may depend on a particular server.
Scaling horizontally becomes more complicated.
Server failures can interrupt user sessions.
Session synchronization introduces additional complexity.

Requirements & Constraints

A well-designed stateless application should:

Process independent requests without relying on previous server-local session data.
Authenticate and authorize requests.
Retrieve persistent information from appropriate storage.
Allow requests to reach different application instances.
Consider the additional payload, network and storage costs of transferring or retrieving state.

Scope & Assumptions

This article focuses on stateless application services and client-server communication, not on the claim that every component in an entire system must be stateless.

02. Build the Foundations

Core Concept — Stateless vs Stateful

Consider two different approaches to remembering a user's identity.

Stateful approach: The server remembers the user's session and associates future requests with that session.

Stateless approach: The client sends an authentication credential with each request. The server validates it independently.

Think of an airport.

In a stateful approach, imagine every checkpoint having to contact the same employee who checked you in to confirm your identity.
In a stateless approach, you present your boarding pass at each checkpoint. Each authorized checkpoint can independently validate it.

The boarding pass carries information, but the airline still maintains the underlying passenger and flight records.

This is an important distinction: stateless does not mean there is no stored information.

Fundamental Principles

Request Independence: A request should not depend on a previous request being processed by the same application instance.
Externalized State: Persistent information is maintained in databases, caches or other appropriate storage systems.
Self-Contained Requests: Requests carry the required context, such as authentication credentials and resource identifiers.
Instance Interchangeability: Any healthy application instance should be capable of processing an eligible request.

Mental Model

Think of a restaurant where every waiter can independently take your order using your table number.

The waiter does not need to be the same person who served you previously. The restaurant's ordering system maintains the actual order information.

The waiter represents the application server; the ordering system represents external state storage.

Key Components

Client: Sends requests containing the necessary context.
Load Balancer: Distributes requests among available application instances.
Application Servers: Process requests without depending on previous client-session state held locally.
Authentication Mechanism: Validates the client's credentials.
Database: Stores persistent application data.
Cache: Provides faster access to frequently requested information.

How It Works

Consider a mobile application using a REST API and JWT authentication.

The user logs in with their credentials.
The authentication service verifies the credentials and issues an access token.
The client stores the token securely.
The client includes the token in subsequent requests, typically through the HTTP Authorization header.
Any application server receiving the request validates the token.
The server retrieves any required application data from external storage.
The server processes the request and returns a response.

The next request can reach a different application server without requiring the previous server to remember the user's session.

Types & Classifications

Two common approaches to managing client-session state are:

Stateful	Stateless
Server retains client-session information.	Server does not retain client-session information between requests.
Requests may depend on session storage.	Requests carry the required context or reference external state.
May require shared session storage or sticky sessions when scaled.	Application instances are easier to distribute interchangeably.
Session management requires coordination.	Context transfer and external state management are important.

Important: These are approaches to session management, not mutually exclusive classifications of entire applications. A system can combine both.

Terminology

Session: A period of interaction between a client and an application.
Session ID: An identifier used to associate requests with a server-managed session.
JWT (JSON Web Token): A token format that can carry signed claims.
Horizontal Scaling: Increasing capacity by adding more instances.
Externalized State: Storing application state outside an individual application instance.
Sticky Sessions: A load-balancer configuration that directs a client to the same server across requests.
Idempotency: A property in which repeating an operation has the same intended effect as performing it once.

03. Design the System

Architecture — A Typical Stateless Application

A common architecture contains:

Client or mobile application.
Load balancer.
Multiple application servers.
External authentication or token-verification capability.
Shared database.
Optional distributed cache.

Components & Responsibilities

The load balancer distributes requests. Application servers independently process them. The authentication mechanism verifies credentials, while the database and cache provide persistent or temporary application data.

The important architectural decision is that no application instance needs to remember the client's previous request.

Component Interaction & Request Lifecycle

Suppose a user requests their profile:

The mobile application sends GET /api/profile.
It includes its access token.
The load balancer forwards the request to Server B.
Server B validates the token.
Server B retrieves the user's profile from the database or cache.
Server B returns the profile.

On the next request, the load balancer may select Server D. The request can still succeed because Server D has the necessary context.

Data Flow

Client → Load Balancer → Application Server → Authentication Validation → Database/Cache → Response.

API Design

A typical request might look like:

GET /api/profile

Authorization: Bearer <access_token>

The server receives the credential with the request instead of relying on a previously established local session.

Data Modeling & Storage

Keep responsibilities separate:

Authentication credentials establish the client's identity.
Database records hold persistent business information.
Cache entries improve retrieval performance.
Application servers handle request processing.

Statelessness does not eliminate database relationships, transactions or persistent business state.

Architecture Diagram

The architecture should demonstrate that multiple application instances can process requests from the same client without session affinity.

04. Make Engineering Decisions

Patterns & Techniques

Common approaches that can support stateless application services include:

Token-Based Authentication: Send access tokens with requests instead of relying on server-local sessions.
External Session Storage: Store session data in shared infrastructure such as Redis. This enables multiple application instances to access session information, although the application remains session-based rather than strictly stateless.
Immutable Infrastructure: Replace instances instead of modifying them in place. This supports predictable deployments but is not itself a statelessness pattern.
Stateless Serverless Functions: Process individual invocations while storing persistent information in external services.

Algorithms

Statelessness does not prescribe a particular algorithm. However, load-balancing algorithms such as Round Robin and Least Connections can distribute requests among application instances.

Alternatives

You can choose between:

Server-managed sessions.
Token-based authentication.
Shared session storage.
Sticky sessions.
Hybrid approaches.

Decision Rationale

Choose stateless application services when independent request processing, horizontal scaling and instance replacement are valuable.

Choose server-managed sessions when centralized session control, immediate session invalidation or simpler browser-session management is more important, provided the infrastructure supports the required scaling.

Pros, Cons & Trade-offs

Advantages

Easier horizontal scaling.
Reduced dependency on individual application instances.
Simpler instance replacement and failover.
Better flexibility in distributing requests.

Disadvantages

Credentials or context must accompany requests.
Token validation introduces processing overhead.
Revocation and authorization changes require careful handling.
Multi-step workflows may need external state storage.
External dependencies can become bottlenecks or failure points.

When to Use / When NOT to Use

Use when:

Building RESTful APIs.
Supporting large numbers of independent client requests.
Running multiple interchangeable application instances.
Building independently invoked serverless functions.

Think carefully when:

Managing long-running workflows.
Maintaining real-time connections.
Handling multi-step transactions.
Requiring immediate session revocation.
Processing operations that depend on accumulated state.

These situations do not prohibit stateless architecture; they require additional state-management strategies.

05. Think About Production

Scalability

Stateless application instances can be added or removed without migrating their local client-session information. A load balancer can distribute incoming requests across healthy instances.

However, the database, cache and other shared dependencies must also handle the increased workload.

Performance

Stateless requests may carry additional data, such as access tokens. Token validation and external data retrieval also consume resources.

For example, verifying a JWT locally using a trusted signing key can avoid a database lookup for every authentication check. However, retrieving current permissions or checking revocation may still require external coordination.

Reliability & Availability

If Server A crashes, another healthy instance can process a new independent request.

But this does not guarantee successful recovery of an operation that was interrupted halfway through. Retries, idempotency and durable data storage are still necessary.

Consistency & Concurrency

Stateless application servers do not eliminate concurrency problems.

Two requests can still update the same database record simultaneously. Transactions, optimistic locking, constraints and other concurrency-control mechanisms may be needed.

Security

Always use HTTPS.
Validate token signatures and claims.
Check expiration and intended audience.
Protect signing keys.
Store client-side credentials securely.
Implement appropriate token revocation and refresh strategies.
Never treat a valid identity token as automatic authorization for every resource.

Observability

Monitor authentication failures, request latency, error rates, application instance health and external dependency performance.

Use correlation IDs and distributed tracing to investigate requests across multiple services.

Cost & Maintainability

Stateless instances can simplify autoscaling and deployment, but external databases, caches, token infrastructure and networking still introduce operational and financial costs.

06. Think Beyond the Happy Path

Failure Scenarios

What happens if:

The database becomes unavailable?
The token expires during a user's workflow?
A signing key is rotated?
An application instance crashes halfway through processing a request?
Redis becomes unavailable?
A client retries a payment request?

Edge Cases

Consider concurrent updates, duplicate requests, expired credentials, permission changes and requests containing incomplete context.

Bottlenecks

Stateless application servers do not guarantee that the whole system scales.

The database, authentication service, network, cache or external API may become the actual bottleneck.

Failure Modes & Recovery Strategies

For example, if an application server crashes, a new request can be routed to another healthy instance.

For interrupted operations, use durable processing, carefully designed retries and idempotency mechanisms where appropriate.

Gotchas

Statelessness does not mean an application has no state.
JWTs are not automatically secure just because they are signed.
A JWT can remain valid until expiration even after a user logs out unless revocation or other controls are implemented.
Externalizing sessions does not make session-based authentication strictly stateless.
Sticky sessions can reintroduce dependency on a particular application instance.
Statelessness does not eliminate database consistency or concurrency problems.

Anti-patterns

Storing client-session information in application-instance memory while expecting requests to move freely between instances.
Putting sensitive information in readable JWT payloads.
Treating every shared storage dependency as infinitely scalable.
Retrying non-idempotent operations without safeguards.

07. Learn From Engineering Experience

Production Example

Consider an e-commerce application with three application servers.

A customer logs in, browses products, adds items to their cart and proceeds to checkout.

The application servers independently handle API requests. Authentication credentials are sent with each request.

However, the shopping cart is persistent application state, so it is stored in a database or shared cache rather than relying on the memory of whichever server handled the previous request.

This is a stateless application architecture managing stateful business data.

Industry Case Study

Large cloud-hosted API platforms commonly use horizontally scaled application instances behind load balancers.

Their individual application instances can be replaced or scaled independently, while databases, caches and other infrastructure maintain the required shared state.

The important lesson is not that every component must be stateless. It is that each component should have clearly defined state ownership and lifecycle.

Senior Engineer Insights

Experienced engineers ask more than whether a service is stateless.

They ask:

Where does the state actually live?
Who owns it?
How is it shared?
What happens when it becomes unavailable?
How is it recovered?
What happens when two requests modify it simultaneously?

Best Practices

Keep application-instance memory free of essential client-session dependencies.
Externalize persistent business state.
Use short-lived access tokens where appropriate.
Implement authorization independently of authentication.
Design retryable operations carefully.
Monitor shared dependencies.
Avoid introducing distributed infrastructure without a genuine requirement.

Fact Check

Common claim	Reality
Stateless means no data is stored.	False. Persistent state can exist externally.
Stateless applications automatically scale.	False. Shared dependencies can limit scaling.
JWT makes an application completely stateless.	Not necessarily. Revocation and authorization may require shared state.
HTTP is stateless.	HTTP's request semantics are stateless, but applications can maintain sessions over HTTP.
Microservices are inherently stateless.	False. Services can maintain their own persistent state.

Big-Word Alert

Session Affinity: A load balancer consistently routes a client to the same application instance. This can simplify local session handling but reduces flexibility when distributing requests.

Externalized State: Moving state out of an individual application instance into shared infrastructure.

Idempotency: Ensuring that repeating an operation does not produce unintended duplicate effects.

5 Whys — Why Do Stateless Systems Scale More Easily?

Why can requests reach different servers? Because they do not depend on locally retained client-session information.
Why does that help? Because servers can process requests independently.
Why is that useful? Because additional application instances can be introduced without transferring local sessions.
Why does this simplify operations? Because instances can be replaced, restarted and scaled independently.
Why does that matter? Because infrastructure can respond more flexibly to changing traffic and failures.

Asides

Statelessness is closely related to the REST architectural constraint of stateless communication.

However, statelessness alone does not make an API RESTful. REST has additional architectural constraints.

Tools & Technologies

Redis: Shared session storage or distributed caching.
JWT libraries: Token generation and verification.
Nginx: Reverse proxy and load balancing.
AWS Lambda: Event-driven serverless execution.
PostgreSQL: Persistent relational data storage.

Evolution of Design

A small application might begin with one application server and a database.

As traffic grows, the application can move to multiple instances behind a load balancer.

As session management becomes important, the team can choose token-based authentication or shared session storage.

As workloads grow further, caching, background processing and database scaling can be introduced independently.

Engineering Decision Check

Whenever you introduce a new architectural component, ask:

→ What problem does it solve?

→ When would I use it?

→ What new problem does it create?

→ How would I know it is failing?

08. Test Your Engineering Thinking

Active Recall

Define a stateless system.
What is the difference between stateful and stateless session management?
Why does statelessness simplify horizontal scaling?
Does statelessness mean that a database is unnecessary?
What is the purpose of sending an authentication token with each request?

Conceptual & Interview Questions

Why is HTTP considered a stateless protocol even though applications can maintain sessions?
Does using JWT automatically make an application stateless?
What is the difference between externalized session storage and stateless authentication?
Why might a team intentionally choose stateful session management?

Scenario-Based Questions

Scenario 1 — The Unexpected Logout

A user logs in through Server A. Their next request reaches Server B and returns an unauthenticated response.

What could be wrong, and how would you investigate it?

Scenario 2 — The Payment Problem

A stateless API receives a payment request. The payment succeeds, but the server crashes before returning a response. The client retries the request.

How would you prevent the customer from being charged twice?

Scenario 3 — The Scaling Problem

Your application has ten stateless application instances, but response times increase as traffic grows.

How would you investigate whether the database, cache, authentication process or network is responsible?

What-If Questions

What if the access token expires while the user is completing checkout?
What if a user's account is disabled but their JWT is still valid?
What if Redis becomes unavailable?
What if two requests simultaneously update the same shopping cart?

Debugging Challenge

Your application uses JWT authentication and three application instances. Users are randomly receiving 401 Unauthorized responses.

Identify at least five possible causes before proposing a fix.

Trade-off Questions

JWT authentication vs server-managed sessions: which would you choose for a browser application?
Stateless services vs sticky sessions: what are the operational implications?
Shared Redis sessions vs self-contained access tokens: what are the trade-offs?

Design Challenge

Design a stateless backend architecture for an e-commerce mobile application.

Requirements:

Support 100,000 registered users.
Run multiple application instances.
Authenticate API requests.
Maintain shopping carts across devices.
Process payments safely.
Remain available when an application instance crashes.

Draw the architecture, identify where state lives, explain how requests flow, and justify your design decisions.

Architecture Review

A developer proposes the following architecture:

"Every application server stores its own user sessions in memory. We will add a load balancer to distribute traffic randomly. If a server crashes, users can simply send their next request to another server."

Identify the architectural weakness. What changes would you recommend?

09. Final Revision

Final Mental Model

Statelessness is about where client-session state is maintained, not about eliminating all state from an application.

A stateless application instance can independently process a request because it receives the necessary context and retrieves persistent information from appropriate external sources.

Key Takeaways

Requests should not depend on previous server-local client-session state.
Stateless application instances are easier to scale and replace.
Databases and caches can still maintain application state.
JWT is one possible authentication mechanism, not a requirement for statelessness.
Statelessness does not eliminate security, consistency, concurrency or failure-management challenges.

Decision Checklist

Before designing a stateless service, ask:

Can any healthy instance process the request?
Where does persistent state live?
How is authentication validated?
How are duplicate operations handled?
What happens when shared dependencies fail?
Can the application scale without introducing hidden dependencies?

Knowledge Connections

Statelessness connects several fundamental system design concepts:

Load Balancing: Requests can be distributed across interchangeable instances.
Horizontal Scaling: New instances can join without inheriting local client sessions.
Distributed Systems: State ownership and coordination become important.
Authentication: Each request must establish the necessary identity and authorization context.
Fault Tolerance: New requests can be handled by healthy instances after an instance failure.

Final Thought: Don't ask whether your application is stateless. Ask whether any individual application instance is holding state that other instances need in order to process the next request.