Every e-commerce team dreams of massive traffic spikes—until those spikes actually happen.
Imagine this: You’ve spent weeks (maybe months) building your e-commerce backend. Your API endpoints are clean, your database schema is normalized, and your authentication flow is solid. You test it with 10 users. It works. You test with 100. It still works.
Then Black Friday hits. Or your product goes viral on TikTok. Or a celebrity tweet about your app. Suddenly, it’s no longer just one request (or even a hundred)—it’s 10,000, all landing on the same endpoint within the same second, all fighting over the same rows in your database.
Your server slows to a crawl. Some users get errors. Others get charged twice. A lucky few get their orders, but the rest get nothing.
This is the exact moment most engineers discover that “it works on my machine” and “it works under concurrency” are two completely different claims.
What Went Wrong?
This isn’t just a “traffic spike” problem. It’s a concurrency, architecture, and systems design problem. If you’re building scalable backends—especially in Node.js, NestJS, or any modern stack—you need to understand exactly what happens under the hood when thousands of requests hit your API simultaneously.
The Mental Model: One Server, Thousands of Requests
Let’s start with a simple question: Does your server create 10,000 copies of itself when 10,000 users hit it at once?
No. It doesn’t.
What actually happens is far more interesting—and more fragile.
Your server has limited resources: CPU cores, memory, network bandwidth, and database connections. When a flood of requests arrives, your server doesn’t magically scale up. Instead, it tries to juggle those requests using the tools and patterns you’ve (hopefully) built into your architecture.
In Node.js, for example, there’s only one main JavaScript thread—but that doesn’t mean it handles requests one at a time. It handles them concurrently, not in parallel.
Concurrency vs. Parallelism: The Core Distinction
This is where most developers get tripped up.
- Concurrency means multiple requests are in progress at the same time but not necessarily executing at the same time.
- Parallelism means multiple requests are actually executing at the same time, on different CPU cores or threads.
A common misconception about Node.js is that because it is single-threaded, high traffic will cause requests to pile up like cars stuck at a single toll booth.
In reality, Node.js is single-threaded but highly concurrent. It uses an event loop to delegate I/O-bound tasks (like database queries, file reads, or external API calls) to the OS or thread pool, then moves on to the next request without waiting.
So, when 10,000 users hit your “Buy Now” endpoint:
Each request enters the event loop.
If the request involves a database query, Node.js offloads that work and immediately starts handling the next request.
When the database responds, Node.js picks up where it left off and sends the response.
This is why Node.js can handle thousands of concurrent connections—even on a single thread.
But here’s the catch: if your code blocks the event loop, everything grinds to a halt.
Where Things Actually Break: The Bottlenecks
Under heavy load, your API doesn’t just fail for no reason—it fails because of bottlenecks. Here are the most common ones:
- Blocking the Event Loop - If your code does something CPU-heavy in JavaScript—like encrypting passwords synchronously, parsing huge JSON payloads, or running complex calculations—it blocks the event loop. While that one request is hogging the thread, all other requests wait. This is why you should never use fs.readFileSync or heavy crypto in production code.
- Database Connection Pool Exhaustion – Your database doesn’t have infinite connections. Most connection pools are configured to handle 10–100 concurrent connections. If 10,000 requests each try to open a new connection, your pool gets exhausted. Requests start queuing—or worse, failing with “too many connections” errors.
- Unindexed or Slow Queries – A single slow query can block hundreds of requests waiting on the same table. Under load, this becomes a cascade failure: one slow query → connection pool fills up → new requests timeout → users see errors.
- External API Calls Without Timeouts – If your “Buy Now” endpoint calls a payment gateway, inventory service, or shipping API—and those calls don’t have timeouts—your server can get stuck waiting for responses that never come. Multiply that by 10,000 requests, and you’ve got a server meltdown.
- Memory Pressure and Garbage Collection – Under heavy load, your server allocates more memory. If you’re not careful, you can trigger frequent garbage collection pauses, which block the event loop and spike latency.
The Hidden Threat: Race Conditions
One of the most dangerous—and subtle—problems that emerges under high concurrency is the race condition. This occurs when two or more requests try to modify the same data simultaneously, and the final result depends on the unpredictable order in which the operations complete.
Consider this classic e-commerce example:
Without proper safeguards, two users can both be told an item is in stock, both complete checkout, and both be charged—even though only one unit was available. This is a race condition in its purest form, and it’s exactly why you need the strategies we’ll cover next.
So, What Actually Happens When 10,000 Users Click “Buy Now”?
Let’s walk through the sequence step by step.
Step 1: Requests Hit the Load Balancer Your traffic doesn’t hit a single server—it hits a load balancer (like AWS ALB, NGINX, or Cloudflare). The load balancer distributes requests across multiple backend instances. If you don’t have a load balancer, all 10,000 requests hit one server. That server will likely crash.
Step 2: Each Request Enters the Event Loop On each backend instance (say, a Node.js server), requests enter the event loop. If your code is non-blocking, the server can handle thousands of concurrent requests per instance.
Step 3: Database Queries Are Executed Each request tries to:
Check inventory
Reserve the item
Process payment
Create an order record
If your database isn’t optimized (no indexes, no connection pooling, no read replicas), this is where things slow down—and where race conditions can wreak havoc.
Step 4: Responses Are Sent Responses don’t go out in the order the requests arrived. They go out as soon as each request completes. Some users get their order confirmation in 200ms. Others wait 10 seconds. A few get timeouts.
How to Design for 10,000 Concurrent Users
You don’t need to over-engineer for day one—but you do need to design for scale. Here’s how:
1. Use Connection Pooling
Configure your database client (like pg for PostgreSQL or mysql2 for MySQL) to use a connection pool. This reuses connections instead of creating new ones for every request.
2. Database-Level Locking
The most direct fix lives at the database layer, using row-level locks. A SELECT ... FOR UPDATE statement tells the database: “Lock this specific row until my transaction finishes—nobody else can read or write it until I’m done.”
Any other transaction trying to touch row 42 simply blocks until this one commits or rolls back. This guarantees correctness, but it introduces lock contention—under extreme concurrency, requests start queuing up waiting for locks, and latency climbs. For a single hyper-popular SKU, this can become a serious bottleneck even though the rest of your database is sitting idle.
An alternative is optimistic locking, which skips the lock entirely and instead checks a version number at write time:
3. Decoupling with Queues
Locking solves correctness at the cost of making requests wait synchronously. A more scalable pattern is to decouple the write from the request entirely using a message queue (Kafka, RabbitMQ, AWS SQS, Google Pub/Sub).
Instead of your API handler directly touching the database:
The incoming request is validated and dropped onto a queue as a message: { userId, productId, requestId }.
The API immediately responds with something like 202 Accepted — processing.
A pool of background workers consumes messages off the queue one at a time (or a few at a time) at a rate the database can actually sustain.
Each worker executes the actual stock-check-and-deduct logic inside a transaction.
The result (success/failure) gets pushed to the client via websocket, polling, or a notification.
This converts a spike of 10,000 simultaneous writes into a controlled, sequential stream that the database can process without falling over. The tradeoff is eventual consistency—there’s a small delay between “I clicked buy” and “the system confirms it,” which needs to be communicated clearly in the UI (Spinners) so it doesn’t feel broken.
But here’s the reality check: Would you really place all 10,000 requests into Kafka? Not necessarily. Because if you only have 100 units in stock, 99% of those requests are doomed to fail from the start. Dropping them into a queue only wastes worker capacity, memory, and network bandwidth.
That’s why large-scale flash-sale systems move inventory protection closer to the edge. They preload inventory into a fast in-memory store (like Redis) and use it as a first gate. If Redis says stock = 0, the request is rejected immediately with a polite "Out of Stock" message, never even touching the queue or the primary database. The queue becomes a pathway only for requests that actually have a fighting chance, keeping your workers lean and your database unscathed.
Add a Caching Layer
4. Add a Caching Layer
For read-heavy traffic—like thousands of users just checking if a product is in stock before deciding to buy—a cache (Redis, Memcached, or a CDN edge cache) intercepts most of that traffic before it ever reaches the database. Re-running the same read query 10,000 times a second against your primary database is pure waste.
In high-concurrency “Buy Now” scenarios, Redis can go further and act as the first gate. Many flash-sale systems preload the available inventory into Redis at the start of the sale. When a request arrives, the first thing the API does is try an atomic decrement:
DECR product:42:stock
If the value was still positive, the request is allowed to proceed (and later confirmed in the database). If Redis returns a value less than zero, the request is rejected immediately with an “Out of Stock” response. It never reaches Kafka, never reaches the database, and never wastes a connection. This is extremely fast and protects the rest of your system.
But… What if Redis crashes after decrementing the stock?
This is why Redis should never be your source of truth. It’s a traffic-shaping layer - an early filter that keeps the stampede from overwhelming the slower, more authoritative systems behind it, not the final authority. The database remains the single source of truth for inventory. If Redis goes down mid-sale, you lose your fast gate, but your database can still fall back on its own row-level locks and atomic updates to guarantee correctness.
However, The classic hard problem remains: cache invalidation. A stale cache can tell users an item is available when it just sold out. That’s why the Redis inventory numbers are usually treated as a temporary, short-lived view that is reconciled with the database on a short interval or after every successful write.
5. Use Atomic Operations Instead of Read-Then-Write
A lot of race conditions disappear entirely if you avoid the "read, then write" pattern altogether and push the arithmetic into the database itself:
This single statement is atomic. The database guarantees that the check and the decrement happen together. Suppose the last item has stock = 1 and two requests arrive at almost the same instant:
Request A’s UPDATE succeeds. Stock becomes 0.
Request B’s UPDATE finds stock > 0 is now false, so zero rows are updated. The request is rejected.
No overselling. Only one transaction can successfully claim the final unit.
That’s the pure inventory decrement.
But here’s where it gets messy. What if you decrement the stock successfully, but the payment gateway times out? Or the user's credit card declines? Or they close their browser mid-checkout?
Imagine this: Stock is 1. User A reserves it. Your atomic UPDATE sets stock to 0. Then the payment fails. Now the product is completely unavailable—even though nobody actually bought it. You've just locked away your last item for a failed transaction. That's a terrible customer experience.
Large e-commerce platforms solve this by separating reservation from purchase. Instead of permanently deducting stock at the start of checkout, inventory first moves into a temporary, time-bound reserved state. The stock stays reserved while the user completes payment, but it's not permanently gone yet.
Only after a successful payment confirmation does the reservation convert into a permanent sale. If the payment fails, or if the user abandons the cart and walks away, the reservation simply expires—say, after 5-10 minutes—and the inventory is atomically released back into the available pool for the next customer.
This two-phase approach adds a bit of complexity (you now need a background job to sweep expired reservations), but it prevents the heartbreaking scenario where a legitimate buyer is told "out of stock" because someone else's credit card just bounced. It turns your atomic operation from a blunt instrument into a surgical tool—and your users will notice the difference.
6. Idempotency Keys
Under load, clients retry. Networks flake, timeouts happen, users double-click. Without protection, this creates duplicate side effects—double charges, double stock deductions.
The standard fix is an idempotency key: a unique identifier generated client-side (often a UUID) and attached to the request. The server stores which idempotency keys it has already processed and, on seeing a repeat, returns the cached original response instead of re-executing the logic:
Stripe/Paystack’s API is the canonical real-world example of this pattern done well—it’s specifically designed so that retried payment requests never double-charge a customer, as long as the same idempotency key is reused.
7. Rate Limiting and Backpressure
Not every request deserves to reach your database at full speed. Rate limiting—using algorithms like token bucket or sliding window—caps how many requests per second a given user, IP, or API key can send, rejecting or delaying the rest with a 429 Too Many Requests response.
This isn’t just about abuse prevention—it’s about protecting your own infrastructure. Related to this is backpressure: a signal that flows backward through a system telling upstream components “slow down, I can’t keep up,” so instead of a slow database causing your API servers to pile up thousands of stalled connections and crash, the system degrades gracefully by shedding or delaying excess load.
No Single Solution Solves the Entire Problem
No single technique here solves the whole problem. Locking without queuing creates painful contention. Queuing without idempotency creates duplicate charges. Caching without invalidation lies to users. And none of this work if you haven’t addressed the underlying race conditions that concurrency exposes.
It’s the combination—each one solving a different failure mode—that lets a system survive a genuine 10,000-request stampede without corrupting data or falling over.
Conclusion: Why This Matters for You
As a backend developer, your job isn’t just to write code that works. It’s to write code that works under pressure.
When you understand what happens when 10,000 users hit your API, you stop thinking in terms of “my code” and start thinking in terms of systems. You ask:
What happens if this endpoint gets 100x more traffic?
Where are the bottlenecks?
How do I fail gracefully?
What metrics should I monitor?
This mindset shift is what makes you valuable in remote job markets, freelance gigs, and high-growth startups.
Top comments (0)