In part 5 of our System Design series, we’re focusing on how systems behave under load and concurrency challenges. These are critical topics that often trip up engineers in interviews and production.
We’ll cover:
- Concurrency – Locks, MVCC, retries
- Multithreading – Pools, contention, switching
- Idempotency – Same request, same effect
- Rate Limiting – Fairness and protection
1. Concurrency
TL;DR: Multiple operations happening at once? Protect shared data and avoid races.
- Locks (mutexes): Simple but can cause deadlocks and performance issues.
- MVCC (Multi-Version Concurrency Control): Every write creates a new version; readers never block writers.
- Retries: Optimistic concurrency by retrying on conflict.
👉 Example: Bank transfer between accounts uses locks to prevent double spending.
👉 Interview tie-in: "How do you handle concurrent writes to the same data?" — Answer with locking strategies or MVCC.
2. Multithreading
TL;DR: Parallelism needs control.
- Thread pools: Pre-allocated worker threads to handle tasks.
- Contention: Avoid too many threads competing for the same resource.
- Context switching: Expensive if overused.
👉 Example: Web server uses a thread pool to handle incoming HTTP requests, limiting concurrency to prevent overload.
👉 Interview tie-in: "Why not just spawn a new thread per request?" — Explain thread pool and resource limits.
3. Idempotency
TL;DR: Repeating the same request should have the same effect.
- Why important: Retries due to network errors should not cause duplicate side effects.
👉 Example: HTTP PUT /users/123/profile is idempotent; POST /orders is not, unless you provide a unique order ID.
👉 Interview tie-in: "Design a payment API that handles retries safely." — Focus on idempotent design using unique transaction IDs.
4. Rate Limiting
TL;DR: Prevent abuse and manage fair usage.
-
Strategies:
- Fixed window (e.g., 100 requests per minute)
- Sliding window
- Token bucket (allows bursts)
👉 Example: Twitter limits API usage per developer key.
👉 Interview tie-in: "How do you prevent a single client from overloading your system?" — Implement rate limiting with sliding window or token bucket.
✅ Takeaways
- Protect shared data using locks or MVCC
- Use thread pools to control parallelism
- Ensure APIs are idempotent to handle retries
- Implement rate limiting to avoid abuse and spikes
💡 Practice Question:
"Design an API for sending SMS messages. How do you ensure idempotency and rate limits are enforced?"
Top comments (0)