When developers talk about performance, latency and throughput are often mentioned together. But they measure two completely different things.
Latency is how long one operation takes.
If you click a button and the API responds in 100ms, that is latency.
Throughput is how much work a system can handle over time.
If your API processes 10,000 requests per second, that is throughput.
The easiest way to remember it is:
Latency = How fast?
Throughput = How much?
Here is where things get interesting: a system can have high throughput and still feel slow.
Imagine your server processes thousands of requests every second, but each request takes 5 seconds to complete. Your system handles a lot of work, but users are still waiting.
On the other hand, an API might respond in 50ms but only handle 100 requests per second. It feels fast with a few users but may struggle when traffic increases.
So:
- Low latency means individual requests are fast.
- High throughput means the system handles more work.
- Good performance often requires both.
When optimizing latency, focus on things like faster database queries, caching, and reducing unnecessary network calls.
When optimizing throughput, focus on removing bottlenecks, scaling workers, and processing work efficiently.
The key takeaway?
A fast response does not mean your system can handle heavy traffic.
Handling heavy traffic does not mean your system gives users a fast experience.
Latency is about speed. Throughput is about capacity.
A good engineer knows which one the system needs most—and when it needs both.
Top comments (0)