Most performance dashboards look fine.
Average latency: low
CPU: stable
Memory: healthy
And yet users complain that “the app feels slow”.
This usually isn’t a mystery. It’s a metrics problem.
Averages hide pain
Imagine this system:
- 99% of requests complete in 10ms
- 1% of requests take 1000ms
The average latency is ~20ms.
Looks great on a chart.
But 1 out of every 100 users experiences a full second pause.
That’s not an edge case — that’s a real user.
Tail latency is the real signal
This is why percentiles matter:
- p50 tells you what’s typical
- p90 / p95 show rising variability
- p99 shows where systems actually break down
If your p99 is bad, your system feels bad — even if averages look perfect.
Jitter makes it worse
High variance is often more damaging than slow performance.
A stable 40ms system feels faster than one that jumps between 5ms and 200ms.
That variability — jitter — is what makes UIs stutter, audio glitch, and frames drop.
A small tool to see this clearly
I wanted a way to analyze raw timing data without dashboards, heavy dependencies, or full observability stacks.
So I built Latency Lens — a small CLI tool focused on:
- p50 / p90 / p95 / p99
- jitter (standard deviation and MAD)
- spike detection
- worst-case time windows
No averages-first thinking. Just tail behavior.
GitHub:
https://github.com/KinanNasri/latency-lens
PyPI:
https://pypi.org/project/latency-lens/
The takeaway
If users say your system feels slow:
- Don’t look at averages
- Look at p99
- Look at variance
- Look at spikes over time
That’s usually where the truth is.
Top comments (0)