DEV Community: Arnav Gupta

p50, p95, p99: What Latency Percentiles Actually Mean for Your Node.js App

Arnav Gupta — Thu, 25 Jun 2026 17:00:00 +0000

Your monitoring dashboard shows average response time: 45ms. Looks great.

Your users are complaining the app is slow.

Both things are true. Here's why.

Averages hide the worst experiences

Imagine 100 API requests. 99 of them take 10ms. One takes 5000ms.

Average: (99 × 10 + 5000) / 100 = 59.9ms

Your dashboard shows ~60ms average. Looks fine. But one in every 100 users waited 5 seconds. If you have 10,000 daily active users, that's 100 people per day getting a 5-second response.

Averages lie because they let the fast majority drown out the slow minority. Percentiles don't.

What percentiles actually mean

p50 (median): 50% of requests finish faster than this. This is your "typical" user experience. If your p50 is 20ms, most users feel your app as fast.

p95: 95% of requests finish faster than this. 1 in 20 requests is slower. On a busy API handling 1000 requests per minute, that's 50 slow requests every minute.

p99: 99% of requests finish faster than this. 1 in 100 requests is slower. This is where tail latency lives — GC pauses, cold caches, network spikes.

p99.9: 1 in 1000 requests is slower. For high-traffic systems, this still affects real users. For most apps, p99 is enough to care about.

The relationship between these numbers tells you a lot:

p50 ≈ p99: Your latency is very consistent. Simple workload, warm cache, no contention.
p99 is 10x p50: You have a significant tail. Something slow is happening for 1% of requests — worth investigating.
p99 is 100x p50: Something is seriously wrong. GC pauses, lock contention, or a broken dependency.

Why Node.js has a particular p99 problem

Node.js is single-threaded. Everything runs on one event loop. This means:

One slow operation blocks everything.

If a database query takes 2 seconds, no other requests can be processed during that time (unless you're careful with async). In a multi-threaded server, one slow query only affects that thread. In Node, it affects everyone.

This makes p99 disproportionately important for Node.js applications. Your p99 latency doesn't just affect 1% of users — it causes latency spikes for everyone who happens to be waiting behind a slow request.

The good news: Node's async model means you can handle thousands of concurrent requests efficiently when everything is fast. The bad news: the tail hurts more.

How to measure your actual percentiles

If you're running Express, add a percentile tracker:

const latencies = [];

app.use((req, res, next) => {
  const start = Date.now();
  res.on('finish', () => {
    latencies.push(Date.now() - start);
    // keep last 1000 only
    if (latencies.length > 1000) latencies.shift();
  });
  next();
});

function percentile(arr, p) {
  const sorted = [...arr].sort((a, b) => a - b);
  return sorted[Math.floor(sorted.length * p / 100)];
}

// log every minute
setInterval(() => {
  if (latencies.length === 0) return;
  console.log({
    p50:  percentile(latencies, 50),
    p95:  percentile(latencies, 95),
    p99:  percentile(latencies, 99),
    count: latencies.length
  });
}, 60_000);

Run this in production for a day. The gap between your p50 and p99 will probably surprise you.

The real-world numbers

Here's what realistic production p50/p95/p99 latency looks like for common dependencies:

Dependency	p50	p95	p99
Postgres (local)	2–5ms	20–50ms	100–300ms
Postgres (remote)	5–15ms	50–150ms	200–800ms
Redis	0.5–2ms	3–8ms	10–30ms
MongoDB	5–15ms	40–80ms	150–400ms
S3 GetObject	20–50ms	100–200ms	400–800ms
Stripe API	100–300ms	500–1000ms	1500–3000ms
OpenAI API	500–1500ms	2000–5000ms	5000–12000ms

Notice the ratios. Postgres p99 is often 40-60x the p50. Redis is more consistent (10-15x). External APIs like Stripe are the worst — p99 can be 10x the p50 and there's nothing you can do about it except handle it gracefully.

What to do with this information

Set timeouts based on p99, not p50.

If your DB p99 is 300ms, your timeout should be at least 400-500ms. Setting it at 100ms means you're timing out 1% of requests unnecessarily — and those timeouts trigger retries, which add load, which makes the DB slower, which triggers more timeouts.

Size your connection pools for the p99 case.

If a DB query takes 300ms at p99, and you're handling 100 concurrent requests, you need at least 30 connections in the pool (100 requests × 300ms / 1000ms). Most developers size pools for the p50 case and wonder why they get connection exhaustion under load.

Build retry backoff around p99, not p50.

Your first retry should wait at least as long as your p99. If you retry after 50ms but your p99 is 200ms, your retry often arrives while the original is still processing — doubling load.

Test with realistic latency.

This is the one most people skip. If you're testing retry logic, timeout handling, or circuit breakers against a flat setTimeout(fn, 200), you're not testing the variance — and the variance is what breaks things.

import { withLatency } from 'slowdep';

// simulate realistic postgres latency in your tests
const db = withLatency(mockDBCall, 'postgres'); // p50:5ms p99:200ms

// now your tests see the actual distribution
// some calls are 3ms, some are 150ms, some are 600ms
// just like production

The mental model

Think of latency like a commute. Your average commute might be 25 minutes. But:

p50: 22 minutes (typical day)
p95: 40 minutes (traffic, occasional delays)
p99: 75 minutes (accident, bad weather, everything went wrong)

If you tell someone "my commute is 25 minutes" and they schedule a meeting 30 minutes after you leave, you'll miss it 5% of the time. That's p95.

Your SLA, your retry timeouts, your user experience — they're all governed by your tail, not your average.

Build for the tail.

If you want to test your Node.js app against realistic latency distributions, I built slowdep — a zero-dependency package that wraps any async function with production-accurate p50/p99 sampling.

github.com/arnnnavvvvv/slowdep

Chaos Engineering for Node.js Without the Infrastructure

Arnav Gupta — Sat, 20 Jun 2026 21:23:30 +0000

Chaos engineering sounds expensive. Netflix built Chaos Monkey to randomly kill production servers. Google runs DiRT (Disaster Recovery Testing) across their entire infrastructure. Amazon does game days where they intentionally take down services.

You're building a Node.js API. You don't have a platform team. You don't have a chaos infrastructure. But you still need to know: what happens when your dependencies get slow?

The good news is that 80% of the value of chaos engineering comes from one question, and you can answer it locally in five minutes.

The one question that matters

What does my application do when a dependency responds slowly or not at all?

Not "what if the server catches fire" — that's infrastructure chaos. What about application chaos: the database is slow, the payment API is timing out, Redis is having a bad day. These happen constantly in production and they're almost never tested.

The failure modes look like this:

Your DB gets slow under load → your API response times climb → your timeout fires → you retry → now you're sending twice the load to an already-slow DB
Your Redis cache goes down → every request hits Postgres directly → Postgres gets slow → same cascade
Stripe's API takes 3 seconds instead of 200ms → your checkout route times out → users get errors → you're losing revenue

Every one of these is a latency failure, not a crash. The service is still up. It's just slow. And slow is the hardest failure mode to test because your local environment is fast.

Why local testing misses this

When you test locally, your "database" is either:

A real local Postgres running on the same machine (sub-millisecond latency, not production-like)
A mock that returns instantly (jest.fn().mockResolvedValue(data))
A fake with a flat delay (await sleep(200))

None of these produce realistic latency. A real production database has:

Fast responses most of the time (p50 ~5ms)
Occasional slowdowns (p95 ~50ms)
Rare but real spikes (p99 ~200ms, p99.9 ~2000ms)

The spikes are what break things. And the spikes only happen when the conditions are right — high load, cold cache, GC pause, noisy neighbor. You can't predict when they'll occur, only that they will.

Simulating realistic failure locally

The key insight: you don't need to kill servers to do chaos engineering. You need to inject realistic latency into your dependency calls.

Here's the minimal version you can write yourself:

function lognormalSample(p50, p99) {
  const mu = Math.log(p50);
  const sigma = (Math.log(p99) - mu) / 2.326;
  const u1 = Math.random(), u2 = Math.random();
  const z = Math.sqrt(-2 * Math.log(u1)) * Math.cos(2 * Math.PI * u2);
  return Math.exp(mu + sigma * z);
}

function withChaos(fn, { p50, p99, errorRate = 0 }) {
  return async function(...args) {
    const delay = lognormalSample(p50, p99);
    await new Promise(r => setTimeout(r, delay));

    if (Math.random() < errorRate) {
      throw new Error('simulated transient error');
    }

    return fn.apply(this, args);
  };
}

Or use slowdep which packages this with built-in presets:

npm install slowdep

import { withLatency, withLatencyAll } from 'slowdep';

// wrap individual functions
const slowQuery = withLatency(db.findUser, 'postgres');

// or wrap entire clients
const slowDB = withLatencyAll(prisma, 'postgres');
const slowCache = withLatencyAll(redis, 'redis');

The chaos test playbook

Here are the three tests every Node.js service should run before shipping:

Test 1: Timeout calibration

import { withLatency } from 'slowdep';

const db = withLatency(realDBCall, 'postgres'); // p99: 200ms

async function getUser(id) {
  const controller = new AbortController();
  const timeout = setTimeout(() => controller.abort(), 250); // your timeout

  try {
    const result = await db.findById(id);
    clearTimeout(timeout);
    return result;
  } catch (err) {
    if (err.name === 'AbortError') {
      throw new Error('DB timeout');
    }
    throw err;
  }
}

// run 1000 times and check your timeout hit rate
let timeouts = 0;
for (let i = 0; i < 1000; i++) {
  try {
    await getUser(i);
  } catch (err) {
    if (err.message === 'DB timeout') timeouts++;
  }
}

console.log(`timeout rate: ${(timeouts / 1000 * 100).toFixed(2)}%`);
// if this is above 1%, your timeout is too tight

Test 2: Retry storm detection

import { withLatency } from 'slowdep';

// simulate DB under stress — higher p99
const stressedDB = withLatency(realDBCall, { p50: 20, p99: 800, errorRate: 0.05 });

let totalCalls = 0;
const originalDB = stressedDB;

// count how many actual DB calls your retry logic makes
const instrumentedDB = async (...args) => {
  totalCalls++;
  return originalDB(...args);
};

// run 100 user requests through your retry logic
for (let i = 0; i < 100; i++) {
  try {
    await withRetry(() => instrumentedDB(i));
  } catch {}
}

console.log(`100 requests → ${totalCalls} DB calls`);
// if this is above 150, your retry logic is amplifying load

Test 3: Cascade failure simulation

import { withLatencyAll } from 'slowdep';

// simulate what happens when your cache is down
// and every request falls through to the DB
const noCache = withLatencyAll(redis, { p50: 500, p99: 3000, errorRate: 0.3 });
const db = withLatencyAll(prisma, 'postgres');

async function getUser(id) {
  try {
    const cached = await noCache.get(`user:${id}`);
    if (cached) return JSON.parse(cached);
  } catch {
    // cache miss — fall through
  }

  return db.user.findUnique({ where: { id } });
}

// does your app stay responsive when cache is degraded?
const start = Date.now();
await Promise.all(Array.from({ length: 50 }, (_, i) => getUser(i)));
console.log(`50 concurrent requests: ${Date.now() - start}ms`);

What you'll find

When you run these tests for the first time, you'll typically discover:

Your timeouts are too tight. Most developers set timeouts based on p50 latency. Your DB responds in 5ms usually, so you set a 100ms timeout. That's fine until p99 hits 200ms and 1% of requests start timing out and retrying.

Your retry logic increases load. Naive retry without backoff sends the same request again immediately. Under stress, this doubles your DB load at exactly the wrong moment.

Your circuit breaker never opens. If you have one. Most local tests never trigger it because the fake is too well-behaved.

Your error handling has gaps. Timeout errors, transient errors, and permanent errors need different handling. Flat fakes always produce the same error type.

This is local chaos engineering

You didn't need Chaos Monkey. You didn't need a dedicated chaos platform. You needed realistic latency in your test environment — which is just math and a 80-line package.

The infrastructure chaos stuff matters too, eventually. But if you haven't tested your application behavior under slow dependencies first, you haven't done the basics.

Start there.

slowdep on npm · github.com/arnnnavvvvv/slowdep

Why setTimeout is Lying to Your Retry Logic

Arnav Gupta — Wed, 17 Jun 2026 21:22:41 +0000

You've written retry logic. It probably looks something like this:

async function withRetry(fn, retries = 3) {
  for (let i = 0; i < retries; i++) {
    try {
      return await fn();
    } catch (err) {
      if (i === retries - 1) throw err;
      await new Promise(r => setTimeout(r, 200 * (i + 1)));
    }
  }
}

You test it locally. You simulate a slow dependency like this:

const fakeDB = async () => {
  await new Promise(r => setTimeout(r, 200)); // simulate DB
  return { id: 1, name: 'test' };
};

Your retry logic works. Tests pass. You ship it.

Then in production, your app starts dropping requests under load.

The problem isn't your retry logic. It's your fake.

Real dependencies don't have flat latency

Here's what your Postgres instance actually looks like in production:

p50: 5ms — half of all queries finish in under 5ms
p95: 50ms — 95% finish under 50ms
p99: 200ms — 99% finish under 200ms
p99.9: 2000ms — that one unlucky query during a GC pause

Your setTimeout(fn, 200) simulates the worst case, every single time. That's not how production works. And because it's not how production works, your retry logic has never actually been tested against reality.

The bugs hide in the variance — not in the slow case, but in the unpredictability.

What the real distribution looks like

Latency in distributed systems follows a lognormal distribution. It's right-skewed: most requests are fast, a meaningful minority are slow, and a small tail is very slow.

This shape comes from how real systems work:

GC pauses — Java, Go, and even Node's garbage collector occasionally stops the world
Cold caches — first query after a cache miss is always slower
Network jitter — packet routing isn't deterministic
Noisy neighbors — other workloads on the same hardware compete for resources
Connection pool exhaustion — when all connections are busy, new queries wait

None of these are constant. They're random, rare, and multiplicative — which is exactly what produces a lognormal shape.

Why this matters for retry logic specifically

Consider this scenario: your p99 latency is 200ms and your timeout is 250ms.

With setTimeout(fn, 200), every test call takes exactly 200ms — safely under your timeout. Tests pass.

In production, the lognormal tail means 0.1% of calls take 500ms or more. Your 250ms timeout fires, your retry triggers, and now you're sending the same request again to an already-stressed database. Under load, this cascades.

This is the exact failure mode that causes retry storms — and it only appears in production because your local tests used flat delays.

The bugs that flat delays hide:

Timeouts that are too tight for the real p99
Retry logic that amplifies load instead of handling it gracefully
Circuit breakers that never open during tests but open constantly in production
Backoff strategies that feel correct locally but collapse under real variance

The fix: simulate real latency distributions

Instead of a flat delay, fit a lognormal distribution to real p50/p99 values and sample from it. Every call gets a different delay — most are fast, some are slow, a few are very slow. Just like production.

Here's the math:

function fitLognormal(p50, p99) {
  // p50 = median = e^mu  →  mu = ln(p50)
  // p99 = e^(mu + 2.326*sigma)
  const mu = Math.log(p50);
  const sigma = (Math.log(p99) - mu) / 2.326;
  return { mu, sigma };
}

function sampleLatency(p50, p99) {
  const { mu, sigma } = fitLognormal(p50, p99);
  // Box-Muller transform
  const u1 = Math.random(), u2 = Math.random();
  const z = Math.sqrt(-2 * Math.log(u1)) * Math.cos(2 * Math.PI * u2);
  return Math.exp(mu + sigma * z);
}

Call sampleLatency(5, 200) ten times and you'll get something like:

3ms, 7ms, 2ms, 12ms, 4ms, 180ms, 6ms, 3ms, 9ms, 440ms

That's what your database actually looks like.

Using slowdep

I built slowdep to make this a one-liner. It wraps any async function with a lognormal latency profile — either a built-in preset or your own p50/p99 values.

npm install slowdep

import { withLatency } from 'slowdep';

// before: flat fake
const fakeDB = async (id) => ({ id, name: 'test' });

// after: realistic latency
const fakeDB = withLatency(async (id) => ({ id, name: 'test' }), 'postgres');

Now run your retry logic against it:

const result = await withRetry(() => fakeDB(42));

You'll immediately see things you didn't see before:

Some retries succeed on the second attempt (realistic)
Occasional calls hit your timeout (revealing tight timeouts)
Rare calls cascade into all retries failing (revealing missing backoff jitter)

Built-in presets cover the most common dependencies:

Preset	p50	p99	Error rate
`'postgres'`	5ms	200ms	0.1%
`'redis'`	1ms	20ms	0.05%
`'stripe'`	200ms	2000ms	0.2%
`'openai'`	800ms	8000ms	0.5%
`'s3'`	30ms	500ms	0.1%

You can also pass custom profiles:

const slowFetch = withLatency(fetchAPI, {
  p50: 100,
  p99: 3000,
  errorRate: 0.02, // 2% transient errors
});

The real test

Here's what testing retry logic actually looks like with realistic latency:

import { withLatency } from 'slowdep';

// realistic postgres simulation
const db = withLatency(async (id) => {
  return { id, name: 'Arnav' };
}, 'postgres');

// your retry logic
async function withRetry(fn, retries = 3, baseDelay = 100) {
  for (let i = 0; i < retries; i++) {
    try {
      return await fn();
    } catch (err) {
      if (i === retries - 1) throw err;
      // exponential backoff with jitter
      const delay = baseDelay * Math.pow(2, i) * (0.5 + Math.random() * 0.5);
      await new Promise(r => setTimeout(r, delay));
    }
  }
}

// now you're actually testing against production-like behavior
const result = await withRetry(() => db.findUser(42));

Run this a hundred times. Watch which calls fail. Tune your timeouts and backoff based on what you see. That's actual resilience testing — not false confidence from a flat 200ms.

Summary

Real dependency latency is lognormal: fast most of the time, occasionally slow, rarely very slow
setTimeout(fn, 200) tests only the worst case, every time — it hides the bugs that only appear from variance
Fitting a lognormal distribution to your p50/p99 values gives you realistic simulation in one function call
slowdep wraps any async function with zero dependencies and built-in presets for postgres, redis, stripe, openai, s3, and more

If your retry logic has never been tested against real latency variance, it probably has bugs you haven't found yet.

Source code and presets: github.com/arnnnavvvvv/slowdep