TL;DR
- On a 2 vCPU nginx (OpenSSL 3.5), switching on X25519MLKEM768 moves the knee (the load at which TLS handshake p99 breaks your latency target) 17–21% earlier. That held at every latency target I tested (1.5, 2, 3 and 5 ms), and with 0% or 90% of connections resumed.
- With a 2 ms handshake-p99 target, the knee sits at 66–72% of the CPU ceiling and 69–77% host CPU. An 85% CPU alert fires after you've already broken the target.
- An SLA-free early warning works: p99 elasticity against load (% rise in p99 per % rise in handshake rate) sat around 1–2 below the knee and reached ≥ 3 at or just before the 2 ms crossing in all four configurations.
- Past the knee, ML-KEM hits the wall sooner. Cold PQ handshake p99 went 8.6 ms → 44 ms → 1.57 s between 4,000 and 4,500 connections/s. The server's maximum (4,364/s) matched the prediction from CPU-per-handshake (4,396/s) to within 0.7%.
Where this came from
Part 7 swapped a live nginx onto OpenSSL 3.5 with X25519MLKEM768 and concluded the swap was roughly free in CPU. A reader in the comments pushed back, three rounds in all, and every round made the results better:
- Zero errors is the floor, not the finding. Measure handshake p99, and separate cold connections from resumed ones. (That became the Part 7 follow-up: ML-KEM costs +68 µs per handshake whether cold or resumed, and handshake p99 goes up even where CPU goes down.)
- That's a capacity question, not only a latency one: +68 µs × 10,000 resumed handshakes/s is about 0.68 of a core.
- So alert on the knee, not on a fixed CPU percentage, and (a fourth comment) on the derivative of p99, which moves before the doubling does. Find the handshakes/s per vCPU at which p99 crosses your latency target, before and after PQ, and alert at a fraction of that measured knee. Log the resumption ratio alongside it, because it moves the knee as much as load does.
This post measures point 3.
What "the knee" is
Plot handshake p99 against offered load. At moderate load the line is almost flat: a handshake arrives, a CPU is free, and the time it takes is its own work. As utilisation climbs, more handshakes arrive while every CPU is busy and have to queue. Queueing delay doesn't grow evenly. It stays small for a long stretch, then shoots up as utilisation nears 100%, so the line bends upward like a knee.
The knee is the highest load at which p99 still meets your latency target (SLA). It's a real capacity number for a node, and unlike CPU %, it's expressed in the same units as your traffic.
Setup
Same kit as Part 7: nginx 1.28.0 with OpenSSL 3.5.2 on a c7g.large (2 vCPU Graviton3), a Go load generator on a c7g.xlarge in the same AZ, ECDSA P-256 certificate, every connection a new TLS 1.3 handshake. Session tickets are on in every run.
- Arms: OpenSSL 3.5 classical (X25519) vs OpenSSL 3.5 with X25519MLKEM768. Same binary, same config apart from the group list, so the only difference is ML-KEM.
-
Mix: 0% or 90% of connections resume a session (verified per request, and from the server's
$ssl_session_reused). - Load: 2,750 to 4,500 new connections/s in 250/s steps, open-loop, 45 s per point.
- Repeats: 2 trials per point, 64 runs in shuffled order.
- Knee: the offered rate at which mean handshake p99 crosses the target, interpolated between steps.
Result
Knee with a 2 ms handshake p99 target, 2 vCPU:
| 0% resumed | 90% resumed | |
|---|---|---|
| OpenSSL 3.5, classical | 3,575/s (1,787 per vCPU) | 4,281/s (2,140 per vCPU) |
| OpenSSL 3.5 + X25519MLKEM768 | 2,909/s (1,455 per vCPU) | 3,389/s (1,694 per vCPU) |
| Knee moves | −18.6% | −20.8% |
The shift doesn't depend on which target you pick:
| Handshake p99 target | 0% resumed | 90% resumed |
|---|---|---|
| 1.5 ms | PQ already above it at 2,750/s | −21.2% |
| 2 ms | −18.6% | −20.8% |
| 3 ms | −17.8% | classical never crossed it |
| 5 ms | −17.2% | classical never crossed it |
Why ~20%, when ML-KEM's CPU cost per handshake is ~16%? CPU per handshake went from 390 to 455 µs cold (+17%) and from 339 to 401 µs resumed (+18%). If CPU were the whole story, the knee would move by about 14–16%. It moves further because ML-KEM also raises the latency floor: even well below the knee, PQ's p99 is higher (1.68 vs 1.08 ms at 2,750/s, cold). So PQ reaches any fixed target sooner than its CPU cost alone would suggest. The commenter's framing was right: the cost shows up in the tail first.
Resumption buys headroom, but doesn't pay for PQ. Moving from 0% to 90% resumed raises the knee by 20% (classical) and 17% (PQ). But PQ at 90% resumed (3,389/s) still bends earlier than classical with no resumption at all (3,575/s).
Why a CPU alert is the wrong tripwire
| CPU ceiling (2 vCPU ÷ cost per handshake) | knee @ 2 ms | knee as % of ceiling | host CPU at knee | |
|---|---|---|---|---|
| classical, 0% resumed | 5,122/s | 3,575/s | 70% | 74% |
| classical, 90% resumed | 5,907/s | 4,281/s | 72% | 77% |
| PQ, 0% resumed | 4,396/s | 2,909/s | 66% | 69% |
| PQ, 90% resumed | 4,985/s | 3,389/s | 68% | 72% |
With a 2 ms target, every combination broke its latency target at 69–77% host CPU. With a 1.5 ms target the knee comes even earlier, at 61–66% of the CPU ceiling; with a loose 5 ms target, it's around 84–87%. Where the knee sits depends on your latency target, which is exactly why a fixed "alert at 85% CPU" can't be right for everyone. For anyone with a tight target, it fires after the damage.
And the knee moves when you turn on PQ, so a threshold tuned before the migration is wrong after it.
An early signal that doesn't need your SLA
The same commenter's next point: neither metric alone is the signal. Watch the derivative of p99, which moves before the doubling does.
The data supports that, with one refinement: take the derivative against load, not time. "p99 rose 20% in the last 5 minutes" also fires on every normal morning traffic ramp, and it can't tell a node that's busier from one that's degrading. Elasticity can: the percentage change in p99 for each 1% change in handshake rate. Below the knee, latency grows roughly in step with load (e ≈ 1–2). Near the knee, queueing makes it climb much faster than load.
| e below the knee | first step with e ≥ 3 | 2 ms target crossed at | |
|---|---|---|---|
| classical, 0% resumed | 1.6–1.7 | 3,250 → 3,500/s (e = 3.9) | 3,575/s |
| classical, 90% resumed | 0.7–2.4 | 4,000 → 4,250/s (e = 3.1) | 4,281/s |
| PQ, 0% resumed | (already 3.1 at the first step) | 2,750 → 3,000/s (e = 3.1) | 2,909/s |
| PQ, 90% resumed | 1.8–2.3 | 3,250 → 3,500/s (e = 3.4) | 3,389/s |
In all four configurations, e ≥ 3 appeared at or one step before the 2 ms crossing, and computing it doesn't require knowing anyone's latency target. Past the knee it explodes: cold PQ went to e = 27, then 63.
Honest limits: that's four curves, 250/s steps, and p99 averaged over 2 trials. The signal is noisy step to step (classical cold dips from 3.9 back to 2.4 before climbing again), so in production you'd smooth it over several windows and require it to persist. Treat e ≥ 3 as a starting point to calibrate against your own ramp, not a universal constant.
A time-only derivative still has one job elasticity can't do: p99 rising while load stays flat. That's what a resumption ratio drifting toward cold connections looks like. It needs no deploy and no code change, because the ratio is a property of the workload and it moves the knee by itself. Run both alerts.
The cliff
Past the knee, the PQ arm with no resumption shows what happens when offered load exceeds what the CPU can do:
| offered | cold PQ handshake p99 | cold classical handshake p99 |
|---|---|---|
| 4,000/s | 8.6 ms | 2.8 ms |
| 4,250/s | 44 ms | 3.5 ms |
| 4,500/s | 1,570 ms | 5.4 ms |
At 4,500/s offered, nginx served 4,361 and 4,367/s (two trials) at 100% CPU. That's the box's ceiling: from the measured 455 µs per handshake, 2 vCPU can do 4,396/s. Everything above that queues. Requests waited about 1.6 s, and the load generator started shedding new requests once 5,000 were in flight (about 2.5% of offered load). Those "errors" are the client giving up, not nginx refusing. Classical, with its higher ceiling (5,122/s), was still at 5.4 ms.
That's the real reason to care: ML-KEM costs only 65 µs, but if your fleet is sized on pre-PQ numbers, those 65 µs can move a busy node from "fine" to "past the cliff".
The runbook (updated)
- Measure your knee, don't assume it. Run an open-loop ramp against one node, with your real certificate, your real resumption mix and your real latency target. Do it once with your current config and once with X25519MLKEM768.
-
Alert on handshakes/s per node, at a fraction of the measured knee (say 70–80% of it), not on CPU %. For the early warning, use one rule that picks between two signals (the switch is the commenter's design):
- Load moved: if the handshake rate across the last several minutes varied by more than its normal noise, and the window contains no change event (autoscale, deploy, rebalance, a cron burst), compute elasticity as the slope of log p99 against log rate over that window, and alert at ≥ ~3.
- Load flat: otherwise elasticity is meaningless (a near-zero denominator turns p99 jitter into huge values), so use the time derivative: alert when p99 rises steadily at flat load. That's what a resumption ratio drifting toward cold connections looks like.
- Change-event windows: skip them for both signals. The event itself (new nodes, rebalanced connections, cold workers) can raise p99 for reasons that say nothing about steady-state capacity.
-
Log per node, per minute: new handshakes/s split by
$ssl_session_reused, and the share negotiating X25519MLKEM768 ($ssl_curve). The resumption ratio changes where the knee is; the PQ share tells you which knee applies. - Re-run step 1 whenever the mix changes: PQ rollout percentage, ticket lifetime, a client update that changes resumption behaviour.
- Size for the cliff, not the average. Keep enough headroom that losing a node doesn't push the survivors' offered load past their PQ ceiling.
The instrument failed again (briefly)
My first knee definition was "the load at which p99 doubles from its low-load value". It said PQ moved the knee only 7%. That's wrong. At my lowest test rate, PQ's p99 was already 1.68 ms against classical's 1.08 ms, so "double its own baseline" set a much higher bar for PQ (3.35 ms vs 2.15 ms). A fixed target is the only fair comparison: users don't experience your baseline, they experience your SLA.
Method and limits
- c7g.large (2 vCPU Graviton3) running nginx 1.28.0 with OpenSSL 3.5.2, 2 workers; c7g.xlarge Go load generator; same AZ, cluster placement group.
- Every connection is a full or resumed TLS 1.3 handshake plus a 3-byte response. Real requests add their own CPU, which lowers every knee in absolute terms; the relative PQ shift is the transferable part.
- ECDSA P-256 certificate. With RSA, the signature costs more, so ML-KEM's relative share (and possibly the shift) would be smaller.
- 45 s per point, 2 trials. Trial-to-trial spread was small except right at the bend (PQ cold, 3,750/s: p99 5.45 ± 0.78 ms), which is typical of queueing near saturation.
- Knees are interpolated between 250/s steps.
- Same-AZ round-trip times: this measures server compute and queueing, not network distance.
- Elasticity is computed between adjacent load steps from mean p99 (
results/7c/knee-elasticity.csv); production use needs smoothing and calibration. - Code, per-run data and analysis: pqc-bench-nginx-swap,
[results/7c/], commit4230bd34f9dbc28a3114d190941c36cb3aa37c2f. - Same failure profile on both arms. A failed handshake is cheaper than a completed one, so a knee comparison is only fair if neither arm is shedding work. Across all 64 runs, the only errors were client-side drops in the two cold-PQ runs at 4,500/s (beyond every knee). There were zero TLS handshake failures, served rate equalled offered rate in every other run, and CPU per handshake stayed flat across the ladder (cold PQ 452–458 µs). The knee shift is completed work that cost more, not skipped work. (Thanks to the same commenter for this check.)
Thanks again to the commenter on Part 7. Five comments turned a "zero errors" post into a capacity model and an alerting design.

Top comments (0)