DEV Community: Walker Miller

Context window sizing for fine-tuning: how long should your training examples be?

Walker Miller — Sat, 08 Aug 2026 05:36:04 +0000

Originally published on Loop & Retry — field notes on building LLM agents that survive production.

Most fine-tuning guides answer "how many examples" and skip "how long should each one be." That second question is the one that quietly decides whether your fine-tune helps at inference or fights it. Example length isn't a property you inherit from your data — it's a design choice, and the default (whatever length your dumped transcripts happen to be) is usually wrong in one of two expensive directions.

The framing I keep coming back to: the context window is a cache, not a memory. Fine-tuning changes what the weights know; it does not change the fact that at inference the model reasons over whatever you put in the window right now. Size your training examples to the window you'll actually serve, or you're training for a world you won't deploy into.

The two failure directions

Too short is the sneakier one. Say your real requests arrive with 6–8K tokens of retrieved context, but your training examples are tidy 800-token snippets because that's what your labeling tool exported. You've now fine-tuned a model whose learned prior is "the answer is near the top of a short prompt." At inference you hand it 8K tokens and the relevant fact sits at position 5,000, and the model underweights it — not because the base model can't attend that far, but because your fine-tune taught a length distribution that never occurs in production. You optimized the model onto a distribution you will never sample from.

Too long is the one that shows up on the invoice. Attention is quadratic in sequence length, so a training set of 32K-token examples doesn't cost 4× a set of 8K-token examples — it costs closer to 16× per step in the attention term, plus the memory that forces you into smaller batches or gradient checkpointing, which slows you down again. Worse, long examples tempt you into teaching the model to memorize reference material that belongs in retrieval. You pay quadratic training cost to bake facts into weights that a RAG lookup would have served fresh, and now those facts are frozen at training time and go stale.

Match the training distribution to the serving distribution

The rule is boring and load-bearing: the length distribution of your training examples should match the length distribution of your production requests. Not the max, the distribution. If prod requests are lognormal with a median of 4K and a p95 of 12K, your training data should look like that too — a spread, not a single padded length.

This is where the cache framing matters. If you're managing the context window as a cache with an eviction policy, the length distribution of the serving examples includes the effects of your eviction decisions — summaries, truncations, reorderings. Training on full, un-truncated examples teaches the model a distribution that doesn't exist at inference.

Measure it before you build the set:

import numpy as np

# token counts of real production prompts (sample from logs)
lengths = np.array([count_tokens(p) for p in sampled_prod_prompts])

for q in (50, 90, 95, 99):
    print(f"p{q}: {np.percentile(lengths, q):6.0f} tokens")
print(f"max sequence to train on: ~p99 = {np.percentile(lengths, 99):.0f}")

Set your training max_seq_len at roughly the p99 of production, not the max. The single 60K-token outlier request shouldn't force every batch to reserve 60K of sequence budget; truncate or drop the long tail and handle it separately. And critically: don't pad-and-collapse your examples to one length. Bucket by length so a batch of short examples trains cheaply and only the genuinely long batches pay the quadratic cost. Length bucketing is the single highest-leverage efficiency lever in fine-tuning and it's routinely skipped.

Where the labels sit changes the sizing

There's a second-order effect people miss. If your examples are long and the label (the tokens you compute loss on) is short and at the end — a classic "long context in, short answer out" shape — then most of the sequence is loss-masked context the model reads but isn't scored on. That's fine functionally, but it means your effective training signal per token is low: you're paying to process 12K tokens to get gradient from 200.

Two consequences. First, you may need more examples than a short-answer intuition suggests, because each one carries little supervised signal relative to its cost. Second, this is often the signal that you should be retrieving that context at inference rather than teaching the model to condition on a specific long document — if the long part is reference material rather than the reasoning you want to instill, it belongs in the cache, not the memory.

Don't fine-tune the summarizer's mistakes in

If your production pipeline compacts context — summarizing earlier turns to fit the window — then your serving distribution includes compacted, lossy context. Your training examples had better include it too. Compaction is a lossy operation: it deliberately drops detail to make room, and that loss changes the information the model sees. Fine-tuning exclusively on full, un-compacted transcripts and then serving compacted ones at inference is another train/serve mismatch: you taught the model to rely on detail that your own pipeline strips before the model ever sees it in production. If you compact at inference, compact (a sample of) your training examples the same way, so the model learns to reason over the degraded input it will actually get.

What I'd actually do

Sample real prod prompts and plot the length distribution first. Everything downstream keys off p50/p95/p99. Guessing here is guessing at the whole design.
Set `max_seq_len ≈ p99 of production, and length-bucket batches.** Don't let the tail dictate the batch, and don't pay quadratic cost on short examples by padding them long.
Match the shape, not just the cap. A spread of lengths that mirrors production beats one padded length, even if the padded length is "safe."
Ask whether the long part is reasoning or reference. Reasoning you want in the weights; reference you want in retrieval. Fine-tuning reference material is paying quadratic cost to freeze facts that go stale.
If you compact at inference, compact your training data too. Train on the distribution you serve, degradations included.

Example length is a lever, and it's one of the few in fine-tuning where the wrong default costs you on both axes at once — quality and dollars. Measure the serving distribution, then build training examples that look like it. The model can only learn the world you show it, and the window is that world.

Quadratic-in-sequence-length is the standard dense-attention cost model; architectures with sparse or linear attention change the constant but not the direction of the argument. Percentile targets and bucket boundaries are workload-specific — the method (match training length distribution to serving length distribution) is what transfers.

Context contamination: why retrying the same prompt makes it worse

Walker Miller — Fri, 07 Aug 2026 17:36:23 +0000

Originally published on Loop & Retry — field notes on building LLM agents that survive production.

The instinct when a tool call fails or an output is wrong is to retry the same way you'd retry a flaky network call: catch it, tell the model what went wrong, resend. That's correct for a genuinely transient failure. It's the wrong move for a failure caused by the model's own reasoning, because "resend" doesn't mean "give it a clean second attempt" — it means "give it a context window with the wrong answer already sitting in it, and ask it to do better." A model doing next-token prediction over a transcript that already contains one confident, wrong attempt is anchored on that attempt, not liberated from it.

I think of this as context contamination: the failed path isn't just a wasted turn, it's now part of the evidence the next turn conditions on. The context window is a cache — what's true for stale data is true here too, except the thing going stale is your own bad move, and you're the one who put it back on the shelf.

Why the naive retry backfires

Picture an agent asked to extract a date from a messy log line, using a regex tool call. First attempt: \d{2}/\d{2}/\d{4}, which misses the line's actual 2026.07.13 format. The naive retry loop appends a tool-result turn ("no match found") and a nudge ("that didn't match — try again"), then re-calls the model with the full transcript, including the original wrong regex, still sitting there in plain view.

What tends to happen next isn't a fresh attempt. It's a near-duplicate of the first: \d{2}\/\d{2}\/\d{4} with an escaped slash, or \d\d/\d\d/\d\d\d\d with the quantifiers spelled out, or the same pattern with \s* sprinkled around it. The model isn't ignoring the feedback — it's doing the statistically reasonable thing given a context where the most recent, most salient prior turn is "here is a plausible-looking regex for this problem," which is exactly the anchor you didn't want it anchored to. This is a cousin of loop drift: drift is the agent convincing itself it's making progress across many turns; contamination is the narrower case where one specific wrong turn keeps re-asserting itself because you never took it out of the window.

Here's a small illustration of the mechanism, not a claim about real pass rates — the point is qualitative, not a benchmarked number:

# Illustrative only: shows the SHAPE of the failure, not a measured pass rate.
naive_retry_messages = [
    {"role": "user", "content": "extract the date from: ERR 2026.07.13 timeout"},
    {"role": "assistant", "content": "tool_call: regex(pattern=r'\\d{2}/\\d{2}/\\d{4}')"},
    {"role": "tool", "content": "no match"},
    {"role": "user", "content": "that didn't match. try again."},
]
# The model's next completion is conditioned on ALL four turns above —
# including its own wrong regex, which is the strongest recent signal
# in the window about "what a plausible answer looks like here."

The transcript is doing the opposite of what you want a retry to do. You wanted the model to reconsider the date format. What it actually saw was: task, an example of a kind of answer, a terse rejection, and an instruction to produce another one of those. Nothing in that shape points away from the failed pattern's neighborhood.

Not every retry needs a scrub

The fix is not "always wipe the context before retrying" — that's wasteful and, for a real class of failures, wrong. As I explored in compaction is a lossy operation, any edit to the transcript — dropping, summarizing, or reordering turns — invalidates the cache and costs you in ways beyond just the context rewrite. The distinction that matters is where the failure lived:

Transient / environmental failure — the tool timed out, the API returned a 503, the network blipped. The model's reasoning was fine; the world hiccupped. Retrying with the same context, maybe the same exact call, is correct and cheap. Scrubbing here would throw away good reasoning for no benefit.
Semantic / reasoning failure — the model picked the wrong regex, the wrong tool, the wrong interpretation of the task. The reasoning itself is the thing that failed, and it's sitting in the transcript as the most recent example of "how to approach this." This is the case that needs a scrub, not a resend.

Conflating the two is how teams end up with a single retry() wrapper that quietly makes semantic failures worse while "fixing" transient ones, and nobody notices because the transient case is the common one in testing and the semantic case is the one that shows up in production on the weird inputs.

The scrub: keep the constraints, drop the transcript

For a semantic-failure retry, the goal is a context that carries forward everything the agent learned about the task's constraints, without carrying forward the specific wrong attempt as a stylistic template. That means replacing the raw failed turns with a short structured statement of what's now ruled out. This is the deliberate, manual version of the cache management from context-window principles — you're deciding what earns its space and what gets evicted. The key difference from a generic old-turns-get-summarized policy is that you're not trying to preserve the detail; you're deliberately erasing the failed path while keeping the constraint it discovered:

def scrub_for_semantic_retry(task: str, ruled_out: list[str]) -> list[dict]:
    """Rebuild a clean retry context: original task + explicit exclusions.
    Drops the raw failed assistant/tool turns entirely — they don't get
    to serve as a template for the next attempt."""
    constraints = "\n".join(f"- {r}" for r in ruled_out)
    return [
        {"role": "user", "content": (
            f"{task}\n\n"
            f"Approaches already ruled out (do not repeat these or close variants):\n{constraints}"
        )},
    ]

# After the regex miss above:
messages = scrub_for_semantic_retry(
    task="extract the date from: ERR 2026.07.13 timeout",
    ruled_out=["slash-delimited MM/DD/YYYY regex — this log uses dot-delimited YYYY.MM.DD"],
)

Notice what's preserved and what isn't. Preserved: the task, and a named reason the first approach failed — specific enough to actually steer the next attempt (dot-delimited, not slash-delimited), not just "that didn't work." Dropped: the literal wrong regex string, the terse rejection, and the turn structure that made the wrong answer look like the most recent worked example in the room. The model gets the lesson without getting the paper trail that taught it the wrong lesson.

What I'd actually do

Classify the failure before you retry, not after. Transient (environment) vs. semantic (reasoning) determines whether you resend or scrub — a single generic retry wrapper can't make that call for you.
Never let a retry's context contain the literal failed attempt as the most recent turn. If it's semantically necessary information, restate it as a ruled-out constraint, not as a transcript the model can pattern-match against.
Cap scrubbed retries too. A scrub buys a genuinely fresh attempt, not infinite ones — after two or three ruled-out constraints pile up, the more honest move is to escalate to a human or a different tool, not keep rebuilding a cleaner box for the model to fail in again.
Log what you scrubbed. The ruled-out list is valuable telemetry independent of whether the retry succeeds — it's a direct readout of which of the model's assumptions were wrong, which is exactly the kind of thing worth catching in an eval before it ships.

A retry is supposed to be a second, independent look at the problem. The naive version isn't independent — it's the first look, plus its own wrong answer, plus an instruction to disagree with something still sitting in front of it. Scrub the transcript, keep the lesson, and the second attempt actually gets to be a second attempt.

Compaction is a lossy operation

Walker Miller — Fri, 07 Aug 2026 05:36:14 +0000

Originally published on Loop & Retry — field notes on building LLM agents that survive production.

Every long-running agent eventually hits the wall: the context window fills, and something has to give. The near-universal fix is compaction — summarize the older turns into a shorter recap, drop the raw transcript, and keep going. It works, right up until the run where the agent cheerfully violates a constraint it was given on turn 3, because turn 3 didn't make it into the summary. Nothing errored. The agent just forgot, and forgot in a way that looks exactly like a reasoning failure instead of the data-loss bug it actually is.

This post is about treating compaction as what it is: a lossy compression step in the middle of your control flow. I've argued before that the context window is a cache, not a memory — that you should run it with a budget and an eviction policy. Compaction is that eviction policy, executed by a model that doesn't know which facts are load-bearing. That's the problem.

Why "summarize the old turns" loses the wrong thing

Compaction usually works on recency. The last few turns stay verbatim; everything older gets squeezed into a paragraph or two of summary. This is a reasonable default for the shape of a conversation — recent context is usually most relevant to the next step — and it is exactly wrong for a specific, common, and costly case.

The facts that matter longest are often stated earliest. The user's hard constraint ("never touch the production database", "the budget is $500, hard cap", "the customer is in the EU so GDPR applies") arrives at the start of the task and stays relevant until the end. A recency-based summarizer sees that fact age out of the verbatim window, tries to compress it alongside fifty other turns of tool calls and chit-chat, and — because a summary's whole job is to drop detail — renders it as "the user described some requirements" or drops it entirely. The load-bearing constraint gets the same treatment as the small talk.

The failure is delayed and disguised. The agent runs fine for eighty turns. Then it reaches the step where the dropped constraint would have applied, doesn't have it, and does the reasonable-looking wrong thing. You debug it as a reasoning error or a prompt problem. It's neither. It's a fact that was in context, got compressed out, and never came back.

Simulating how often the fact survives

Let's put numbers on it. Model a run as a stream of facts arriving over turns. Most are ordinary (tool results, intermediate reasoning). One, planted early, is load-bearing: it's needed at the very end. When the transcript exceeds a budget, we compact. Two policies:

Recency: keep the most recent facts that fit; summarize the rest into a lossy blob that retains each old fact only with probability RETAIN (a summary keeps some things, drops others).
Salience: same, but facts tagged load-bearing are pinned — never summarized away.

We measure one thing: at the final turn, is the load-bearing fact still present?

import random

TURNS      = 120      # length of the run
BUDGET     = 30       # facts we can hold verbatim before compacting
RETAIN     = 0.30     # chance the running summary keeps a given old fact
KEY_TURN   = 3        # the load-bearing constraint arrives early

def simulate(policy, trials=200_000):
    survived = 0
    for _ in range(trials):
        kept = []            # facts still held verbatim (True = load-bearing)
        key_state = None     # None=still verbatim, True=in summary, False=dropped
        for t in range(TURNS):
            kept.append(t == KEY_TURN)
            if len(kept) > BUDGET:
                # compact: oldest facts leave the verbatim window into the summary
                old, kept = kept[:-BUDGET], kept[-BUDGET:]
                for is_key in old:
                    if not is_key:
                        continue
                    if policy == "salience":
                        key_state = True                     # pinned: always retained
                    elif key_state is None:                  # decided once, then carried
                        key_state = random.random() < RETAIN
        present = any(kept) or key_state is True
        survived += present
    return survived / trials

for policy in ("recency", "salience"):
    print(f"{policy:9} load-bearing fact present at end: {simulate(policy)*100:5.1f}%")

Running it:

recency   load-bearing fact present at end:  30.0%
salience  load-bearing fact present at end: 100.0%

Under recency-based compaction, the fact the whole task depends on is gone 70% of the time by the end of a long run. Not because the window was too small to hold it — it's one fact — but because the compaction policy had no idea it was special. It aged out, got summarized, and the summary rolled the dice and lost. The salience policy keeps it every time, at the cost of pinning one fact.

The exact percentage isn't the point (it's set by RETAIN, which you can argue about). The point is the shape: once an early critical fact ages out of the verbatim window, its survival collapses to whatever your summarizer's retention rate happens to be — here 30% — and no additional run length ever recovers it. The longer the agent works, the further behind that fact falls, and the more certain it is to have forgotten why it started.

The rule: compaction needs a schema, not just a summarizer

The fix isn't a better summarization prompt. A better prompt still asks one model to guess, in one shot, which of a hundred turns will matter a hundred turns from now — and it will still sometimes guess wrong, silently. The fix is to stop treating all context as fungible text to be compressed uniformly, and give compaction a schema of what must never be dropped.

Concretely, before you compact, separate context into two bins:

Durable state — pinned, never summarized. Constraints, hard limits, IDs and keys, the task goal, decisions already made, and anything the user flagged as important. This is small and it is load-bearing. It should live in a structured slot that compaction physically cannot touch — not buried in the transcript hoping a summarizer keeps it. If you can't enumerate this bin for your agent, that's the actual gap: you don't yet know what your agent must remember.
Transient context — free to compress. Tool call chatter, intermediate reasoning, superseded attempts, resolved sub-tasks. Summarize this aggressively; it's genuinely recency-biased and losing detail here is fine.

The mechanism that makes this work is the same one from the cache post: an eviction policy that knows the cost of a miss. A cache that evicts your session token because it hasn't been read in a while is broken; so is a compactor that summarizes away the one constraint the task hinges on. Both are eviction without regard to what a miss costs. Pinning durable state is just declaring, up front, which misses are unaffordable.

Two guardrails I'd add on top: make the durable bin auditable — log what's pinned, so when the agent does something that violates a constraint you can immediately check whether the constraint was even present (turning a mystery reasoning bug into a one-line data-loss check); and verify after compaction, not just before — a cheap assertion that every pinned fact is still retrievable in the compacted context catches a broken compactor the same run it breaks, instead of eighty turns later.

Compaction is not a neutral housekeeping step. It's a lossy write in the middle of your agent's memory, performed by a component that doesn't know what's load-bearing unless you tell it. Tell it. The alternative is an agent that runs beautifully for a hundred turns and then forgets the one thing you gave it first.

The simulation here is a toy model of fact survival, not a benchmark of any specific compaction implementation — RETAIN and the run length are stated so you can plug in numbers that match yours. The lesson (recency compaction degrades toward its retention rate for early critical facts, pinning durable state fixes it) is architecture-level and provider-independent.

Cheap first, smart later: model routing that cuts cost without cutting quality

Walker Miller — Thu, 06 Aug 2026 17:43:35 +0000

Originally published on Loop & Retry — field notes on building LLM agents that survive production.

Here's the asymmetry that makes model routing worth the trouble: most of what an agent handles in production is easy — a well-formed tool call, a summary of a short document, a classification with an obvious answer — and you're routing all of it through the same model you needed for the 10% of requests that are actually hard. A cascade fixes that by trying the cheap model first and escalating only when it's warranted. Done right, this cuts the token bill 40-70% with no quality loss. Done wrong, it just moves your failures downstream where they're harder to see.

This post covers the pattern itself, the one design decision that determines whether it works (the escalation trigger), the failure modes that show up when you get that decision wrong, and when the added architectural complexity isn't worth it.

The pattern

A routing cascade is a pipeline, not an agent — the control flow is yours, not the model's, which matters because it means you can reason about it and test it (see when not to build an agent for why that distinction is the whole ballgame). The shape:

def route(request, cheap_model, expensive_model, escalate_fn):
    cheap_response = cheap_model.complete(request)
    if escalate_fn(request, cheap_response):
        return expensive_model.complete(request)
    return cheap_response

That's the entire pattern. Everything that makes it work or fail lives inside escalate_fn. A cascade with a bad escalation trigger is worse than not having one, because it adds latency (you paid for the cheap call and the expensive one) without saving money on the requests that needed escalating anyway, or worse — it fails to escalate the ones that did.

The trigger is the whole design problem

There are three broad strategies for escalate_fn, in order of how much I trust them:

1. Structural signals — cheapest to compute, hardest to game. Does the cheap model's output pass a schema check? Did it call a tool with valid arguments? Did it produce a response of plausible length for the task? These are binary, deterministic, and don't require another model call. If you're doing structured extraction or tool-calling, this alone catches a large fraction of the cases that need escalation, because a model that's out of its depth on a task usually fails structurally before it fails semantically — malformed JSON, a tool call with an argument that doesn't type-check, an empty required field.

def escalate_on_structure(request, response):
    try:
        parsed = json.loads(response.text)
    except json.JSONDecodeError:
        return True
    if not schema.validate(parsed):
        return True
    return False

2. Self-reported confidence — cheap, but only trustworthy if you've measured it. Asking the cheap model to emit a confidence score alongside its answer costs nothing extra (same call, one more field), but the score is only meaningful if you've correlated it against ground truth for your task on your model. Confidence scores from LLMs are not calibrated out of the box — a model saying "0.9 confident" has no guaranteed relationship to a 90% chance of being right unless you've checked. Treat the raw score as a ranking signal, not a probability, and pick your threshold empirically:

def escalate_on_confidence(request, response, threshold):
    # threshold is not 0.5 by default — set it from a labeled
    # validation set for this task, this cheap model, this prompt.
    return response.confidence < threshold

The measurement step here is not optional. I've seen teams ship this with a threshold picked by feel, and the cascade quietly escalates 80% of requests (no savings) or 5% (all the savings, none of the safety).

3. A judge model — most expensive, most flexible. For tasks where structure and self-report both fall short (open-ended generation, nuanced classification), you can have a third, cheap-but-not-trivial model score the response before deciding whether to escalate. This is the LLM-as-judge pattern applied to a single response instead of a full eval, and it inherits the same biases — position bias, length bias, self-preference if the judge is a sibling of the model being judged. Use it only when the first two options genuinely don't apply, and validate the judge against labeled examples before trusting it in the loop.

Where cascades break

Escalation latency compounds on the requests that most need to avoid it. The hard requests — the ones that escalate — now pay the cheap model's latency plus the expensive model's latency, serially. If your P99 latency budget is set assuming a single model call, a cascade blows through it precisely on the tail you cared most about. Measure P99 with escalation included, not average latency, or you'll ship a cascade that looks fine in aggregate and pages someone at 2am on the hard cases.

Structural checks don't catch confident wrongness. A cheap model can produce a perfectly valid, schema-conforming, plausible-length response that's just wrong — hallucinated a field value, picked the wrong tool for a subtly different task, summarized the wrong section. Structural signals catch "this output is malformed," not "this output is incorrect." If your task has a failure mode that looks structurally fine but is semantically wrong, you need a confidence or judge signal, not just a schema check — and you need eval data showing your structural checks actually correlate with correctness for your task, not just an assumption that they do.

Threshold drift. The cheap model gets updated by the provider, your prompt changes, your input distribution shifts — any of these silently moves the relationship between your confidence threshold and actual accuracy. A threshold tuned once and never revisited degrades quietly: you don't get an error, you get worse escalation decisions that show up as a slow quality decline nobody traces back to the cascade. Re-validate the threshold on a schedule or when you change the cheap model, not just at launch.

The cascade becomes the thing you're debugging instead of your actual product. Every layer you add — structural check, confidence gate, judge call — is a component with its own failure modes, and now you're maintaining a routing system alongside the thing it routes for. This is the real cost that doesn't show up in the token bill: engineering time spent tuning thresholds and debugging misroutes instead of the product.

The arithmetic on whether it's worth building

Say the expensive model costs 15x the cheap one per request (a reasonable ratio between a small and a frontier model), and 70% of your traffic is genuinely easy. Route everything through the expensive model: cost is N × 15. Route with a cascade at 70% cheap-only: cost is 0.7N × 1 + 0.3N × 16 (the 30% pays for both calls) = 0.7N + 4.8N = 5.5N. That's a 63% reduction — real money at volume, and it's the headline number that makes cascades attractive.

But that arithmetic assumes your escalation trigger correctly identifies the 70% that don't need escalating. If it's wrong 10% of the time in the direction of not escalating requests that needed it, you haven't just lost some savings — you've shipped wrong answers to 7% of your total traffic, silently, at whatever quality bar the cheap model has for hard problems it doesn't recognize as hard. That's the trade a cascade actually makes: token savings you can calculate in advance, against an error rate you can only know by measuring, not assuming.

Build one when: your traffic has a genuine easy/hard split (check this — don't assume it), you can build a structural or validated-confidence trigger for your specific task, and you can afford the eval work to validate the threshold before it's live.

Skip it when: your traffic is uniformly hard (no savings available), you can't validate the trigger against labeled data (you're guessing at the threshold), or the engineering cost of building and maintaining the router exceeds what you'd save — which is common at low volume, where the token savings are real but small and the failure modes are exactly as expensive as they are at high volume.

What I'd do

Instrument your traffic first: log what fraction of requests the cheap model alone would get right, using whatever ground truth you have (even a small labeled sample). If that fraction isn't large, a cascade is solving a problem you don't have. If it is, start with the structural trigger — it's free, deterministic, and catches more than you'd expect — and only add a confidence or judge layer if structural checks leave real failures uncaught. Measure P99 latency with escalation in the path, not around it, before you ship. And put the threshold re-validation on a calendar, because the cascade that was correctly tuned at launch is not the cascade you're running six months later unless you checked.