A Senior Pairing Stopped Invented API Load With One Kept Decision

Generated API tests stay honest only after pairing freezes a captured request mix, before any model writes load code. Models readily invent concurrent clients, think times, and endpoint weights that look professional while describing traffic that never existed. A senior pairing session can block that fiction by keeping one decision: the mix file is the only legal input. This article records the questions, the dead ends, and the small contract that followed from that kept decision.

Why invented load fails later

API performance work collapses when the request mix is a story rather than a sample from a real window. Synthetic clients then hammer rare write paths, skip cacheable reads, and report latency that no production user would ever observe. Reviewers argue about percentiles while the underlying shape of traffic remains an unexamined guess. Pairing with a senior engineer is useful here because the senior can refuse progress until that guess is replaced by a file.

Tool-calling demos and browser agents make this failure mode easier, not rarer, because fluent clients appear in minutes. A generated k6 or pytest harness can look complete while encoding a popularity ranking the logs never supported. The pairing problem is therefore not how to prompt a better client. The pairing problem is how to keep one traffic decision before any client exists.

A labeled pairing setup

The walkthrough below is a labeled pairing example, not a report of one named incident or a measured production outage. One senior engineer owns questions, vetoes, and the kept decision recorded in the repository. One implementer owns the fixture file, the loader, and the gate that later blocks generated tests. A shared scratch buffer holds the question log so dead ends remain visible after the session ends.

The pair agreed to treat OpenAPI as a catalog of legal routes, never as evidence of route weight. They also agreed that a previous load-test folder was historical material, not a default mix. No model was allowed to edit tests until the mix file had a hash in version control. That constraint felt slow at the start and cheap after the first dead end.

Questions the senior kept asking

The senior did not start with tools, frameworks, or model prompts, and that restraint shaped the rest of the session. Each question had to be answerable from logs, traces, or an explicit unknown, rather than from model fluency. The implementer wrote answers in the scratch buffer before anyone opened a code generator.

1. Name the ten-minute window this test may imitate, including weekday, hour, and traffic class.
2. List the routes that cover at least ninety percent of observed calls inside that window.
3. State the maximum concurrency the pair will still call realistic, rather than a coordinated stampede.
4. Separate latency budgets from sample observations, and refuse to let a generator blur those two numbers.
5. Define the failure when a generated client invents a route the mix file does not list.
6. Record who may refresh the mix file, and which pull request template must quote the new capture window.

After those answers, the senior still refused to let a model draft locust, k6, or pytest-benchmark glue. The unknown list was still longer than the known list, and that gap was treated as a blocker. Pairing continued into dead ends instead of into generated clients. The scratch buffer stayed the source of truth for what had already been rejected.

Dead ends the pair walked into

Dead end one treated a previous load-test repository as if it were current traffic from the same service. The old script used equal weights across every route, which flattered a checkout path that was rare in the weekday sample. Running it would have produced a confident p95 that described a synthetic crowd, not the captured window. The senior discarded the script because copying it would have laundered an invented mix through a familiar file name.

Dead end two asked a coding model to make the test realistic from the OpenAPI document alone. The generated client invented think times, a burst pattern, and a webhook route that the captured window never showed. The output compiled, named fixtures clearly, and still described a product that did not match production shape. The pair kept that prompt only as evidence of failure, then closed the branch without merging a single helper.

Dead end three tried to derive weights from handler counts in the router module, as if existence implied popularity. Route existence is not route popularity, and the senior rejected that proxy as a category error. A health endpoint and a catalog listing would have received similar weight under that rule, which the access sample immediately contradicted. The implementer then proposed sampling live traffic for ten minutes and writing weights by hand into a checked-in JSON file.

That third proposal survived because it produced an artifact a later gate could hash, diff, and refuse to skip. The senior asked one more question about freshness, then accepted a dated captured_at field as part of the contract. Pairing time spent on the three dead ends was shorter than the review time those dead ends would have created after merge.

The decision the pairing kept

The kept decision was narrow on purpose, so later models could not widen it through helpful extras. Generated tests may read fixtures/api_mix.json and must fail closed if the file is missing, empty, or lists no routes. Generated tests may not add routes, weights, think times, or concurrency values that the mix file does not already contain. Reviewers treat any pull request that hard-codes a mix in Python as a pairing violation, even when the numbers look conservative.

The decision also split two jobs that generators like to fuse. Capturing and editing the mix file remains a human pairing task with a dated window and a named source. Generating a loader, a schema check, and a client that only iterates the file is allowed after that freeze. If the file cannot explain a request, the client does not send that request.

Artifact: mix file, loader, and a refusing gate

The example fixture below is labeled sample data for the workflow, not a claim about one production cluster. Teams should replace the routes and weights with a window they actually captured. The notes field exists so a later reader can see the limit of the sample without rereading the pairing log.

{
  "captured_at": "2026-09-21T18:00:00Z",
  "source": "edge-access-log-sample",
  "window_seconds": 600,
  "routes": [
    {"method": "GET", "path": "/v1/catalog", "weight": 0.62, "p95_ms_observed": 48},
    {"method": "GET", "path": "/v1/catalog/{id}", "weight": 0.27, "p95_ms_observed": 61},
    {"method": "POST", "path": "/v1/checkout", "weight": 0.11, "p95_ms_observed": 220}
  ],
  "think_time_ms": {"p50": 80, "p95": 400},
  "max_concurrency": 12,
  "notes": "Weights from a ten-minute weekday sample. Not a capacity or soak test."
}

The loader below rejects unknown keys on each route and refuses to run when weights do not sum near one. It also refuses extra routes supplied by a caller, which is the usual way generated clients quietly widen the mix. The functions are ordinary Python so a later generator can wrap them without replacing the rules.

# mix_gate.py — labeled example, not production telemetry
from __future__ import annotations

import json
from pathlib import Path

REQUIRED_ROUTE_KEYS = {"method", "path", "weight", "p95_ms_observed"}
MIX_PATH = Path("fixtures/api_mix.json")


def load_mix(path: Path = MIX_PATH) -> dict:
    if not path.is_file():
        raise FileNotFoundError(f"mix file missing: {path}")
    data = json.loads(path.read_text(encoding="utf-8"))
    routes = data.get("routes") or []
    if not routes:
        raise ValueError("mix file lists no routes")
    total = 0.0
    for route in routes:
        extra = set(route) - REQUIRED_ROUTE_KEYS
        missing = REQUIRED_ROUTE_KEYS - set(route)
        if extra or missing:
            raise ValueError(f"route key mismatch extra={extra} missing={missing}")
        total += float(route["weight"])
    if abs(total - 1.0) > 0.02:
        raise ValueError(f"weights must sum to 1.0 +/- 0.02, got {total}")
    if int(data.get("max_concurrency", 0)) < 1:
        raise ValueError("max_concurrency must be >= 1")
    return data


def assert_request_allowed(mix: dict, method: str, path: str) -> None:
    allowed = {(r["method"], r["path"]) for r in mix["routes"]}
    if (method, path) not in allowed:
        raise AssertionError(f"request not in captured mix: {method} {path}")

The test module is the pairing gate. It fails before any HTTP client is imported, which keeps generated glue from shipping on an empty fixture. A second test walks a hostile list of invented routes and expects each one to raise.

# tests/test_mix_gate.py
import pytest
from mix_gate import assert_request_allowed, load_mix


def test_mix_file_loads_and_weights_sum():
    mix = load_mix()
    assert mix["window_seconds"] == 600
    assert mix["max_concurrency"] == 12


@pytest.mark.parametrize(
    "method,path",
    [
        ("DELETE", "/v1/catalog/{id}"),
        ("POST", "/v1/webhooks/retry"),
        ("GET", "/internal/debug/cache"),
    ],
)
def test_invented_routes_are_rejected(method, path):
    mix = load_mix()
    with pytest.raises(AssertionError, match="not in captured mix"):
        assert_request_allowed(mix, method, path)

Commands for the same gate stay boring on purpose. The pair ran them before any model was asked to write a client loop. A green result meant only that the freeze held, not that the service was fast.

mkdir -p fixtures tests
# save the JSON and Python files, then:
python -m pytest tests/test_mix_gate.py -q

Numbered workflow after the freeze

1. Capture one short access window and write route weights by hand into fixtures/api_mix.json.
2. Commit the mix file in its own change, with captured_at, source, and a note about what the window is not.
3. Add mix_gate.py and the refusing tests, and merge them before any load client exists in the tree.
4. Only then generate a client that iterates mix["routes"], sleeps using think_time_ms, and caps threads at max_concurrency.
5. Reject any generated patch that introduces a path string absent from the mix file, including “helpful” health checks.
6. Refresh the mix file through pairing, not through a model, when the window or the product shape changes.

The workflow is intentionally hostile to convenience. Convenience is how invented checkout bursts enter a suite that later becomes the team’s memory of production. A dated JSON file is a poorer story and a better input. Generated code can still save typing after that input exists.

Where a free model may enter

After the mix file and the refusing tests exist, an implementer can ask a coding assistant to draft only the loader wrappers and the client loop. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode’s free model access and free server option can host that narrow generation step once the pairing decision is already written down. The assistant should receive the mix file as read-only context and a rule that unseen routes are bugs, not features.

The product is not a substitute for the captured window, and it should not be asked to invent weights. Teams that skip the pairing freeze will still get fluent clients, and those clients will still lie in a confident tone. The useful split is human custody of the mix, then mechanical generation of iteration code. That split remains valuable if the assistant is later replaced by another tool.

Limitations

This approach does not measure capacity, saturation, or tail latency under incident load, because the sample window is short and observational. Weights from one weekday slice will mislead a holiday sale, a batch window, or a regional failover, and the gate cannot detect that semantic stale state. The loader also cannot prove that p95_ms_observed is a budget; it is only a number copied from a sample. Teams still need separate soak tests, error-injection tests, and human review of capture quality.

The pairing log does not replace authorization to read production logs, and this article does not provide a collection pipeline. If the service has no access sample, the honest move is to refuse generated load tests rather than to ask a model for realism. Equal-weight tests can still be useful as crash finders, but they should be named as crash finders. Calling them performance tests is the error the kept decision exists to prevent.

Who should not use this

Solo prototypes with no production traffic should not pretend a mix file is empirical. Protocol work that has never seen users yet should keep functional tests and skip shape-faithful load. Teams without log access, or without a senior who will veto invented routes, will only add ceremony around the same fiction. Security-sensitive environments should also keep capture, redaction, and fixture review on a path the pairing can audit.

The kept decision is small enough to reuse, and strict enough to annoy a generator that wants to be helpful. That annoyance is the point of the pairing session. A captured mix file is a poorer narrative than a confident client, and it is the only input the pair allowed. Teams that want the implementer side of this pairing to run against a free model on a free server can try MonkeyCode after the mix file is frozen.