If the Round Trip Was Instant, You Cheated

Your agent did not pass. It never waited for anything.
Local tool calls hide timeouts, retries, and partial payloads.
An instant round trip is not evidence. It is a missing network.

Instant success is a different program

You run the agent on your laptop first.
Every tool is a function in the same process.
The fake API returns in one millisecond. No packet left the box.

That program is not the one you will ship.
The shipped program waits, aborts, retries, and sometimes gets half a JSON body.
Those branches never run when the network is a function call.

Browser-only agent demos make the cheat quieter.
The loop stays in one origin, one event loop, one warm cache.
You are grading a rehearsal, not a call path.

The claim you should argue with

Here is a clear position, not a hedge.
If your eval never blocked on I/O, you do not have an eval.
You only have a unit test of prompt text.

Timeouts are not a late ops concern.
They change which tool the model chooses next.
Retries can duplicate side effects on the wire.
Slow tools starve later steps under a wall clock.

You cannot see that failure on localhost.
A green local run can still be a silent network.

Record waits, not just answers

Stop logging only the final assistant string.
Log the wait that produced each tool result.
If you cannot replay the waits, you cannot debug the agent.

Capture at least these fields on every call:

• tool name and attempt number
• monotonic start time and end time
• timeout budget for that attempt
• abort flag and retry flag
• bytes received before the first parse
• HTTP status or transport error, if any

If attempt number is always one, you learned little.
If duration is always under 20ms, you learned nothing.
That second case is the silent cheat in most laptop evals.

Three waits you must treat as bugs

Do not dump every delay into one p95 bucket.
Name the wait, then write an assertion for it.

1. Deadline miss. The tool exceeds its budget. The agent must abort and choose a fallback. A hang is a failed eval, not a slow pass.
2. Retry storm. The first call dies. The second call repeats a POST. Unless you sent an idempotency key, you may have charged a card twice.
3. Partial body. Bytes arrive, then the socket dies. Your parser throws. The agent must not treat a truncated object as success.

Localhost almost never produces case two or three.
A remote hop produces them without extra theatre.
That is why a quiet laptop is a biased lab.

Copy this harness

The following is a proposed example, not a scored benchmark.
Wire it to your real tool runner before you trust it.
Do not treat the 20ms cutoff as a universal law.

# latency_harness.py
# Example only. Enforce timeout_s inside your real transport.

from __future__ import annotations

import json
import time
import uuid
from dataclasses import asdict, dataclass
from typing import Callable

@dataclass
class ToolEvent:
    run_id: str
    tool: str
    attempt: int
    timeout_s: float
    started_at: float
    ended_at: float
    aborted: bool
    error: str | None
    bytes_in: int
    implausible_local: bool

    @property
    def wait_ms(self) -> float:
        return (self.ended_at - self.started_at) * 1000.0


IMPLAUSIBLE_MS = 20.0  # example: in-process cheat detector


def call_with_budget(
    run_id: str,
    tool: str,
    attempt: int,
    timeout_s: float,
    fn: Callable[[], bytes],
) -> ToolEvent:
    started = time.monotonic()
    aborted = False
    error = None
    payload = b""
    try:
        payload = fn()
    except TimeoutError as exc:
        aborted = True
        error = str(exc)
    except Exception as exc:
        error = str(exc)
    ended = time.monotonic()
    wait_ms = (ended - started) * 1000.0
    return ToolEvent(
        run_id=run_id,
        tool=tool,
        attempt=attempt,
        timeout_s=timeout_s,
        started_at=started,
        ended_at=ended,
        aborted=aborted,
        error=error,
        bytes_in=len(payload),
        implausible_local=wait_ms < IMPLAUSIBLE_MS and error is None,
    )


def write_trace(path: str, events: list[ToolEvent]) -> None:
    with open(path, "w", encoding="utf-8") as handle:
        for event in events:
            handle.write(json.dumps(asdict(event)) + "\n")


def assert_eval_was_real(events: list[ToolEvent]) -> None:
    if not events:
        raise AssertionError("no tool events; this is not an agent eval")
    waits = [e.wait_ms for e in events]
    if all(w < IMPLAUSIBLE_MS for w in waits):
        raise AssertionError(
            "every tool returned under 20ms; you never left the process"
        )
    if all(e.attempt == 1 and not e.aborted for e in events):
        print("warning: no abort or retry observed")

Keep the assertion in CI, not in a slide.

# test_agent_waits.py
# Example assertions. Replace fake_remote() with a real invocation.

import uuid
from latency_harness import assert_eval_was_real, call_with_budget


def test_remote_search_must_honor_a_deadline():
    run_id = str(uuid.uuid4())

    def fake_remote() -> bytes:
        raise TimeoutError("deadline exceeded")

    event = call_with_budget(run_id, "search", 1, 8.0, fake_remote)
    assert event.aborted is True
    assert event.error is not None


def test_reject_all_instant_successes():
    run_id = str(uuid.uuid4())
    events = [
        call_with_budget(run_id, "search", 1, 8.0, lambda: b'{"ok":true}')
    ]
    try:
        assert_eval_was_real(events)
        raise AssertionError("instant local success should not pass")
    except AssertionError as exc:
        assert "never left the process" in str(exc)

Run the tests the boring way:

python -m pytest test_agent_waits.py -q

You still need two traces, not one green file.
One trace from the laptop process.
One trace from a host that is not your laptop.

python - <<'PY'
import json, sys

def load(path):
    with open(path) as f:
        return [json.loads(line) for line in f]

local = load("trace-local.jsonl")
remote = load("trace-remote.jsonl")
local_waits = [row["ended_at"] - row["started_at"] for row in local]
remote_waits = [row["ended_at"] - row["started_at"] for row in remote]

if local and max(local_waits) < 0.02 and remote and max(remote_waits) > 0.2:
    print("local eval was a rehearsal; remote eval actually waited")
    sys.exit(1)
print("next: diff abort flags and attempt counts")
PY

That comparison script is a proposal.
Tune the numbers after you collect your own traces.
Do not copy 20ms into production policy blindly.

A decision table, not a vibe

Use this table before you trust a green agent build.

Signal you recorded	What it usually means	Ship it?
All tools under 20ms, same process	You tested a function, not a network	No
Remote p95 is high, no timeout handler	The model will hang or skip steps	No
Timeout fires, retry repeats a POST	You may have duplicated a side effect	No
Timeout fires, retry sends an idempotency key	You have a real recovery path	Maybe
Abort mid-body, parser throws, agent stops	Partial JSON is an untested branch	No
Abort mid-body, agent requests a retry tool	You tested the ugly path	Yes, if traces agree
Browser demo only, no remote hop	You tested UI glue	No for backend agents
Local and remote traces pick different tools	The pass was environment-specific	No

Read the last row twice.
Latency is not the only drift you will see.
Tool choice drifts as soon as waits appear.
The model is not independent of time.

Idempotency is part of the latency test

A timeout without an idempotency key is an incident draft.
You do not know whether the first call committed.
The agent also does not know. It will guess.

Give every mutating tool a key you control:

# Example header. Your API may use a different name.
headers = {
    "Idempotency-Key": f"{run_id}:{tool}:{attempt}",
}

Then assert the remote trace.
A retry must reuse the key, not mint a second one.
If the key changes on attempt two, fail the job.
That bug is invisible when the first call never dies.

Why a free remote hop belongs in this method

You do not need a cluster for this control.
You need a machine that is not yours.
You also need a model endpoint that is not a local stub.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option.
That pair is useful as a control, not as a slogan.
Run the same harness against the free server.
Keep the laptop run as the suspect baseline.

If the free remote trace never waits either, your tools are still in-process.
If it waits and your code has no abort path, you found the bug.
That is the only reason a free box belongs in this article.

Do not treat free access as a scoreboard.
Do not treat it as a capacity promise.
Use it once to make the network real.

Limitations

This approach will waste time in some shops.
Skip it when the wait cannot exist.

Skip it if:

• your agent has no tools and only writes text
• every tool already has an injected delay in unit tests
• you already run evals in an isolated remote job with real deadlines
• policy forbids sending traces or prompts off the laptop

Do not mix unlike waits in one assertion.
A 20ms cutoff only hunts in-process cheats.
A multi-second DNS blip is a different test.
This harness does not prove task correctness.
It only proves the agent was forced to wait.

No speedup is claimed here.
No model ranking is claimed here.
The claim is narrower: your green bar may be a missing sleep.

What you should do tomorrow

Pick one agent path that always works locally.
Wrap each tool with the event recorder.
Run it on the laptop. Save trace-local.jsonl.

Then run the same path on a host you did not bootstrap by hand.
Save trace-remote.jsonl.
Diff abort flags, attempt counts, and wait buckets.

If local waits are all tiny, fail the job.
If remote waits exist and your code ignores them, fail the job.
If both traces wait and recover, you finally have an eval.

A free remote endpoint is enough to start that control.
If you already have MonkeyCode's free server, point the harness there once and keep the traces.

Do not announce a pass until something had to wait.