The Agent Loop Was Fast. The Trace Lied.

Your agent loop is probably faster than you think. Mine looked slow for a week. Then I moved one timer and the slowness evaporated.

Here is the mistake. You wrap the entire round trip in one clock. The number comes back huge, so you blame the model and start shopping for a smaller one. Wrong layer. A model round trip holds two clocks, and one timer smears them together.

So stop timing. Start tracing.

Two clocks hide inside one box

Time-to-first-token is queue plus prefill. Everything after that first token is decode. If TTFT dominates, you have a scheduling problem. If decode dominates, you have an output-length problem. Those two fixes share nothing. Which one is yours? A total will never tell you.

Then there is a third clock nobody wraps. The tool call. Bash, tests, a git status. Those block your loop while doing zero inference. How often is your slow model really a slow subprocess?

You need three spans, not one timer.

The harness

Each span appends one JSONL line. Extra fields ride along on the record.

# trace.py
import json, os, time, httpx
from contextlib import contextmanager

TRACE = 'spans.jsonl'

@contextmanager
def span(name, **fields):
    rec = {'span': name, 'start_ns': time.perf_counter_ns(), **fields}
    try:
        yield rec                     # add keys to rec before the block ends
    finally:
        rec['dur_ms'] = (time.perf_counter_ns() - rec['start_ns']) / 1e6
        with open(TRACE, 'a') as f:
            f.write(json.dumps(rec) + '\n')

The streaming call is where the split happens. Mark the first content chunk, then keep counting.

def round_trip(messages, tool):
    with span('prompt_build') as s:
        payload = {'model': MODEL, 'messages': messages, 'stream': True,
                   'stream_options': {'include_usage': True}}
        s['payload_bytes'] = len(json.dumps(payload))

    text = ''
    with span('model_round_trip') as s:
        t0 = time.perf_counter_ns()
        with httpx.stream('POST', BASE + '/chat/completions', json=payload,
                          headers={'Authorization': 'Bearer ' + KEY},
                          timeout=120.0) as r:
            r.raise_for_status()
            for line in r.iter_lines():
                if not line.startswith('data: '):
                    continue
                body = line[6:]
                if body == '[DONE]':
                    break
                chunk = json.loads(body)
                delta = chunk['choices'][0]['delta'].get('content')
                if delta and 'ttft_ms' not in s:
                    s['ttft_ms'] = (time.perf_counter_ns() - t0) / 1e6
                text += delta or ''
                if chunk.get('usage'):
                    s['prompt_tokens'] = chunk['usage']['prompt_tokens']
                    s['out_tokens'] = chunk['usage']['completion_tokens']

    with span('tool_exec', tool=tool['name']) as s:
        ok = tool['fn'](text)
        s['ok'] = bool(ok)
    return text, ok

One ttft_ms per round trip is enough. You are looking for a ratio, not a thesis.

The graph you keep

Do not keep a dashboard. Keep one bar stack and one ratio line. Run this after thirty or forty turns.

# report.py
import collections, json, statistics

spans = [json.loads(l) for l in open('spans.jsonl')]
by = collections.defaultdict(list)
for s in spans:
    by[s['span']].append(s['dur_ms'])

top = max(statistics.median(v) for v in by.values())
for name, xs in sorted(by.items(), key=lambda kv: -statistics.median(kv[1])):
    xs.sort()
    p50, p95 = statistics.median(xs), xs[int(len(xs) * 0.95) - 1]
    bar = '#' * int(40 * p50 / top)
    print(f'{name:17} n={len(xs):4d} p50={p50:8.1f}ms p95={p95:8.1f}ms {bar}')

rt = [s for s in spans if s['span'] == 'model_round_trip' and s.get('ttft_ms')]
q = sum(s['ttft_ms'] for s in rt)
d = sum(s['dur_ms'] - s['ttft_ms'] for s in rt)
print(f'queue+prefill {q:8.0f}ms  decode {d:8.0f}ms  ratio {q/d:.2f}')

Illustrative sample output from the script above, not a measurement of your loop:

model_round_trip  n=  62 p50=  1840.2ms p95=  9110.4ms ########################################
tool_exec         n=  58 p50=   118.7ms p95=   640.1ms ##
prompt_build      n=  62 p50=     0.9ms p95=     3.2ms
queue+prefill    68400ms  decode    49600ms  ratio 1.38

Read the ratio first. Above 1 means waiting outweighs generating. Below 0.2 means you are paying to write long answers. Then read the p50 gap between spans. If tool_exec sits in your top two, your model was never the problem.

One caveat worth repeating. A client-side TTFT includes your own network round trip. You cannot split scheduler delay from transit without server timestamps. Treat it as a proxy and stay honest about it.

What each shape tells you to fix

Trace shape	Likely cause	First change
Ratio above 1, p50 growing per turn	Context resends the whole history	Hash the history, send a diff
p50 flat, p95 on model_round_trip huge	Retries and timeouts	Cap retries, set an explicit timeout
tool_exec p50 over 1s	Blocking shell or test run	Run tools concurrently, cache results
prompt_build in your top three	Re-serializing large payloads	Build once, reuse the bytes

Now the metric that actually matters. Round trips are motion. Accepted edits are progress. Divide your token usage by accepted edits, meaning patches that survive review or pass a test. If a fix cut p50 in half but doubled tokens per accepted edit, you made the loop cheaper and the work worse.

Where does MonkeyCode fit here? Nowhere in the loop logic, honestly. It fits as a place to run the harness. The operator states that model access is free within a 10M-token allowance and that a free server option is available, which is enough to collect a few thousand spans without your laptop fan becoming a variable. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

That allowance is an iteration budget, not an infinite one. The arithmetic is yours to run: 10,000,000 divided by your own tokens per accepted edit gives a rough ceiling on how many useful turns you get. I will not guess that number for you, because it depends entirely on how chatty your prompt is.

Limitations, and who should skip this

Client spans cannot separate network from server scheduling. If you need that, you need server-side tracing instead.

Appending a JSONL line per span is real IO. Buffer it, or sample every tenth turn in production. Do not leave this in a hot path.

A single client in a single region is not an SLO rig. It is also not a benchmark, no matter how nice the bars look. And if your real cost is human review time, no trace will show it, because the clock you forgot to wrap is the one sitting in a chair.

Finally, skip all of this if you already know your bottleneck. Instrumentation you do not read is just overhead with good intentions.

Point the harness at any endpoint you have credentials for and run it for an hour. If you want somewhere free to spend your first thousand spans, MonkeyCode's free tier is one option, but start the trace before you judge it. You might find out the model was fine all along.

MonkeyCode provides free models that can run this workflow.