Green Agent Loops Never End. Require a Terminal Span

Here is a reconstructed staging incident from logs.
A staging bot was asked to open one pull request.
Every recorded tool span still reported status ok.

The promised pull request never appeared on GitHub.
The run lasted forty-one minutes after the last write.
Logs kept growing while the heartbeats stayed pretty.

No span ever said the loop was finished.
Operators marked the job successful anyway.
Success meant "no exception," not "goal met."

Green spans are not a stop signal

Most agent traces record tools instead of goals.
A read_file span can succeed one hundred times.
That success does not mean the job completed.

Dashboards hide the gap behind a zero error rate.
People watch errors and miss runs that never stop.
The process can stay busy until someone kills it.

You need a contract the trace can fail.
The failure contract remains simple and locally testable.
Every run must emit one terminal span.

What a terminal span must prove

A terminal span is not another tool result.
It names why the loop stopped acting.
It also carries the iteration that stopped.

Accept only these stop reasons at first:

• done means the stated goal was actually met
• rejected means a guard blocked the next act
• budget means iteration or time ran out
• blocked means a tool returned a hard failure
• handoff means another run now owns the goal

If the stream ends without one reason, the run is open.
Open runs are incidents, not quiet successes.
Treat them like missing health checks on a service.

Decision table for a single run

Observation	Class	Action
last span is a tool with ok	open_loop	cap the run and page
terminal done and iter within budget	closed_ok	keep a sample only
terminal reason equals budget	closed_cap	inspect the last plan
two terminal spans in one run	double_stop	fix the emitter
iteration sequence contains gaps	bad_clock	sort by seq, not time
same args_hash for three iterations	no_progress	halt on the next match

Do not fold these classes into HTTP status codes.
A tool HTTP 200 is not done.
The done flag is a claim about the goal.

JSONL schema you can emit tonight

Keep one JSON object on each line.
Do not pretty-print objects inside the file.
Stable keys beat nested vendor blobs every time.

{"seq":1,"ts":"2026-09-11T02:00:01Z","run_id":"run_7f3","span_id":"sp_01","type":"loop_start","iter":0,"goal":"open_pr"}
{"seq":2,"ts":"2026-09-11T02:00:03Z","run_id":"run_7f3","span_id":"sp_02","parent_id":"sp_01","type":"tool","iter":1,"name":"read_file","args_hash":"a91c","result_hash":"b33e","status":"ok"}
{"seq":3,"ts":"2026-09-11T02:00:08Z","run_id":"run_7f3","span_id":"sp_03","parent_id":"sp_01","type":"tool","iter":2,"name":"edit_file","args_hash":"c10d","result_hash":"c10d","status":"ok"}
{"seq":4,"ts":"2026-09-11T02:41:12Z","run_id":"run_7f3","span_id":"sp_88","parent_id":"sp_01","type":"terminal","iter":38,"reason":"budget","status":"ok","summary":"no_pr_url"}

The seq field is the order the emitter observed.
Do not trust wall clocks across two hosts.
The args_hash field is a short digest of arguments.

Never write secrets into this trace file.
Hash payloads and drop tokens before flush.
A debug loop that leaks keys is worse than silence.

Reproducible checker

The script below is a local proposal.
The script reads JSONL records from standard input.
It prints one verdict line for each run_id.

Treat the listing as an unexecuted local example.
Wire your own emitter before you trust results.

#!/usr/bin/env python3
"""Fail closed when an agent run never terminals."""
from __future__ import annotations

import json
import sys
from collections import defaultdict

BUDGET = 24
NO_PROGRESS_WINDOW = 3
TERMINAL_REASONS = {"done", "rejected", "budget", "blocked", "handoff"}
FAIL = {
    "open_loop",
    "no_progress",
    "double_stop",
    "bad_terminal",
    "bad_clock",
}


def load_spans(stream):
    runs = defaultdict(list)
    for line_no, raw in enumerate(stream, 1):
        text = raw.strip()
        if not text:
            continue
        span = json.loads(text)
        span["_line"] = line_no
        runs[span["run_id"]].append(span)
    return runs


def tool_arg_hashes(spans):
    values = []
    for span in spans:
        if span.get("type") != "tool":
            continue
        values.append(span.get("args_hash") or "")
    return values


def classify(spans):
    ordered = sorted(spans, key=lambda item: item.get("seq", item["_line"]))
    iters = [item.get("iter") for item in ordered if item.get("iter") is not None]
    terminals = [item for item in ordered if item.get("type") == "terminal"]
    max_iter = max(iters) if iters else 0

    if len(terminals) > 1:
        return "double_stop", max_iter

    for left, right in zip(iters, iters[1:]):
        if left + 1 < right:
            return "bad_clock", max_iter

    chain = tool_arg_hashes(ordered)
    if len(chain) >= NO_PROGRESS_WINDOW:
        tail = chain[-NO_PROGRESS_WINDOW:]
        if tail[0] and len(set(tail)) == 1:
            return "no_progress", max_iter

    if not terminals:
        return "open_loop", max_iter

    reason = terminals[0].get("reason")
    if reason not in TERMINAL_REASONS:
        return "bad_terminal", max_iter
    if reason == "done" and max_iter <= BUDGET:
        return "closed_ok", max_iter
    if reason == "budget":
        return "closed_cap", max_iter
    return f"closed_{reason}", max_iter


def main() -> int:
    failed = 0
    runs = load_spans(sys.stdin)
    for run_id, spans in sorted(runs.items()):
        verdict, max_iter = classify(spans)
        print(f"{run_id}\t{verdict}\titer={max_iter}\tspans={len(spans)}")
        if verdict in FAIL:
            failed = 1
    return failed


if __name__ == "__main__":
    raise SystemExit(main())

Save and run

Save the file as terminal_contract.py beside fixtures.
Pipe a known-bad trace through it first.
Confirm a non-zero exit before wiring CI.

python3 terminal_contract.py < traces.jsonl
echo $?

A non-zero exit means some run violated the contract.
Put that exit code on recorded fixtures in CI.
Do not wait for a human to scroll JSONL by hand.

Known-bad fixture

This tiny fixture mimics the staging incident.
There is no terminal span in the stream.
Iteration three already repeats the same edit hash.

{"seq":1,"run_id":"run_bad","type":"loop_start","iter":0,"goal":"open_pr"}
{"seq":2,"run_id":"run_bad","type":"tool","iter":1,"name":"edit_file","args_hash":"aaaa","status":"ok"}
{"seq":3,"run_id":"run_bad","type":"tool","iter":2,"name":"edit_file","args_hash":"aaaa","status":"ok"}
{"seq":4,"run_id":"run_bad","type":"tool","iter":3,"name":"edit_file","args_hash":"aaaa","status":"ok"}

Expected classification is no_progress for run_bad.
The checker should exit with a non-zero status.
If it prints closed_ok, your emitter is lying.

Where a quiet server actually helps

The checker itself is CPU-light and sequential.
It needs a box that stays up between agent runs.
A sleeping laptop will drop the JSONL tail.

MonkeyCode's free server option fits that collector role.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Park the tailer there and keep fixtures beside it.

Free model access is optional on the same box.
Use a model only after the contract already failed.
Ask it to cluster summary strings, not to invent spans.

Do not send raw tool payloads to any model.
Send hashes, tool names, and stop reasons only.
The verdict must still come from the script.

A debug loop you can reuse

1. Emit loop_start with an explicit goal field.
2. Stamp every tool span with iter plus args_hash.
3. Emit exactly one terminal span when work stops.
4. Run terminal_contract.py against the JSONL file.
5. Freeze the run on open_loop or no_progress.
6. Label leftover summaries with a model last.

This loop does not replace a tracing backend.
It is a closed-world test on your own JSONL.
Start here, then export spans if files grow.

What this does not catch

The contract cannot see side effects outside the trace.
A done span can still lack a real pull request.
Verify the artifact with a separate probe each time.

Short hashes can hide real argument drift under collision.
Use a longer digest if tests ever collide.
Do not treat four hex characters as a security boundary.

Wall-clock sorting will mis-order concurrent emitters.
That is why seq exists in the schema.
If seq is missing, the checker becomes a guess.

Parallel tools may share one iteration on purpose.
The gap check allows repeated iteration numbers on purpose.
It only flags skipped numbers in the sequence.

This is also not retry accounting on one tool.
Retries share one tool goal and one family of attempts.
Loop stalls keep planning and still go nowhere new.

Who should skip this approach

Skip this if your runs have no stable run_id.
Skip this if legal holds need immutable audit stores.
Skip this if traces still contain live credentials.

Multi-tenant platforms need redaction before any JSONL drop.
This script does not redact fields for you.
Add that layer first, or do not collect traces.

Teams without a hard iteration budget will ignore verdicts.
Pick a number and write it in the file header.
A budget you never enforce is decoration on logs.

Practical next step

Add type=terminal to one agent tonight.
Replay last week's traces if those files still exist.
Count how many green jobs never actually stopped.