The Truncated Tool Call That Passed Validation

Illustrative scenario, not a real incident. Picture a nightly job that asks a model to emit a JSON tool call, then writes the result into a config table. One morning the config table contains half a record, the job exited zero, and nobody noticed for two days because the row still parsed.

This walkthrough is about that specific failure shape: a response cut off at an output cap, a permissive parser, and a pipeline with no notion of how much it had already spent. You can reproduce the diagnosis on any laptop, and you can rehearse the fix on a free tier before you touch production.

What the postmortem would actually contain

The trigger is boring, which is the point. The model hit its output ceiling mid-object, the API returned a finish reason that meant "stopped early", and your code only checked whether the body was non-empty. Three contributing factors usually stack here: an output cap set for cost rather than for safety, a parser that tolerates partial input, and a writer with no transaction boundary around the result set.

None of those is individually fatal. Together they turn a rate-limit hiccup into silent data loss, because every layer assumed the layer above it had verified completeness.

Start from the finish reason, not the diff

Before you read a single line of your prompt, confirm what the provider actually told you. The field name varies by vendor, so check your provider's response schema rather than trusting a copied snippet.

# illustrative triage, adapt paths and field names to your logs
grep -nE '"(finish_reason|stop_reason)"' agent.log | tail -n 40
grep -c 'max_tokens' agent.log
git log --oneline -n 20 -- parser/ writer/

If the grep shows a cluster of early-stop reasons on one prompt version, you have found the boundary. If it shows nothing, your logging is dropping the field, and that is the first thing to fix, because you cannot reason about completeness you never recorded.

A parser that refuses partial input

The fix that ages best is a parser whose default answer is "no". The version below is illustrative and has not been executed; treat it as a shape to adapt, not a library to import.

# illustrative only — not executed, not tested against any provider
import json

class TruncatedOutput(ValueError):
    pass

def parse_tool_call(raw: str, *, required: tuple[str, ...]) -> dict:
    # A closed object is a cheap proxy for "the stream finished".
    if not raw.strip().endswith("}"):
        raise TruncatedOutput("output does not close its JSON object")
    payload = json.loads(raw)  # raises on malformed bodies
    missing = [k for k in required if k not in payload]
    if missing:
        raise TruncatedOutput(f"missing required keys: {missing}")
    return payload

The endswith check is deliberately crude, and that is fine, because it fails loudly instead of writing junk. Pair it with a schema validator on the caller side, and put the database write inside one transaction so a rejected payload leaves no half-row behind.

A budget guard you can reason about

The second half of the fix is accounting. Before the job may call the model again, it needs to know what it has already spent against a cap it can name.

# illustrative only — counters come from your own ledger
class Budget:
    def __init__(self, cap_tokens: int, reserve_tokens: int = 0) -> None:
        self.cap = cap_tokens
        self.reserve = reserve_tokens
        self.spent = 0

    def can_spend(self, estimate: int) -> bool:
        return self.spent + estimate + self.reserve < self.cap

    def record(self, used: int) -> None:
        self.spent += used

The reserve matters more than the cap. It is what lets a failed attempt be retried once with a smaller output ceiling, instead of the job either dying or spinning until an external timeout kills it.

Rehearsing the replay loop cheaply

Here is where a free model tier and a free server genuinely help: you want to re-run the same prompts against parser variants many times, and you do not want that loop living on your laptop or your production quota. MonkeyCode is described by its operator as an open-source project offering free model access and a free server option; the specific models, allowances and server limits are the operator's stated terms, so check the project's current documentation before you build a plan around them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A replay harness does not need to be clever. It needs to fail visibly.

# illustrative replay harness — unexecuted
set -euo pipefail
mkdir -p out
for p in prompts/*.txt; do
  name=$(basename "$p" .txt)
  ./run_agent.sh --prompt "$p" --max-output-tokens 800 --json > "out/$name.json" || true
  python -m json.tool "out/$name.json" > /dev/null 2>&1 \
    || echo "INVALID_OUTPUT: $name"
done

Run it against a deliberately tiny output cap first. You want the harness to report invalid output, because a replay suite that never fails has told you nothing about the bug you are chasing.

Choosing where the loop should run

The table below is a decision aid, not a benchmark. It encodes the trade-off I would apply, and you should re-derive the numbers for your own workload.

Situation	Free tier is a reasonable fit	Pay for capacity instead
Re-running fixed prompts after a parser change	Yes, volume is bursty and uncritical	—
Latency-sensitive user-facing path	—	Yes, you need predictable tail latency
Regulated or customer-identifiable data	—	Yes, check data-handling terms first
Long-context evaluation sweeps	Only if the stated allowance covers it	Yes, sweeps exhaust caps fast
One-off reproduction of a reported bug	Yes, this is the best case	—

Who should not use this approach

If your pipeline needs a throughput guarantee, a named model version pinned for reproducibility, or a signed data-processing agreement, a free tier is the wrong foundation and no amount of harness discipline fixes that. The same applies if your incident involves customer data that may not leave your boundary at all.

And if your parser already validates strictly, the interesting part of this article is the budget guard rather than the JSON check. Read the failure mode, not the code.

How you verify the fix landed

Write three tests before you ship: a truncated body must raise, an object missing a required key must raise, and a valid payload must reach the writer exactly once inside a transaction. Then re-run the replay harness with the old cap and confirm it now reports invalid output instead of writing rows, because that inversion is the actual evidence that your validation changed behaviour.

If you want to try that replay loop somewhere disposable, the project's free server option is the least wasteful place to start — just confirm the current terms yourself first, since allowances are the kind of thing that changes quietly.