Your Agent Assumed the Schema. That Was the Bug

The model did not invent that broken payload. Your tool contract left a hole. I still watch teams debug the wrong layer first. They swap models before they freeze a single field. They add system prompts instead of required keys. Freeze the schema first. Then judge the model.

Why start with contracts, not prompts? Free models fail in a loud way. They will not hide a sloppy tool contract. That loud failure is a useful feature today. Use it before you spend a paid token.

What this piece actually covers

This is a catalog, not a launch post. Four anti-patterns. Symptoms. Root cause. A replacement you can ship. Then a tiny contract suite you can run tonight.

I mention a free model server once, late. It is only a cheap place to fail. Skip that part if you already own an endpoint.

Anti-pattern 1: Optional everything

Symptom

The tool schema marks every field optional. The agent still reports a clean success. Your logs show three shapes for one tool. Downstream code then parses a maybe-object.

Ask yourself a blunt question here. Did the model pick a field? Or did your schema refuse to choose?

Root cause

You copied a fat OpenAPI dump into the tool list. Optional fields feel safer than required ones. The model fills that vacuum with a confident guess. Your harness accepts the guess because JSON still parses.

Replacement

Mark the real inputs required, not polite. Split rare fields into a second tool. Reject extra keys. Do not coerce them into defaults.

{
  "name": "repo_search",
  "parameters": {
    "type": "object",
    "additionalProperties": false,
    "required": ["query", "path"],
    "properties": {
      "query": { "type": "string", "minLength": 2, "maxLength": 200 },
      "path": { "type": "string", "pattern": "^src/" },
      "limit": { "type": "integer", "minimum": 1, "maximum": 20, "default": 8 }
    }
  }
}

See additionalProperties: false in that block? That one line kills a whole drift class. If a field is truly rare, it is another tool.

Anti-pattern 2: Stringly-typed tools

Symptom

One parameter. Type string. You named it payload or json. The model pastes a novel into it. Your parser throws on the second brace. Then you blame the weights.

Sound familiar in your last trace?

Root cause

Nested objects felt annoying inside the harness. A string always “validates” at the edge. Real validation moved to runtime, after the call. The model never saw the shape you actually needed.

Replacement

Declare the object in the tool list. Validate arguments before any HTTP call. If you need an escape hatch, version that hatch.

# labeled: proposal for your own harness
REQUIRED = {"query", "path"}
ALLOWED = {"query", "path", "limit"}

def assert_tool_args(name: str, args: dict) -> None:
    if name != "repo_search":
        raise AssertionError(f"unknown tool: {name}")
    missing = REQUIRED - set(args)
    if missing:
        raise AssertionError(f"missing: {sorted(missing)}")
    if not isinstance(args["query"], str):
        raise AssertionError("query must be str")
    extra = set(args) - ALLOWED
    if extra:
        raise AssertionError(f"extra keys: {sorted(extra)}")

Run this guard before the network hop. Not after the model “looks right” in a screenshot.

Anti-pattern 3: HTTP 200 with a buried error

Symptom

The tool returns status 200 every time. The body still contains "error": "not found". The agent writes a patch against empty hits. Your diff looks busy. Your tests stay red.

Why would the loop stop there? You labeled the call a success.

Root cause

You wrapped every exception in a friendly JSON blob. The model reads tone, not protocol. A 200 is a green light with extra words. Retries then amplify the wrong branch.

Replacement

Map failures to explicit tool errors. Keep a tiny error taxonomy. Four codes are enough for file tools.

Code	Meaning	Agent should
INVALID_ARGS	Schema failed	Fix args, retry once
NOT_FOUND	Path or query missed	Ask, do not invent files
FORBIDDEN	Path outside allowlist	Stop the loop
TRANSIENT	Timeout or lock	Retry with backoff

class ToolError(Exception):
    def __init__(self, code: str, message: str):
        self.code = code
        self.message = message

def run_repo_search(args: dict) -> dict:
    try:
        assert_tool_args("repo_search", args)
    except AssertionError as exc:
        raise ToolError("INVALID_ARGS", str(exc)) from exc
    path = args["path"]
    if not path.startswith("src/"):
        raise ToolError("FORBIDDEN", "path must start with src/")
    hits = search(args["query"], path, args.get("limit", 8))
    if not hits:
        raise ToolError("NOT_FOUND", "no files matched")
    return {"hits": hits, "count": len(hits)}

Return data or raise a coded error. Never both in one body.

Anti-pattern 4: Unbounded tool dumps

Symptom

One read_file call returns twelve thousand lines. The next model turn is garbage. You call it context rot in the standup. Is it rot, or did you hose the window?

Root cause

The tool has no byte budget at all. The model cannot refuse a huge result. Your loop then “summarizes” the damage with another call. Tokens vanish. The schema still looks fine.

Replacement

Cap bytes on every read. Return a cursor. Force a second, smaller call.

MAX_BYTES = 8_000

def read_slice(path: str, offset: int = 0) -> dict:
    if ".." in path or path.startswith("/"):
        raise ToolError("FORBIDDEN", "path not allowed")
    with open(path, "r", encoding="utf-8") as handle:
        data = handle.read()
    chunk = data[offset: offset + MAX_BYTES]
    return {
        "path": path,
        "offset": offset,
        "bytes": len(chunk.encode("utf-8")),
        "truncated": (offset + MAX_BYTES) < len(data),
        "next_offset": offset + len(chunk),
        "text": chunk,
    }

Truncation is a signal the agent can use. Hide it, and the agent keeps guessing missing lines.

The artifact: contract tests, not a vibe check

I do not “try a prompt” to prove tools work. I freeze three fixtures and fail the build. No model sits in this loop. If this suite is red, stop blaming tokens.

# labeled: reproducible contract checks
CASES = [
    {
        "name": "missing_path",
        "args": {"query": "ToolError"},
        "expect": "INVALID_ARGS",
    },
    {
        "name": "path_escape",
        "args": {"query": "foo", "path": "../secrets"},
        "expect": "FORBIDDEN",
    },
    {
        "name": "empty_hits",
        "args": {"query": "zzz-no-hit", "path": "src/"},
        "expect": "NOT_FOUND",
    },
]

def classify(exc):
    if isinstance(exc, ToolError):
        return exc.code
    raise

def test_tool_contracts():
    for case in CASES:
        try:
            run_repo_search(case["args"])
        except Exception as exc:
            got = classify(exc)
            assert got == case["expect"], (case["name"], got)
            continue
        raise AssertionError(f"{case['name']} should fail")

Three cases cover the lies I see most. Missing required keys. Path escape. Empty result treated as success. Add your own poison next.

Want a command, not a story?

python -m pytest test_tool_contracts.py -q

Green means the runtime tells the truth. Red means the agent never had a chance.

Where a free model server actually helps

Need a live model after the suite is green? I use a free endpoint for that next layer.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. I treat that box as a schema furnace, not a demo stage. Same tools. Same byte caps. Same error codes. If the free model invents keys, the contract is still too loose. If it stops cleanly, the schema is doing real work.

I am not claiming a quota, a ranking, or a hardware spec. I am claiming a workflow you can copy. Fail on a free server first. Pay only after traces stay boring.

# labeled: local workflow; swap in your endpoint
python -m pytest test_tool_contracts.py -q
python replay_traces.py --endpoint "$FREE_SERVER_URL" --limit 20

Replay frozen traces, not vibes from chat. Twenty pinned tasks beat one flashy gist.

Decision table I keep beside the harness

If you see	Do not	Do instead
Extra JSON keys	Add a prompt rule	Set additionalProperties: false
Parse errors on payload	Switch models	Kill stringly-typed tools
Patches on 200 plus error	Write “be careful”	Raise ToolError codes
Context explosions	Summarize harder	Cap bytes and return a cursor
Flaky retries	Raise temperature	Split INVALID_ARGS from TRANSIENT

Print the table. Stick it on the harness. Argue with the row, not the model card.

Limitations

This will not fix a bad retrieval index. This will not catch prompt injection. This will not make a weak model design a new architecture. Contract tests only prove the tool told the truth.

The suite assumes you own the tool runtime. It assumes a single-process harness on disk. It assumes file tools, not billing APIs with side effects. Schemas drift across pull requests. Re-run the suite on every tool change. A green run yesterday is not a green run today.

Who should not use this

Do not use this if you cannot pin tool versions. Do not use this if every tool is a raw shell. Do not use this if you need multi-tenant isolation tomorrow. Those are product problems, not schema nits.

If your agent is a cron job with two branches, stop. You do not have a schema problem. You have a script. Keep the script. Skip the catalog.

What I want you to run tonight

Pick one tool. Only one. Make three fields required. Add additionalProperties: false. Write three fixtures that must fail. Then, if you want a cheap live check, point the same harness at a free MonkeyCode server and replay traces.

That is the whole move. The model was never the first bug.