The Pairing Froze a Search Tool Schema Before Generated Clients Could Call Out

A generated tool client should not call a live API until the pairing writes a frozen schema and a failing contract test. The session below reconstructs that sequence as a worked example, not as a production war story with invented metrics. The senior engineer treated every extra field, extra tool, and extra HTTP verb as a defect, even when the draft compiled. The kept decision was a one-page schema freeze that both humans and any later model had to obey.

Why the pairing started with a schema, not a client

The team needed a small Python client that would call one internal search tool through a model-mediated function call. Generated drafts looked complete, imported cleanly, and printed plausible JSON, which made the danger easy to miss. The senior pairing partner refused to review implementation files until a schema file existed in the repository. That refusal became the working method of the pairing, and it later blocked an invented second tool.

Tool calling is now a common way models reach HTTP APIs, which makes invented function names a merge-time defect instead of a demo joke. A client that compiles can still post the wrong verb, require a field the server ignores, or swallow an error envelope the on-call playbook does not know. The pairing therefore treated the JSON schema as the product, and treated generated Python as a disposable adapter. Clear names in the freeze file mattered more than tidy generated comments in the client module.

Questions the senior asked before any model ran

The senior did not start with a prompt. The senior started with a short oral checklist that had to produce written answers in the repository. Those answers were copied into contracts/search_tool.json before anyone opened a chat panel. The pairing recorded the questions as statements so later reviewers could audit them without watching the call.

1. Name the single tool the production gateway actually exposes, including the exact spelling the router already uses.
2. List required arguments, optional arguments, and arguments the server must never receive from a client.
3. State the HTTP method, path, and content type the gateway already documents for that one tool.
4. Describe the success envelope and the error envelope in field names, not in prose about likely status codes.
5. Decide which generated file is allowed to import the freeze file, and which files must stay empty until tests fail.

Each answer had to fit in the freeze file with no commentary about future tools. The junior engineer wanted a generic invoke_tool() helper that could grow with the model. The senior rejected that helper because growth was the failure mode the pairing was there to stop. One named tool stayed in scope for the rest of the afternoon.

Dead ends the pairing walked into and then discarded

The first dead end was a generated client that added list_indexes beside search_docs because the prompt mentioned a corpus. The extra tool looked helpful, imported without error, and would have been a production 404 with a confident traceback. The senior asked the junior to delete the file instead of commenting out the extra function. Deletion was the review comment, because leftover helpers invite the next model to revive them.

The second dead end was a required limit field that the live search endpoint treats as optional and caps server-side. The generated client refused to send a query without limit, which would have broken an existing caller that only passes q. The pairing restored limit as optional with a maximum of 50 in the freeze file. The client was not allowed to invent a default that the gateway did not already apply.

The third dead end was a polite GET with query parameters, which matched many public search APIs and matched nothing on this gateway. The documented route was POST /v1/tools/search_docs with a JSON body. The senior made the junior write that method and path as literals in the freeze file, not as comments above a requests call. After that change, any generated GET became a test failure rather than a style debate.

The fourth dead end was an invented error body with a retry_after field copied from unrelated HTTP client examples. The gateway returns { "error": { "code": "string", "message": "string" } } and nothing else. The pairing added a strict decoder that rejects unknown keys. Unknown keys are how generated clients smuggle policy that nobody on-call agreed to run.

The decision the pairing kept

The kept decision was written in the pull request, not in the chat transcript. It said the repository would ship one tool named search_docs, one freeze file, and one contract test that fails on extra tools, extra required fields, extra HTTP methods, and extra error keys. Generated clients could exist only as adapters that import that freeze file. Models could propose Python after the test failed, and they could not propose a second schema.

Signal in a draft	Pairing action	Reason kept in the freeze
New tool name	Delete the file	The gateway exposes one search tool
New required field	Fail the test	Callers already omit optional limit
GET or query string	Fail the test	The route is POST with a JSON body
Extra error key	Fail the test	On-call runbooks decode code and message
Helper named invoke_tool	Reject the helper	Genericity hides invented verbs

That table is the entire review policy for this worked example. It is boring on purpose, because boredom is cheaper than a wrong POST from a fluent client. The junior engineer kept a copy of the table next to the freeze file so later pairing sessions would not reopen the same four dead ends.

Artifact: freeze file, decoder, and a failing contract test

The artifact is a tiny Python package that any later model must import. The freeze file is the source of truth. The decoder refuses unknown keys. The test should fail before a generated client is allowed to exist in src/.

{
  "tool": "search_docs",
  "http": {
    "method": "POST",
    "path": "/v1/tools/search_docs",
    "content_type": "application/json"
  },
  "arguments": {
    "required": ["q"],
    "optional": ["limit"],
    "forbidden": ["index", "api_key", "retry_after"]
  },
  "limits": {
    "q_max_chars": 200,
    "limit_max": 50
  },
  "success": {
    "type": "object",
    "required": ["hits"],
    "properties": {
      "hits": { "type": "array" }
    },
    "additionalProperties": false
  },
  "error": {
    "type": "object",
    "required": ["error"],
    "properties": {
      "error": {
        "type": "object",
        "required": ["code", "message"],
        "properties": {
          "code": { "type": "string" },
          "message": { "type": "string" }
        },
        "additionalProperties": false
      }
    },
    "additionalProperties": false
  }
}

# tool_contract.py
from __future__ import annotations

import json
from pathlib import Path
from typing import Any, Mapping

FREEZE_PATH = Path("contracts/search_tool.json")


def load_freeze() -> dict[str, Any]:
    return json.loads(FREEZE_PATH.read_text(encoding="utf-8"))


def assert_no_unknown(obj: Mapping[str, Any], allowed: set[str], label: str) -> None:
    extra = set(obj) - allowed
    if extra:
        raise ValueError(f"{label} has unknown keys: {sorted(extra)}")


def validate_call(payload: Mapping[str, Any], freeze: Mapping[str, Any] | None = None) -> dict[str, Any]:
    freeze = freeze or load_freeze()
    args = freeze["arguments"]
    required = set(args["required"])
    optional = set(args["optional"])
    forbidden = set(args["forbidden"])
    allowed = required | optional

    assert_no_unknown(payload, allowed, "call")
    missing = required - set(payload)
    if missing:
        raise ValueError(f"missing required arguments: {sorted(missing)}")
    blocked = forbidden.intersection(payload)
    if blocked:
        raise ValueError(f"forbidden arguments present: {sorted(blocked)}")

    query = payload["q"]
    if not isinstance(query, str) or not query.strip():
        raise ValueError("q must be a non-empty string")
    if len(query) > freeze["limits"]["q_max_chars"]:
        raise ValueError("q exceeds q_max_chars")

    if "limit" in payload:
        limit = payload["limit"]
        if not isinstance(limit, int) or isinstance(limit, bool):
            raise ValueError("limit must be an int")
        if limit < 1 or limit > freeze["limits"]["limit_max"]:
            raise ValueError("limit outside allowed range")

    return {
        "tool": freeze["tool"],
        "method": freeze["http"]["method"],
        "path": freeze["http"]["path"],
        "json": dict(payload),
    }


def validate_error_body(body: Mapping[str, Any], freeze: Mapping[str, Any] | None = None) -> None:
    freeze = freeze or load_freeze()
    err_schema = freeze["error"]["properties"]["error"]
    if set(body) != {"error"}:
        raise ValueError("error envelope must contain only error")
    inner = body["error"]
    if not isinstance(inner, Mapping):
        raise ValueError("error must be an object")
    assert_no_unknown(inner, set(err_schema["properties"]), "error")
    for key in err_schema["required"]:
        if key not in inner or not isinstance(inner[key], str) or not inner[key]:
            raise ValueError(f"error.{key} must be a non-empty string")

# tests/test_tool_contract.py
import pytest
from tool_contract import load_freeze, validate_call, validate_error_body


def test_freeze_names_exactly_one_tool():
    freeze = load_freeze()
    assert freeze["tool"] == "search_docs"
    assert freeze["http"]["method"] == "POST"
    assert freeze["http"]["path"] == "/v1/tools/search_docs"


def test_call_accepts_q_without_limit():
    built = validate_call({"q": "pairing schema freeze"})
    assert built["json"] == {"q": "pairing schema freeze"}
    assert built["method"] == "POST"


def test_call_rejects_invented_tool_fields():
    with pytest.raises(ValueError, match="unknown keys"):
        validate_call({"q": "x", "index": "secret-corpus"})


def test_call_rejects_required_limit():
    freeze = load_freeze()
    assert "limit" not in freeze["arguments"]["required"]
    validate_call({"q": "x"})  # must remain legal


def test_error_rejects_retry_after():
    with pytest.raises(ValueError, match="unknown keys"):
        validate_error_body({"error": {"code": "busy", "message": "wait", "retry_after": 2}})

Example commands for the local loop, labeled as a proposed workflow rather than a measured benchmark:

python -m pip install pytest
python -m pytest tests/test_tool_contract.py -q

The tests above are the pairing's gate. A generated client that builds a GET URL, requires limit, or decodes retry_after should lose in pytest before it loses in staging. The freeze file is short enough to paste into a prompt, which is the point of keeping one tool.

Numbered loop the pairing ran after the freeze existed

1. Commit contracts/search_tool.json and the two Python files with no client module under src/.
2. Run python -m pytest tests/test_tool_contract.py -q and confirm the suite is green on the freeze alone.
3. Add a failing adapter test that imports a not-yet-written src/search_client.py and checks it uses validate_call.
4. Only then ask a model to write src/search_client.py against the freeze file, with the freeze pasted into the prompt.
5. Re-run the suite and delete the adapter if it introduces a second tool, a second verb, or a second error key.
6. Keep the pull request description as a copy of the decision table, not as a chat summary of how clever the draft looked.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. After the freeze file landed, the pairing used MonkeyCode's free model access and free server option to draft the adapter that had to import validate_call instead of inventing a payload. The product was treated as a constrained typist sitting behind the pytest gate, not as an owner of the HTTP contract. The humans still owned the kept decision, the unknown-key decoder, and the merge button.

A useful prompt shape for that step is boring and local. It names the freeze path, names the forbidden tools, and forbids new JSON keys. Example prompt text, not executed in this article: write src/search_client.py that imports validate_call, posts only the returned method and path, and contains no other function that sends HTTP.

Limitations, and who should not use this freeze

The freeze does not prove the gateway is healthy, fast, or correctly authorized. It only proves the client cannot invent a second contract while remaining green. A free model can still hallucinate if the freeze file is missing from the prompt, and a free server option is not a replica of production latency, headers, or identity. Teams still need a real staging route, a real credential store, and a human who can say the tool name out loud.

This approach is a poor fit when nobody on the team owns the live schema, because the freeze then becomes another invented document. It is also a poor fit for multi-tool agents that must discover routes at runtime, or for write endpoints where a wrong POST is destructive even with a strict decoder. Do not paste secrets into the prompt that carries the freeze file. Do not let the adapter retry, page, or mutate indexes unless those verbs are written in the freeze and in an on-call document.

Generated comments do not repair an invented tool. A fluent client with extra helpers is still a contract break, even when the helpers look like clean code. The pairing kept the schema small so later sessions could argue about search quality instead of arguing about which function names a model dreamed up.

Readers who want the same freeze-then-generate loop on a throwaway branch can try MonkeyCode's free model access and free server option after the contract test already fails for the right reason.