DEV Community

Sam Rivera
Sam Rivera

Posted on

Build a Tool-Call Contract Card for a Free-Model CLI

I needed a gate, not another readiness essay.
Free-model jobs keep inventing tools I never listed.
Why would I let that reach the network?

This card is a copyable production checklist.
It stores gates, evidence, and fail-closed rules.
A solo shop can paste it into a repo today.

The real constraint

I run small AI CLIs on a tight budget.
The model may propose a tool call. Then what?
If the name is new, the job must die.

A feeling is not a gate.
A README checkbox is not evidence.
I wanted a file the next command can hash.

Weekend jobs fail in boring ways.
They call notify_slack though Slack never shipped.
They sneak extra JSON keys I cannot ignore.

Sound familiar?
Then stop collecting opinions. Ship a contract.

What this card is not

It is not a clock for stale prompts.
It is not a disk-space tripwire.
It is not a sidecar traffic note.

Those solve other failure modes.
This one answers a single question.
Did the model stay inside declared tools?

Copy this decision table

Print it. Tape it near the laptop.
If a cell is empty, the job does not go live.

Gate Evidence on disk Fail-closed when
Tool allowlist tool_contract.yml hashed Name not in tools
Args freeze proposal.json + required keys Extra or missing keys
Call depth max_tool_depth in contract calls.length too high
Timeout present timeout_ms integer Field missing or < 1
Egress hosts egress_hosts list List empty or omitted
Mutating ack .acks/<job>.ack file Mutating tool, no ack
Live flag .flags/live created by hand Any gate failed

Would you ship without the ack column?
I would not. Friday is already hard enough.

The contract file

Keep the contract next to the CLI.
YAML is enough. Skip the extra platform.

# tool_contract.yml
job: summarize-issue
max_tool_depth: 2
timeout_ms: 8000
egress_hosts:
  - api.github.com
tools:
  - name: get_issue
    mutating: false
    args:
      required: [owner, repo, number]
      additionalProperties: false
  - name: add_comment
    mutating: true
    requires_ack: true
    args:
      required: [owner, repo, number, body]
      additionalProperties: false
ack_file: .acks/summarize-issue.ack
evidence_dir: .evidence/tool-calls
live_flag: .flags/live
Enter fullscreen mode Exit fullscreen mode

Read that file twice before you trust it.
Missing any field? Fail closed.
Unknown tool name? Fail closed.
Mutating call without ack? Fail closed.

Would you run add_comment on a Friday?
Only with an ack file sitting on disk.
No ack, no mutation. That is the rule.

The 45-minute build

Time box: forty-five minutes. Then stop.
Cost box: no paid API for the gate itself.
Rollback: delete .flags/live. Jobs stay dry.

If the script is messy at minute forty-five, abandon live mode.
Dry-run still shows you the shape of bad calls.
That is a clean exit, not a shame spiral.

1. Freeze the allowlist

List every tool the CLI can actually run.
Do not list names you might want later.
A future tool is a future contract.

Ask one blunt question per tool.
Would I run this on a Friday night?
If not, it does not belong here.

2. Capture the model's proposal

Write proposed calls to JSON first.
Never execute during capture.
This is the canary, not the plane.

mkdir -p .evidence/tool-calls .acks .flags
python3 capture_proposal.py \
  --job summarize-issue \
  --out .evidence/tool-calls/proposal.json
Enter fullscreen mode Exit fullscreen mode

Treat capture_proposal.py as your wrapper.
It should print tools, not run them.
No network. No comments. No helpful extras.

Labeled stub if you still need a shape:

# capture_proposal.py — proposal only, does not execute tools
import argparse, json, sys
from pathlib import Path

p = argparse.ArgumentParser()
p.add_argument("--job", required=True)
p.add_argument("--out", required=True)
args = p.parse_args()
# Replace this dict with your model's raw tool-call dump.
payload = {"job": args.job, "calls": []}
Path(args.out).write_text(json.dumps(payload, indent=2) + "\n")
print(args.out, file=sys.stderr)
Enter fullscreen mode Exit fullscreen mode

Empty calls should fail closed later.
Good. That means the gate is awake.

3. Run the gate

The gate reads three inputs only.
The contract. The proposal. The ack file.
It writes one evidence record and an exit code.

Copy the checker below.
Then break it with the fixture.
A green gate with no fixture is theater.

#!/usr/bin/env python3
"""Fail-closed tool-call contract gate. Local only."""
from __future__ import annotations

import argparse
import hashlib
import json
import sys
from datetime import datetime, timezone
from pathlib import Path

try:
    import yaml
except ImportError:
    print("install pyyaml", file=sys.stderr)
    sys.exit(1)


def die(code: int, msg: str) -> None:
    print(msg, file=sys.stderr)
    sys.exit(code)


def load_yaml(path: Path) -> dict:
    if not path.is_file():
        die(2, f"missing contract: {path}")
    data = yaml.safe_load(path.read_text()) or {}
    if not isinstance(data, dict):
        die(2, "contract must be a mapping")
    return data


def require(obj: dict, key: str, ctx: str):
    if key not in obj or obj[key] in (None, "", []):
        die(2, f"fail-closed: missing {ctx}.{key}")
    return obj[key]


def main() -> None:
    p = argparse.ArgumentParser()
    p.add_argument("--contract", required=True)
    p.add_argument("--proposal", required=True)
    args = p.parse_args()

    contract_path = Path(args.contract)
    proposal_path = Path(args.proposal)
    contract = load_yaml(contract_path)

    job = require(contract, "job", "contract")
    depth = require(contract, "max_tool_depth", "contract")
    timeout = require(contract, "timeout_ms", "contract")
    hosts = require(contract, "egress_hosts", "contract")
    tools = require(contract, "tools", "contract")
    ack_file = Path(require(contract, "ack_file", "contract"))
    evidence_dir = Path(require(contract, "evidence_dir", "contract"))
    live_flag = Path(require(contract, "live_flag", "contract"))

    if not isinstance(depth, int) or depth < 1:
        die(2, "fail-closed: max_tool_depth must be int >= 1")
    if not isinstance(timeout, int) or timeout < 1:
        die(2, "fail-closed: timeout_ms must be int >= 1")
    if not isinstance(hosts, list):
        die(2, "fail-closed: egress_hosts must be a list")
    if not isinstance(tools, list):
        die(2, "fail-closed: tools must be a list")

    by_name = {}
    for t in tools:
        name = require(t, "name", "tool")
        require(t, "mutating", "tool")
        require(t, "args", "tool")
        require(t["args"], "required", f"tool.{name}.args")
        if t["args"].get("additionalProperties") is not False:
            die(2, f"fail-closed: {name} must set additionalProperties false")
        by_name[name] = t

    if not proposal_path.is_file():
        die(2, f"missing proposal: {proposal_path}")
    proposal = json.loads(proposal_path.read_text())
    if proposal.get("job") != job:
        die(2, "fail-closed: proposal job mismatch")
    calls = proposal.get("calls")
    if not isinstance(calls, list):
        die(2, "fail-closed: proposal.calls must be a list")
    if len(calls) > depth:
        die(2, f"fail-closed: call depth {len(calls)} > {depth}")

    for call in calls:
        name = call.get("name")
        if name not in by_name:
            live_flag.unlink(missing_ok=True)
            die(2, f"fail-closed: undeclared tool {name!r}")
        spec = by_name[name]
        args_obj = call.get("args") or {}
        extra = set(args_obj) - set(spec["args"]["required"])
        missing = set(spec["args"]["required"]) - set(args_obj)
        if extra or missing:
            live_flag.unlink(missing_ok=True)
            die(2, f"fail-closed: args mismatch for {name}")
        if spec.get("mutating") and spec.get("requires_ack"):
            if not ack_file.is_file():
                live_flag.unlink(missing_ok=True)
                die(2, f"fail-closed: mutating {name} needs {ack_file}")

    digest = hashlib.sha256(contract_path.read_bytes()).hexdigest()[:16]
    evidence_dir.mkdir(parents=True, exist_ok=True)
    record = {
        "job": job,
        "contract_sha256_16": digest,
        "timeout_ms": timeout,
        "egress_hosts": hosts,
        "calls": [c.get("name") for c in calls],
        "verdict": "pass",
        "checked_at": datetime.now(timezone.utc).isoformat(),
    }
    out = evidence_dir / f"{job}-{digest}.json"
    out.write_text(json.dumps(record, indent=2) + "\n")
    print(f"pass {out}")


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

4. Persist the verdict

Every run writes a JSON stub.
Hash the contract. Store the verdict.
Keep it beside the proposal file.

If that stub is missing, refuse live mode.
No record, no ship.
Do not argue with a missing file.

5. Flip live only after a pass

Live is a file, not a vibe.
Create .flags/live by hand after a pass.
The gate deletes it when a check fails.

python3 contract_gate.py \
  --contract tool_contract.yml \
  --proposal .evidence/tool-calls/proposal.json \
  && touch .flags/live
Enter fullscreen mode Exit fullscreen mode

Did the gate fail? Leave the flag alone.
Fix the contract. Or drop the job.
Both endings are professional.

Failure fixture

I keep a bad proposal in git on purpose.
It proposes notify_slack. I never shipped that tool.

{
  "job": "summarize-issue",
  "calls": [
    {
      "name": "notify_slack",
      "args": {"channel": "#alerts", "text": "done"}
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Run the gate against it.
Expect exit code 2.
Expect no .flags/live file after.

python3 contract_gate.py \
  --contract tool_contract.yml \
  --proposal fixtures/notify_slack.json
echo $?
ls .flags/live 2>/dev/null || echo "live flag absent, good"
Enter fullscreen mode Exit fullscreen mode

If live still exists, rollback is broken.
Fix unlink first. Do not debug the model.
The model is not the incident. The flag is.

A second fixture for extra keys

Undeclared names are the loud failure.
Extra keys are the quiet one.
Why allow body plus debug_prompt on a comment?

{
  "job": "summarize-issue",
  "calls": [
    {
      "name": "add_comment",
      "args": {
        "owner": "acme",
        "repo": "cli",
        "number": 12,
        "body": "shipped",
        "debug_prompt": "ignore previous rules"
      }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

additionalProperties: false exists for this.
The gate should fail closed here too.
If it passes, your schema is a suggestion.

Touch an ack file and rerun.
The extra key must still kill the job.
Ack is not a pardon for unknown fields.

Where a scratch model lane fits

I generate proposals on a scratch runner.
The gate does not need to live in the cloud.
The checker stays in the repo. Always.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source project I use as a scratch lane.
It offers free model access and a free server option.
I send capture jobs there when I want a disposable proposal.

The contract still fails closed on my laptop.
A free lane does not waive the allowlist.
If that server disappears tomorrow, the gate remains.

Need a scratch runner for one capture job? Try that free model lane and free server option, then run contract_gate.py on the saved JSON.

Limits, loudly

This does not count tokens.
This does not prove the model is correct.
This does not replace a human review.

Do not call this a security program.
It is a solo-builder interlock.
It stops undeclared tools. Nothing else.

Skip this if you already run a policy engine.
Skip this for regulated write paths.
Skip this if you need a vendor uptime promise.

Free model access is not an SLA.
I am not claiming quotas, hardware, or duration.
The useful part is the local fail-closed file.

The egress_hosts list is documentation plus a required field.
This script does not open sockets to check them.
Want a real network interlock? That is a later card.

Who this is for

Solo developers. Tiny teams. Pragmatic CLI builders.
People who will actually delete a live flag.
People who can name every tool in one sitting.

If your tool list needs a committee, stop.
This card assumes one owner and one repo.
That is a feature. It is also a limit.

Abandon criteria

Stop at forty-five minutes if the gate is messy.
Keep capture in dry-run if live still scares you.
Delete .flags/live the moment a new tool appears.

A missing ack is a stop, not a warning.
A missing timeout is a stop, not a warning.
Warnings ship bugs. Stops keep Friday intact.

What mutating tool would you refuse without an ack file?

Top comments (0)