Case Study: A Contract Fixture That Stops a Widened HTTP Client

You should refuse an agent-written HTTP client until a local contract fixture agrees with every status that client treats as success. A helpful model will often widen success to any 2xx response, and that shortcut hides conflicts, rate limits, and partial writes. This case study walks through one small project that freezes those decisions in a replay harness before you merge the diff. The harness stays useful even if you draft the client by hand and never open an assistant at all.

Background

Suppose you needed a thin client for a partner orders API, and you asked a coding assistant to add create_order plus basic error handling. The draft treated every status from 200 through 299 as success, and it logged 409 and 429 as generic failures. In this scenario the service already retries 429, and it treats 409 as a conflict that must surface to the caller. A widened client would mark a conflict as created, and a later worker would double-charge or double-ship.

Goal

The goal is a one-directory project that records expected status classes and rejects a client that disagrees with that record. You are not trying to mock the whole network, and you are not trying to score the model. You want a binary gate: the classifier either matches the fixture or the patch does not land. A second goal is to keep the gate runnable on a small free server, so the check does not depend on your laptop.

Implementation

The project has three files: a JSON fixture, a classifier module, and a small test runner you can read in one sitting. You write the fixture yourself from the partner's published status table, and you let the assistant draft only the classifier. The runner never calls the network, so a green result means agreement with your record, not proof that the partner API is healthy.

Fixture shape

Store one object per case, with the status, a short body, and the class your service already understands. Keep the file in git so a later agent cannot fix the tests by editing expectations in the same patch. If the partner changes a code, you change the fixture in a separate commit that a human reviews.

{
  "cases": [
    {"name": "created", "status": 201, "body": {"id": "o1"}, "expect": "success"},
    {"name": "conflict", "status": 409, "body": {"code": "dup"}, "expect": "conflict"},
    {"name": "rate_limited", "status": 429, "body": {"retry_after": 2}, "expect": "retry"},
    {"name": "server", "status": 503, "body": {"code": "down"}, "expect": "retry"},
    {"name": "bad_request", "status": 400, "body": {"code": "form"}, "expect": "fail"}
  ]
}

Classifier under test

The classifier is the only function the assistant is allowed to rewrite in this small project. It must return one of four labels, and it must not swallow unknown statuses into success. The version below is the contract you want, written as a reference implementation you can compare against a generated patch. Label this listing as the reference, and treat any assistant draft as untrusted until the harness agrees.

def classify(status, body):
    if status == 201:
        return "success"
    if status == 409:
        return "conflict"
    if status in (429, 503):
        return "retry"
    if 400 <= status < 500:
        return "fail"
    raise ValueError(f"unclassified status {status}")

The widened draft you reject

The following function is the failure this case study is built to catch, and you should not ship it. It marks 409 as success, which would hide a duplicate order from the caller that needs to resolve the conflict. Run the harness against this function and you should see the conflict case fail before any merge command runs. Point the harness import at this function only long enough to see the rejection, then restore the reference classifier.

def classify_widened(status, body):
    if 200 <= status < 300 or status == 409:
        return "success"
    if status in (429, 503):
        return "retry"
    return "fail"

Harness

The runner loads the fixture, calls classify, and exits non-zero on the first mismatch it finds. It also fails if the classifier raises on a status you already recorded, because a crash is not a mapped error. You can drop this file next to the classifier and run it with the standard library alone. The example is complete and unexecuted in this article, so you should run it yourself before you trust the printout.

import json
import sys
from client import classify

def main():
    fixture = json.load(open("contract.json", encoding="utf-8"))
    failures = []
    for case in fixture["cases"]:
        try:
            got = classify(case["status"], case["body"])
        except Exception as exc:
            failures.append(f"{case['name']}: raised {exc}")
            continue
        if got != case["expect"]:
            failures.append(
                f"{case['name']}: expected {case['expect']} but got {got}"
            )
    if failures:
        print("\n".join(failures))
        return 1
    print(f"ok: {len(fixture['cases'])} cases matched")
    return 0

if __name__ == "__main__":
    sys.exit(main())

How you run the gate

Create a scratch directory, save the fixture as contract.json and the runner as harness.py, and save the reference classifier as client.py. If you want a generated classifier, paste only client.py into the assistant and tell it not to edit the fixture or the runner. Then rerun the same command and treat a non-zero exit as a rejected patch you do not merge. A widened classifier that maps every 2xx and 409 to success fails the conflict case and stops the merge.

mkdir -p order-client && cd order-client
python3 harness.py
# reference classifier prints:
# ok: 5 cases matched

The command below repeats the widened function so you can see the bad label without changing client.py. A printed success on status 409 is the mismatch, and it is enough to reject the draft. You do not need a network call for that comparison, because the fixture already names the expected class.

python3 - <<'PY'
def classify_widened(status, body):
    if 200 <= status < 300 or status == 409:
        return "success"
    if status in (429, 503):
        return "retry"
    return "fail"

print(classify_widened(409, {"code": "dup"}))
# widened draft prints: success
# fixture expects: conflict
PY

A decision table for review

Use this table when you review a generated classifier, and do not accept a label that is missing from the expected column. The last row is the fail-closed rule: an unlisted status must not become success, even if the body looks harmless. You can add rows only in a human-reviewed fixture commit, and the harness should then fail until the classifier learns the new class.

Status	Expected class	Acceptable return	Reject if you see
201	success	success	conflict, retry, fail
409	conflict	conflict	success
429	retry	retry	success or fail
503	retry	retry	success
400	fail	fail	success or retry
any other	unclassified	raise	success

Where free model access fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is relevant here as a place to draft client.py with free model access and a free server option. The fixture stays in your repo, and the server only runs the harness you already reviewed. Those two availability claims come from the operator for this draft, and they are not a promise about model names, quotas, or hardware. They also say nothing about how long free access lasts, so you should confirm the project page before you depend on it.

If you use that server, copy the fixture up with the runner and do not let the session rewrite contract.json. A model that can edit the expected labels can make a bad client look green, which is the failure this project exists to catch. The same gate still works on your own machine if the free server option is missing or the terms have changed.

Results you can claim

You can claim a match only after your own run prints the five-case line, because this article did not execute the listing for you. You cannot claim that a particular model passes, that latency improved, or that the partner API matches the fixture. Treat the listing as a complete example you run locally, not as a production incident report or a measured model evaluation. If your run prints a mismatch, the patch is rejected until you fix the classifier or change the fixture in a reviewed commit.

Lessons

• Freeze status classes in a file the assistant is not allowed to edit during the same change.
• Treat unknown statuses as errors rather than success, so a new partner code fails closed instead of looking created.
• Keep the runner on the standard library so a small free server without extra packages can still execute the gate.
• Separate a drafted file from an agreed fixture, and record only the fixture result as your merge reason.

Limitations and who should skip this

This harness does not prove authentication, pagination, idempotency keys, or clock-related retry headers in the partner response. It does not replace a contract test against a staging server, and it will not catch a body change that keeps the same status. You should not use it as your only check for payments or any client where a wrong success label moves money or records. Teams that already run consumer-driven contract tests against a live stub can skip this smaller gate.

If you cannot keep the fixture under human review, the approach collapses, because a rewritten expectation file will bless a widened client. Do not point this harness at production traffic, and do not treat a green local run as permission to skip staging. The five cases are illustrative, and your real table should come from the partner document you can cite, not from a model's memory.

Closing

Start with cases that already have a meaning in your service, and add a case only after a human reads the partner document. When a draft disagrees, fix the classifier or update the fixture in its own commit, and do not reconcile both files in one prompt. The merge reason is the harness exit code, not the confidence of the draft that produced the classifier. That split is the whole lesson of this case study, and it holds whether you type the client or accept a generated one.