DEV Community

Emery Chen
Emery Chen

Posted on

Spend the Free Allowance on Hostile Fixtures

Spend the free allowance on hostile fixtures only. Do not spend it on flattering model summaries. A summary that praises your diff is cheap comfort.

A fixture that breaks your spec is the actual work. That is the position, not a soft suggestion. If your free-tier log is mostly recap text, you wasted the pool.

Say the rule before the first call

You get a free model path and a free server. You also get a strong urge to request a nice summary. Refuse that urge before you write the first prompt.

Ask for a contradiction inside the spec instead. Ask for a missing field or a broken boundary. Then store the failure beside the claim you named.

A kind paragraph does not stress any contract. A hostile fixture does stress the contract directly. You should point at the broken case during review.

Why summaries feel like progress

Summaries are fast, and they read like finished judgment. You paste them into the pull request body. Reviewers see fluent prose and relax their questions.

The spec never met a single counterexample there. Free access makes this failure cheaper to repeat often. There is no invoice to embarrass the tenth recap.

You can generate twenty summaries and learn nothing new. That pattern is decoration, not real technical exploration. Hostile fixtures burn tokens because each call falsifies one claim.

If the claim survives, you learned one small thing. If the claim fails, you learned something you can file. Either result beats another flattering recap of the diff.

What hostile means here

Hostile does not mean illegal, abusive, or aimed outward. It means unfriendly to your own written specification. You are attacking the contract, not a person or company.

Use three fixture families and keep each family boring. Boring fixtures are easier to rerun after a model move. Clever prompts are harder to defend in review later.

Three families, nothing cleverer

  • Contradiction means two rules in the spec cannot both hold.
  • Omission means a required field, state, or error path is missing.
  • Boundary means an empty input, duplicate event, or hard limit appears.

Do not ask the model to invent crimes or exploit steps. Stay inside your repository and your documented rules only. If a fixture needs a real secret, stop and use a fake.

A harness you can copy

The script below is an unexecuted local example only. It does not call a model or open a socket. It checks that your log spent calls on hostile classes.

You run it locally before anyone cites the session. Adapt the filename if your repo uses another path. Do not treat a green exit as proof of product truth.

#!/usr/bin/env python3
"""Reject a free-tier log that is mostly summaries. Unexecuted example."""
import json
import sys
from collections import Counter
from pathlib import Path

ALLOWED = {"contradiction", "omission", "boundary"}
BANNED = {"summary", "praise", "rewrite"}

def load(path: Path):
    rows = []
    for line in path.read_text(encoding="utf-8").splitlines():
        if line.strip():
            rows.append(json.loads(line))
    return rows

def main() -> int:
    if len(sys.argv) != 2:
        print("usage: hostile_gate.py calls.jsonl")
        return 2
    rows = load(Path(sys.argv[1]))
    if len(rows) < 3:
        print("FAIL: need at least three hostile attempts")
        return 1
    kinds = Counter()
    for i, row in enumerate(rows, start=1):
        kind = str(row.get("fixture", "")).strip().lower()
        claim = str(row.get("claim_id", "")).strip()
        if not claim:
            print(f"FAIL: row {i} has no claim_id")
            return 1
        if kind in BANNED:
            print(f"FAIL: row {i} is a {kind}, not a fixture")
            return 1
        if kind not in ALLOWED:
            print(f"FAIL: row {i} fixture {kind!r} is not allowed")
            return 1
        kinds[kind] += 1
    missing = ALLOWED - set(kinds)
    if missing:
        print("FAIL: missing families: " + ", ".join(sorted(missing)))
        return 1
    print("OK " + " ".join(f"{k}={kinds[k]}" for k in sorted(kinds)))
    return 0

if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode
python3 hostile_gate.py calls.jsonl
echo "exit=$?"
Enter fullscreen mode Exit fullscreen mode

A synthetic log you can run

Create a synthetic log before you point the gate at real calls. These three lines are fake fixtures, not live model output. They exist so you can see a passing exit on your machine.

mkdir -p spec-lab && cd spec-lab
cat > calls.jsonl <<'EOF'
{"claim_id":"retry-once","fixture":"contradiction","input":"spec says at-least-once and exactly-once","survived":false,"note":"both sentences are present"}
{"claim_id":"retry-once","fixture":"omission","input":"no rule for a null receipt","survived":false,"note":"null path is unspecified"}
{"claim_id":"retry-once","fixture":"boundary","input":"duplicate delivery","survived":false,"note":"handler applied the side effect twice"}
EOF
python3 hostile_gate.py calls.jsonl
Enter fullscreen mode Exit fullscreen mode

What a passing log must contain

Three logged calls are a minimum, not a quality score. You still need one contradiction, one omission, and one boundary. A log of fifty summaries fails this gate immediately.

That immediate failure is the feature you wanted. It stops you from mistaking volume for coverage. Add more fixtures after the three families already exist.

The line format

Keep each attempt on one single JSON line. Name the claim you tried to break in that call. Name the fixture family and whether the spec survived.

{"claim_id":"retry-once","fixture":"boundary","input":"duplicate delivery","survived":false,"note":"handler applied the side effect twice"}
Enter fullscreen mode Exit fullscreen mode
{"claim_id":"retry-once","fixture":"contradiction","input":"spec says at-least-once and exactly-once","survived":false,"note":"both sentences are present"}
Enter fullscreen mode Exit fullscreen mode
{"claim_id":"retry-once","fixture":"omission","input":"no rule for a null receipt","survived":false,"note":"null path is unspecified"}
Enter fullscreen mode Exit fullscreen mode

Quote the spec line in the note when you need a pointer. Do not store customer payloads or live credentials there. Do not store prompts copied from other people's private chats.

Check the file before you cite it

Check that every line parses before you trust the mix. Then run the gate and keep the printed exit code. Neither command proves your product behavior is correct.

They prove you spent the session trying to break a claim. That is a lower bar than truth, and it is still useful. A session without that bar is just generated prose.

python3 -c 'import json,sys; [json.loads(line) for line in sys.stdin if line.strip()]' < calls.jsonl
python3 hostile_gate.py calls.jsonl
echo "exit=$?"
Enter fullscreen mode Exit fullscreen mode

Decision table for the allowance

Use this table before you send another free call. If the row says stop, you stop without bargaining. Do not bargain with a fluent model answer.

You are about to ask for Spend a free call? Why
A short summary of the diff No Decoration, not a fixture
Praise for the approach No Flattery hides missing cases
A contradiction inside the spec Yes One family you must cover
A missing error path Yes Omission is a real bug class
A boundary input against one claim Yes This is the useful burn
A rewrite of the whole module No That is a new project, not a test
Exploit steps against a live company No Out of scope and harmful
A score that ranks models No This harness is not a benchmark

Read the yes rows as your actual token budget. Your free calls should cluster on those rows. If they do not, the allowance taught you nothing.

How a free server fits the habit

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is described here as an open-source coding project. The brief states free model access and a free server option.

The same brief states a free allowance of ten million tokens. Treat that figure as a brief claim, not a measurement. Nobody measured the live quota, model list, or hardware for this piece.

Check the current terms before you depend on any number. Point the free server at this harness, not at release prose. Hostile fixtures consume calls quickly, so a free pool helps.

Keep production secrets off that server even when the price is zero. A zero price does not change your data handling duties. Synthetic inputs belong in the log, not customer records.

Do not invent a model winner from one free session. Model names move, and published limits move with them. Your calls file is the artifact you can still defend.

If you try this, read MonkeyCode's current free-server terms first. Then run the gate against a fresh hostile log. Use the allowance to break one claim already in your repo.

A workflow for one claim

Pick one claim and run the loop below without shortcuts. The order matters because the label comes before the call. If you label the run after you like the answer, you will cheat.

Seven steps, in order

  1. Pick one sentence from your spec and assign a claim id.
  2. Write a contradiction prompt that quotes only that sentence.
  3. Write an omission prompt for a missing field or error path.
  4. Write a boundary prompt using an empty, duplicate, or maximum input.
  5. Send those three prompts to the free server and save every reply.
  6. Mark survived yourself, and do not let the model mark it.
  7. Run the gate and commit the log beside the spec change.

Step six matters more than the wording of the model answer. The model can suggest a possible break in the spec. You decide whether that spec sentence actually failed.

If you skip that human decision, you outsourced judgment. A scratch host should not own your release standard. Keep the decision in the note field, in your own words.

Limits of this habit

This gate counts fixture families, not real-world truth. A silly contradiction can still pass the script cleanly. You must read the note before you trust the OK line.

Three calls can easily miss the dangerous case. The minimum exists to stop summary spam, not to certify coverage. Add a fixture when a bug escapes, and keep the old ones.

Free tiers change without asking for your permission first. A workflow that needs today's pool next month will stall. Store the prompts in git so any later host can rerun them.

The script trusts the fixture label you typed yourself. You can lie in the log and still receive OK. That lie fools a script, not a reviewer who reads the input.

What this is not

This piece reports no latency, accuracy, or savings figure. None of those quantities were measured for this workflow. If a quote adds a number, that number was invented later.

Who should not use it

Skip this habit if your change has no written claim. Hostile fixtures need a sentence you can actually falsify. Pure visual polish often has no such sentence yet.

Skip this if the diff holds secrets or personal data. Do not upload that material to a free server to save money. Rewrite the fixture with synthetic values before any call.

Skip this if counsel has not approved third-party model hosts. A free server is still an external processor until policy allows it. A zero invoice is not a legal argument for uploading code.

Skip this if you wanted a public model ranking. The decision table already forbids that use of the gate. Measure on a set you control, and publish the method separately.

Hold the line in review

Ask one question when a teammate pastes a model summary. Which claim did this call try to break today? If they cannot name the claim id, the call was entertainment.

Ask to see all three families inside the saved log. If a family is missing, send them back to the free bench. Do not block the merge on the model's stated confidence.

Block the merge on a missing fixture you already required. Free tokens are a research stipend for your own spec. Spend them on contradictions, omissions, and hard boundaries.

Leave the flattering summary unsent, unquoted, and out of the review. A free run that only comforts you did not earn a citation. Write the broken case down, or admit you did not look.

Top comments (0)