Learn Answer Freeze by Building a Tiny Response Receipt

Last Thursday I almost pasted a blank paragraph into a portfolio README and called it a result. The lab clock was past midnight, the project note was still open, and the answer box on the free server had returned three spaces. Spaces. I refreshed, got a fluent sentence about a leaderboard the note never mentioned, and still wanted to ship it because the demo slot was in the morning. Have you ever trusted a text box just because it used to work?

The question for this tiny case is narrow. If I freeze one model answer next to the context I actually supplied, can a short Python receipt refuse a blank or ungrounded demo before it reaches a README? Not whether the model is smart. Not whether the host is fast. Just this: would I show that card to another student?

Background

I have been turning course notes into small checkable artifacts instead of screenshots of chat windows. A screenshot smiles, then lies a week later when the same prompt comes back empty. A receipt is closer to a bus ticket. It does not prove the city is beautiful. It proves you boarded with a specific note in your pocket.

The project note I cared about was dull, which is why it was easy to skip. It said the demo must show a text receipt, not only a color badge, and that the sample file is CC0. That sentence is the whole world of this case. If the answer cannot touch it, I do not get to call the answer a result.

For the single live call I used MonkeyCode's free model access and the free server option as a study bench, the way I would use a library computer for one run. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am not claiming a quota, a model name, hardware, a duration, or that the free option stays available. I saved the text, closed the tab, and did the rest offline. The network is how you collect a card. It is not the demo.

Goal

I wanted one command that reads a JSON card and exits non-zero when the answer is missing, blank, or shares no meaningful tokens with the context. The failure had to show up in the terminal. A green README next to a red process is an honest pair. If the receipt passes, I still have to read the sentence myself. The script is a gate, not a grade.

I also wanted the intermediate state to be boring enough to commit. Prompt, context, answer, source label. No secret. No screenshot. If a classmate clones the folder on a train with no signal, the check should still run.

What I built

Prerequisites are plain on purpose. Python 3.11 or newer, a terminal, and no third-party packages. I checked the lab image with python3 --version and saw 3.11.6. I did not retest 3.9, so I will not pretend that older hint syntax is part of the result.

Save the checker as receipt.py.

#!/usr/bin/env python3
"""Freeze one model answer and refuse a blank or ungrounded demo card."""

import json
import re
import sys
from pathlib import Path

REQUIRED = ("prompt", "context", "answer", "source")
STOP = {"the", "and", "for", "with", "that", "this", "from", "into", "your", "not", "only"}

def tokens(text: str) -> set[str]:
    words = re.findall(r"[a-z0-9]+", text.lower())
    return {word for word in words if len(word) > 2 and word not in STOP}

def receipt(row: dict) -> dict:
    missing = [key for key in REQUIRED if key not in row]
    if missing:
        return {"ok": False, "reason": "missing_fields", "missing": missing}
    answer = str(row["answer"]).strip()
    if not answer:
        return {"ok": False, "reason": "empty_answer", "overlap": []}
    shared = tokens(answer) & tokens(str(row["context"]))
    reason = "grounded" if shared else "ungrounded"
    return {
        "ok": bool(shared),
        "reason": reason,
        "overlap": sorted(shared),
        "source": str(row["source"]),
    }

def main() -> int:
    if len(sys.argv) != 2:
        print("usage: python3 receipt.py card.json", file=sys.stderr)
        return 2
    row = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8"))
    report = receipt(row)
    print(json.dumps(report, indent=2))
    return 0 if report["ok"] else 1

if __name__ == "__main__":
    raise SystemExit(main())

The card I was willing to keep went in fixtures/grounded.json. The answer repeats a claim that is actually in the note. That is the easy case, and also the case a nervous student is tempted to skip because it looks obvious.

{
  "prompt": "What must the demo show besides a color badge?",
  "context": "The project note says the demo must show a text receipt, not only a color badge, and the sample file is CC0.",
  "answer": "The demo must show a text receipt, not only a color badge.",
  "source": "frozen-project-note"
}

Then I ran it.

python3 receipt.py fixtures/grounded.json
echo "exit=$?"

Exit code 0. The overlap list includes demo, show, text, receipt, color, and badge. not and only are in the stop list, so they do not pretend to be evidence. The source field is a label I typed. It is not a signature from the server, and it should not be read as one.

That printout is the intermediate state I want on a portfolio page. Not a vibe. A card. If you squint, it is the same habit as keeping a lab sketch instead of saying "it worked on my machine."

The input that should fail

Here is the card that almost entered the README. I saved it as fixtures/blank.json. The answer is three spaces. A chat window will sometimes render that as a polite pause. A receipt should not.

{
  "prompt": "Summarize the project note in one sentence.",
  "context": "The project note says the demo must show a text receipt, not only a color badge, and the sample file is CC0.",
  "answer": "   ",
  "source": "live-free-server"
}

python3 receipt.py fixtures/blank.json
echo "exit=$?"

What I get back is small enough to read aloud.

{
  "ok": false,
  "reason": "empty_answer",
  "overlap": []
}

The process exits 1. Silence is not a summary. I did not retry the call three times and average the feeling. A blank is already an answer to the question I asked the script.

I also kept a fluent liar, because a non-empty string is how demos usually fool me. fixtures/ungrounded.json uses the same context and this answer: The model scored 99 percent on a private leaderboard. Predict the exit before you run it. There is no shared content word with the project note, so reason is ungrounded, overlap is [], and the exit code is 1 again. If your first draft only checks that the string is non-empty, this one sails through. That was my first draft. It felt kind. It was wrong.

Results I will actually stand behind

Three cards, three decisions. The grounded card passed. The blank card failed closed. The fluent card failed for lack of shared tokens. I did not time the free server, and I am not going to invent a latency number so the post looks measured. The result that mattered in the morning was social. I could point at a red exit code instead of saying the model usually answers.

What should you understand after you run it? A demo answer is a record with fields, not a mood. Free model access is useful for the one live call that creates the card. It is a poor dependency for the moment someone else opens your README on a train. The receipt checks overlap with the note you supplied. It does not check truth, licenses, or whether a paraphrase is fair.

I also learned where my own impatience lives. The leaderboard sentence sounded like a result because it had a number in it. Numbers are costumes. 99 never appeared in the project note, and the script did not care that the sentence was grammatically fine. Have you shipped a costume like that in a README?

Where the idea breaks

Copy the context into the answer and the receipt applauds you. That is echo, not understanding. Paraphrase cleanly, share no content words, and the receipt scolds a correct sentence. I hit that with "visitors need words, not a green dot, and the example data is public domain." My context said receipt, badge, and CC0. The paraphrase shared nothing after the stop list, so ok was false. Annoying, and also the point. A lexical gate is a smoke alarm, not a fire inspector.

Who should skip this approach? If you need citation checking, grading, or anything a recruiter might treat as an evaluation harness, this is the wrong tool. If you need the live server to stay up during the demo, you need a different plan, because I am not promising uptime and I did not measure it. If your context is shorter than a sentence, token overlap becomes a coin flip. Do not bolt this onto a production agent and call it safety.

Common mistakes I already made: treating whitespace as an answer, printing the JSON and ignoring the exit code, and letting the count as evidence before the stop list existed. Another one is storing a key in the fixture. Do not. The card holds the prompt, the context, the answer text, and a source label. Secrets stay out of the portfolio repo. A fifth mistake is retrying a blank until a pretty sentence appears, then freezing the pretty sentence without the note. That is how a demo becomes fiction.

A small extension

Change the pass rule so ok requires at least two overlap tokens, then rerun the grounded card. Does text plus receipt still feel like grounding, or did you just reward a copy? Add a fourth fixture whose answer is a true paraphrase with zero shared words, and decide whether you want a fail or a human note in source.

If you try this before your next portfolio review, tell me which fixture you expected to fail and which one actually embarrassed you. I would rather see a minimal counterexample than a clap.