Separate the Spec from the Scratch Pad

You open the take-home on a Thursday night. The README is smooth. A demo script prints three green lines, and a note at the bottom says a model "handled the edge cases."

You close the pasted chat, stop the process, and start it again. The third request returns 200. The spec you wrote on Monday said 429. The polish was never the contract. It was a scratch pad that remembered the prompt better than the program did.

That is the failure mode showing up in AI-assisted submissions right now. Fluency got cheap. A restart did not. If you hire people who will ship code a model helped draft, the take-home has to separate those two facts before a live interview is spent on a demo that only works inside one session.

Buy a contract, not a tour

You are not buying a tour of a free model. You are buying a small program you can re-check without the author's laptop, the author's transcript, or the author's confidence. A submission that only runs while the chat is open is a demo. A submission whose tests still mean the same thing after a new process is a candidate.

You are not asking them to rebuild a machine from a blank image. You are asking whether the number 2 still means 2 when the chat window is closed.

Write the prompt so the spec is a file in the repo. Tell them a model may draft the implementation. Tell them the draft is untrusted until a test they also commit fails for the right reason and then passes for the right reason. If they cannot explain a red assertion without scrolling a transcript, they do not own the work yet.

Order matters more than tone. Ask for the tests before the implementation, and ask the README to name one case the tests do not cover. People who only steered a model often cannot name a hole. People who wrote the spec usually can, and the hole they name tells you how they think once the happy path is already green.

Here is a prompt you can send. Treat it as a proposal you still need to edit for your stack, not as a record of a live loop.

Build a tiny retry gate.

POST /retry?key=... may succeed at most 2 times for a given key.
The third call returns 429 and must not increment the counter.
GET /retry?key=... returns the number already used.
State lives on disk. Stopping the process and starting it again
must not reset the count.
Do not commit secrets, tokens, or a model transcript.
You may draft with a model. Commit the tests first.
In the README, name one case the tests do not cover,
and narrate the red-green sequence in your own words.
Commit timestamps are not evidence. They can be rewritten.

The narration line is doing quiet work. Git history looks like proof until someone rebases. A short paragraph in the candidate's own words is harder to fake with a paste, and it gives you something concrete to ask about on the call.

A sample you can fail on purpose

The Python below is an unexecuted sample, not a benchmark and not a hiring key. It shows the shape of a pass so your rubric has a concrete object. Run it, break it, and only then decide whether it belongs in your packet.

# gate.py — sample solution, not measured on a hosted box
import json
from pathlib import Path

class RetryGate:
    def __init__(self, root: Path, limit: int = 2):
        self.root = root
        self.limit = limit
        self.root.mkdir(parents=True, exist_ok=True)

    def _path(self, key: str) -> Path:
        safe = "".join(ch for ch in key if ch.isalnum() or ch in "-_")[:64]
        if not safe:
            raise ValueError("empty key")
        return self.root / f"{safe}.json"

    def used(self, key: str) -> int:
        path = self._path(key)
        if not path.exists():
            return 0
        data = json.loads(path.read_text())
        return int(data["used"])

    def allow(self, key: str) -> bool:
        path = self._path(key)
        used = self.used(key)
        if used >= self.limit:
            return False
        # Teaching stand-in, not a database. Two writers can still race.
        path.write_text(json.dumps({"used": used + 1}))
        return True

A new object pointed at the same directory is the restart. You do not need a cluster to teach that idea. You need a file that outlives the process, and a test that constructs a second object instead of reusing the first.

# test_gate.py — unexecuted example
from gate import RetryGate

def test_third_call_sticks_after_new_object(tmp_path):
    first = RetryGate(tmp_path, limit=2)
    assert first.allow("job-9") is True
    assert first.allow("job-9") is True
    assert first.used("job-9") == 2
    restarted = RetryGate(tmp_path, limit=2)
    assert restarted.allow("job-9") is False
    assert restarted.used("job-9") == 2

def test_rejected_key_does_not_create_a_file(tmp_path):
    gate = RetryGate(tmp_path, limit=2)
    try:
        gate.allow("***")
    except ValueError:
        pass
    assert list(tmp_path.glob("*.json")) == []

The second test is the one hurried drafts skip. A model that only heard "store the count" will happily write a file from raw user input, or it will swallow a bad key and return false without saying why. You want the exception, and you want the directory empty afterward. That is a small contract, and it is enough for this exercise.

python -m pytest test_gate.py -q
# on the box, after the process is listening:
curl -s -o /dev/null -w "%{http_code}\n" -X POST "http://127.0.0.1:8080/retry?key=job-9"
curl -s "http://127.0.0.1:8080/retry?key=job-9"
# stop the process, start it again, POST until you expect 429

If the count returns to zero after the stop, the green demo was session memory. Fail that behavior even if the README is kind. If the third call stays at 429 and the stored count stays at 2, you have a pass on the only state claim this task makes.

A reviewer script keeps you from grading by vibe. This one is also unexecuted proposal code. Point it at the key file after your restart, and let the exit code be the note you keep.

# review_check.py — proposal, unexecuted
import json, sys
from pathlib import Path

def main() -> None:
    used = int(json.loads(Path(sys.argv[1]).read_text())["used"])
    if used != 2:
        raise SystemExit(f"expected used=2 after restart, got {used}")
    print("restart count held")

if __name__ == "__main__":
    main()

python review_check.py /var/lib/retry-gate/job-9.json

Do not treat a missing file as a clever zero. A missing file after two successful posts means the gate wrote somewhere you are not looking, which is a fail with a different name.

Use the free door for the draft, not for the spec

You can run this exercise on a laptop. The reason to involve a hosted box is narrower than a launch post implies. Candidates should not have to open a cloud account, pick a region, and wait on a billing ticket before they can show you a restart.

A free server option removes that toll booth. Free model access lets them draft without you pretending the draft is the submission.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode fits this workflow only as that pair of conveniences: free model access for the scratch pad, and a free server so the restart is not "trust my laptop." Repeat those two availability claims and stop. They are not a chip count, a model menu, a token cap, or a promise that the free tier will look the same next quarter. This article does not freeze any of those.

Before you send the prompt, open the current limits page and copy only the constraints you just read. If the page is vague, say that in the task. Inventing a machine size to sound precise is how take-homes become unfair.

Use the model the way you would use a whiteboard that talks. Sketch RetryGate if you want. Then throw the sketch at the test that already existed. If the test was written after the model was already green, you graded a memory.

The README narration is where that slip shows up. A candidate who says "I asserted 429 because the chat said the third call should fail" has described a scratch pad. A candidate who says "I asserted 429 because the prompt defined a budget of 2, and I checked the file after a new object" has described a spec.

If you want a single next step, read MonkeyCode's current notes on free model access and the free server, paste the limits you actually see into the prompt, and send the task to one person before you standardize it.

Score the restart, then stop

Read the test file before the README. Assertions that restate the prompt in code mean the candidate understood the contract. Assertions that only check for a dict key named ok mean they tested a shape the scratch pad likes. You can feel the difference quickly, which is the point of keeping the task small.

Then restart, once, on the box named in the prompt. A pass that exists only inside one process is a partial. You may still talk to a partial if the role is early-career and the writing is honest. Do not relabel it as a pass on state.

Honesty looks like a README that says "two processes can race on the same key, and I did not take a lock." Evasion looks like a transcript dropped under docs/ and a sentence that says the model verified the edges.

Keep your own review inside the prompt. A clever extra endpoint is not credit if /retry forgets its count. A plain function that survives the restart is a pass. You are hiring for ownership of a contract, not for taste.

The failures repeat, and they are boring, which is useful. One candidate commits the transcript and calls it design notes. Another keeps the count in a global, so the restart check fails immediately. Another builds the storage path with string concatenation aimed at a shell, so the key is no longer just data.

Another mocks allow() and asserts the mock was called, which proves the test runner, not the disk. Any one of those earns a short note. You do not owe a novel, and you should not soften the note because the prose around the bug was confident.

There is a subtler miss. The implementation writes the JSON through a temp file and os.replace, which is a fair durability improvement, but the tests never stop the process mid-write. If the README claims crash safety you did not ask for, ask them to show the test or withdraw the claim. Extra claims without extra checks are the same scratch-pad habit in a nicer coat.

Who should leave this on the shelf

Skip the task when the role is a pure algorithm screen. A restart file adds noise when you only wanted to watch someone reason about a graph. Skip it when policy forbids sending exercise text to a third-party model, including a free one.

Skip it when you need a named uptime target, a specific machine, or a frozen token budget. None of those are established here, and a free box that shifts under the candidate will punish them for a reason that is not theirs.

The filter also misses collaboration, deletion, and design depth. A person can pass test_third_call_sticks_after_new_object and still be a poor teammate. Use the hour to see whether they can keep a spec off the scratch pad. Then interview the human.

Free access is a door, not a measurement. If the door is closed on review day, reschedule. Do not lower the rubric because the box was slow, and do not raise it because the draft sounded sure. The spec you froze before the draft is the only sentence that should still be true after the reboot.