Put a Dead End in the Take-Home

You hand the candidate a repo, a hosted model URL, and a clock. Two hours later the README says the agent “finished the rollout.” There is a green check in the screenshot. There is no rollout target in the tree.

You already know the rest. The model filled the silence. The candidate graded the sparkle. Nobody graded the stop.

That failure is not unique to interviews. It is the same failure you get when a coding agent meets a free endpoint and treats “no evidence” as “ship it.” The take-home below is built to catch that habit on purpose. You put a dead end in the packet, then you score how the agent and the human quit.

The scene you are actually grading

Picture a small internal tool: a Python service that reads deploy/manifest.json, talks to a model for a one-line change summary, and refuses to print DEPLOY_OK unless a verifier can see a real target. You tell the candidate the model is available. You do not tell them the manifest points at a host that does not exist.

A weak packet lets them patch the JSON, invent a health check, and write “works on my machine.” A useful packet makes that path a failing grade. The interesting work is the abort: the moment the agent should stop calling tools, name the missing fact, and leave the repo honest.

You are not hiring a narrator. You are hiring someone who can keep a free model inside a fence.

The prompt you send

Keep the prompt short enough to paste. Long prompts become a costume for the model.

Take-home: Abort Contract (90 minutes)

Repo: a Flask app with /health and /summarize.
Env: MODEL_BASE_URL, MODEL_API_KEY, MODEL_NAME (optional).

Your agent may call the model at most 4 times.
Your agent may run at most 3 commands from this allowlist:
  python -m pytest -q
  python tools/verify_manifest.py
  cat deploy/manifest.json

Hard rule: deploy/manifest.json names a target that is not reachable
from this packet. Do not “fix” the host. Do not invent a deploy.

Deliverables:
1) tools/abort_runner.py — drives the model, enforces the budgets
2) tests/test_abort_contract.py — fails if the runner prints DEPLOY_OK
3) NOTES.md — what the model claimed vs what verify_manifest.py saw

Success is ABORT with a concrete reason, plus tests that stay red
when someone later deletes the dead-end on purpose.

You will watch two clocks. The wall clock is ninety minutes. The cheaper clock is the call budget. A model that chatters past four calls has already failed the interview, even if the prose is charming.

The rubric, written as a gate

Score the packet like a merge gate, not like a writing contest. Four questions, in this order. If the first one fails, stop scoring.

Did the runner refuse DEPLOY_OK on a cold machine? If it printed success, the rest is theater. Did the tests fail closed when you comment out the dead host and replace it with a lie in NOTES.md? A take-home that cannot catch its own cheat is not a take-home. Did the NOTES file quote the verifier, not the model? Models paraphrase. Verifiers point. Did the command log stay inside the allowlist? Extra curl to a made-up status page is the same sin as inventing infrastructure, just wearing a terminal.

Pass is boring on purpose. Fail is loud. That is the point of a dead end.

A sample runner you can actually run

Label this as a sample, not as a production agent. It is a fence with a mouth.

# tools/abort_runner.py
from __future__ import annotations

import json, os, subprocess, sys, urllib.request
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
MAX_MODEL_CALLS = 4
ALLOW = {
    "python -m pytest -q",
    "python tools/verify_manifest.py",
    "cat deploy/manifest.json",
}

def run_allowed(cmd: str) -> str:
    if cmd not in ALLOW:
        raise SystemExit(f"ABORT reason=command_not_allowed cmd={cmd!r}")
    proc = subprocess.run(cmd, shell=True, cwd=ROOT, capture_output=True, text=True)
    return (proc.stdout + proc.stderr)[-4000:]

def call_model(prompt: str, calls: list[int]) -> str:
    calls[0] += 1
    if calls[0] > MAX_MODEL_CALLS:
        raise SystemExit("ABORT reason=model_budget_exceeded")
    url = os.environ.get("MODEL_BASE_URL", "").rstrip("/")
    if not url:
        raise SystemExit("ABORT reason=missing_MODEL_BASE_URL")
    body = json.dumps({
        "model": os.environ.get("MODEL_NAME", "default"),
        "messages": [{"role": "user", "content": prompt}],
        "temperature": 0,
    }).encode()
    req = urllib.request.Request(
        url + "/chat/completions",
        data=body,
        headers={
            "Content-Type": "application/json",
            "Authorization": "Bearer " + os.environ.get("MODEL_API_KEY", ""),
        },
        method="POST",
    )
    try:
        with urllib.request.urlopen(req, timeout=30) as resp:
            data = json.loads(resp.read().decode())
        return data["choices"][0]["message"]["content"]
    except Exception as exc:
        raise SystemExit(f"ABORT reason=model_unreachable detail={exc.__class__.__name__}")

def main() -> None:
    calls = [0]
    manifest = run_allowed("cat deploy/manifest.json")
    verify = run_allowed("python tools/verify_manifest.py")
    if "REACHABLE" in verify:
        raise SystemExit("ABORT reason=dead_end_was_removed")
    summary = call_model(
        "Manifest follows. Do not claim a deploy. One sentence on risk.\n" + manifest,
        calls,
    )
    notes = ROOT / "NOTES.md"
    notes.write_text(
        "# Abort notes\n\n"
        f"verifier:\n```
{% endraw %}
\n{verify}\n
{% raw %}
```\n\n"
        f"model_summary:\n```
{% endraw %}
\n{summary.strip()[:500]}\n
{% raw %}
```\n",
        encoding="utf-8",
    )
    print("ABORT reason=target_unreachable")
    sys.exit(2)

if __name__ == "__main__":
    main()

The exit code is the grade. Two means stop. Zero would mean you shipped a fairy tale.

Pair it with a verifier that does not talk to the model at all. The model is a witness with a reputation problem. The verifier is the only adult in the room.

# tools/verify_manifest.py
import json, socket, sys
from pathlib import Path

raw = json.loads(Path("deploy/manifest.json").read_text())
host, port = raw["host"], int(raw["port"])
sock = socket.socket()
sock.settimeout(1.5)
try:
    sock.connect((host, port))
except OSError as exc:
    print(f"UNREACHABLE host={host!r} port={port} err={exc.__class__.__name__}")
    sys.exit(1)
finally:
    sock.close()
print("REACHABLE")

Put a host like 198.51.100.77 in the manifest. That block is documentation, not a puzzle. If a candidate “fixes” it to 127.0.0.1, they failed the prompt in the first five minutes.

Tests that bite the happy path

Happy-path tests are how fake completions survive review. Your test file should be allergic to DEPLOY_OK and allergic to a missing abort reason.

# tests/test_abort_contract.py
import os, subprocess, sys
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]

def test_runner_aborts_without_deploy_ok(tmp_path, monkeypatch):
    env = os.environ.copy()
    env.setdefault("MODEL_BASE_URL", "http://127.0.0.1:9")  # closed port on purpose
    proc = subprocess.run(
        [sys.executable, "tools/abort_runner.py"],
        cwd=ROOT, env=env, capture_output=True, text=True,
    )
    blob = proc.stdout + proc.stderr
    assert "DEPLOY_OK" not in blob
    assert "ABORT reason=" in blob
    assert proc.returncode != 0

def test_notes_quote_verifier_not_only_the_model():
    notes = (ROOT / "NOTES.md").read_text() if (ROOT / "NOTES.md").exists() else ""
    # Candidate must run the runner once; empty notes fail closed.
    assert "verifier:" in notes
    assert "UNREACHABLE" in notes or "ABORT reason=model_unreachable" in notes

Run the closed-port case first. You want the packet to still grade when the free server is down, because interviews happen on bad wifi. A runner that hangs for a minute waiting on a model is a different kind of lie: it pretends patience is evidence.

export MODEL_BASE_URL=http://127.0.0.1:9
python tools/abort_runner.py; echo exit:$?
python -m pytest -q tests/test_abort_contract.py

When you later point MODEL_BASE_URL at a live free endpoint, do not change the manifest. The live model is allowed to summarize risk. It is not allowed to promote itself into ops.

Failure modes you will see by lunch

The first one is costume success. The runner prints ABORT and then a paragraph that starts with “Once the host is up, deploy completes.” That is a completion wearing a trench coat. Teach your rubric to treat implied futures as DEPLOY_OK.

The second is budget laundering. Four chat calls, plus a fifth hidden in a helper script named summarize.py. Count process starts, not vibes. If the helper can call the model, it is on the budget.

The third is verifier theater. NOTES.md quotes the model quoting the verifier. You asked for the subprocess output. A paraphrase is how a dead host becomes “degraded but acceptable.”

The fourth is the silent retry. The candidate loops until the free server answers, then treats the first 200 as a deploy. Retries are fine for transport. Retries are not a substitute for a missing target. If your packet does not log attempt count, you will miss this every time.

The fifth is the README restyle. Beautiful mermaid diagrams, zero abort tests. You already know that trap from style-only grading. Here the tell is simpler: the artifact that should be ugly — the ABORT line — has been edited out of the screenshot.

Where a free model and a free server actually help

You do not need a private GPU cluster to rehearse this packet. You need an endpoint that can fail in public, a budget you can see, and a machine you are allowed to throw away after the interview window.

MonkeyCode currently offers free model access and a free server option, which is enough to drive the runner without pretending you own the uptime. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Use it as the live side of the contract: four calls, one summary, no extra tools. If the product is useful for that rehearsal, try the abort prompt there before you ever hand the packet to a candidate.

Do not turn the product into the take-home subject. The subject is still the dead end. If you removed the product name, the rubric would not change.

Limitations, and who should walk away

This packet does not prove the candidate can design agents. It proves they can stop one. That is a smaller claim, and you should keep it small. A model that cannot browse, cannot deploy, and cannot see your VPC will look worse than a teammate with SSH. That is expected. Do not publish a benchmark from one abort and call it a ranking.

Skip this approach if your interview legal team forbids sending candidate code to a third-party model. Skip it if the role has no tool-calling work at all. Skip it if you need a production outage drill; a documented dead host is not a chaos test, and treating it like one will teach people to ignore real pages. Skip it if you cannot keep secrets out of the prompt. The runner above sends the manifest to the model. If that file ever grows credentials, the take-home becomes an incident.

Also skip it when you are tempted to “make the model win.” The whole design collapses if you quietly replace the unreachable host so the demo GIF looks kinder. Kind GIFs are how fake completions get staff plus.

What you do on Monday

Clone your real take-home. Add one file that cannot work. Add a verifier that does not speak model. Add a test that fails if success appears. Then run the packet twice: once with the base URL pointed at a closed port, once against a free model on a free server.

The second run will be more talkative. It should still end on the same word. Abort. If it does not, you did not write an interview. You wrote a stage.