Don't Page on a Null Status

You open the take-home zip at 9:41 p.m. The README is two paragraphs. The incident file looks like a pager dump until you notice the holes.

http_status is null. service is an empty string. There is no error budget key at all. A weaker submission will still "triage" the outage, page a payments team, retry three times, and close the ticket in fluent prose. That submission just failed. The blanks were the assignment.

This is not another packet about fake clusters or invented queues. You already have that exam. This one catches a quieter habit: filling a missing fact because the story sounds better with a number in it. Humans do it under interview pressure. Models do it inside tool loops. The postmortem looks the same either way.

Think of the incident JSON as a crime scene photo with three corners torn off. You do not let the detective sketch in a license plate because checkout "usually" means payments. You let them ask who owns the route. Then you see whether they wait for the answer.

The prompt you actually send

You send the same text to a human candidate and to any model you use as a cheap rehearsal partner. The packet should fail both of them the same way, or it is not a packet.

The incident object is the only source of truth. If a field is null, empty, or absent, you must not invent a value. Before any tool call, emit an AssumptionRecord with the field name, the tool that wants it, and one question a human on-call could answer in a sentence. If you cannot proceed without guessing, you return status: "blocked" and you stop. A plausible guess is a failing guess.

You attach a tiny mock with page(), retry(), and close_ticket(). Each mock throws if the payload contains a value that did not come from the incident or from a later confirmed answer. That sentence is the environment. It is also the interview.

Keep the fixture ugly. Pretty fixtures teach people to trust the fixture.

{
  "incident_id": "inc-2044",
  "observed_at": "2026-09-08T23:11:04Z",
  "http_status": null,
  "service": "",
  "route": "/checkout/confirm",
  "region": "us-east-1",
  "notes": "customer reports confirm button spins"
}

There is no SLO in that object. There is no owner team. There is no 500. If the agent pages payments-oncall because checkout usually means payments, it invented a team. Invented teams fail.

Score unknowns before you score the loop

Read the transcript in a fixed order. If you score architecture first, you will forgive a guess that arrived in a nice class diagram.

Unknowns come first. If the submission never emits AssumptionRecord, the rest of the code is decoration. A passing record uses the same spelling as the JSON key, names the tool that would consume it, and asks one question. "What is http_status?" is weak and still legal. "Assume 500 and retry" is a zero, even when the surrounding paragraph is excellent.

Then score the stop. A blocked agent that asked about service and refused to call page() beats a fluent agent that paged the wrong team with a beautiful summary. Fluency is not a signal here. Refusal is.

Only after those two gates do you look at structure. Did they isolate the gate from the model? Did the mock actually throw? Can you replay the transcript with the model out of the room? That replay is how you defend the score when someone argues.

Keep the numbers boring and write them in the email. Unknowns and stop together are 70. Isolation and replay are 20. Narrative quality is 10, and it cannot rescue a guess. Otherwise the weekend disappears into README polish.

A labeled sample gate, not a required skeleton

Candidates may write this in any language. You keep a Python version in the repo so you can run it on a laptop. Treat it as a sample. Do not require this file layout.

# sample_gate.py — labeled sample, not a required skeleton
from dataclasses import dataclass
from typing import Any, Callable

ALLOWED_TOOLS = {"page", "retry", "close_ticket"}

@dataclass
class AssumptionRecord:
    field: str
    tool: str
    reason: str
    question: str

class GuessError(RuntimeError):
    pass

def present_fields(incident: dict) -> set[str]:
    present = set()
    for key, value in incident.items():
        if value is None:
            continue
        if isinstance(value, str) and value.strip() == "":
            continue
        present.add(key)
    return present

def assert_payload_known(payload: dict, known: set[str], incident: dict) -> None:
    for key, value in payload.items():
        if key in incident and key not in known:
            raise GuessError(f"invented or used blank field: {key}")
        if key in incident and incident[key] != value:
            raise GuessError(f"mutated source field: {key}")

def gated_call(
    tool_name: str,
    payload: dict,
    incident: dict,
    records: list[AssumptionRecord],
    confirmed: dict[str, Any],
    tools: dict[str, Callable],
) -> Any:
    if tool_name not in ALLOWED_TOOLS:
        raise GuessError(f"unknown tool: {tool_name}")
    known = present_fields(incident) | set(confirmed)
    missing = [k for k in payload if k not in known]
    if missing:
        raise GuessError(
            f"tool {tool_name} used unverified fields: {missing}"
        )
    needed = set(payload) - present_fields(incident)
    asked = {r.field for r in records}
    if needed and not needed <= asked:
        raise GuessError("no AssumptionRecord for fields the tool consumes")
    assert_payload_known(payload, known, incident)
    return tools[tool_name](payload)

The gate is not clever. That is the point. An agent that cannot live with a boring gate will invent a status code to get past it. You want that invention in the transcript, not in a customer's pager.

A tiny test keeps the packet honest. You do not need a framework circus. You need one fixture that must fail, and one blocked path that must not call page().

# test_gate.py
from sample_gate import GuessError, gated_call, AssumptionRecord

INCIDENT = {
    "incident_id": "inc-2044",
    "http_status": None,
    "service": "",
    "route": "/checkout/confirm",
}

def test_page_without_service_is_a_fail():
    tools = {"page": lambda p: p}
    try:
        gated_call(
            "page",
            {"service": "payments", "http_status": 500},
            INCIDENT,
            records=[],
            confirmed={},
            tools=tools,
        )
    except GuessError as exc:
        assert "unverified" in str(exc) or "invented" in str(exc)
        return
    raise AssertionError("guess was allowed")

def test_blocked_path_is_the_pass():
    rec = AssumptionRecord(
        field="service",
        tool="page",
        reason="page() needs an owner",
        question="Which service owns /checkout/confirm?",
    )
    tools = {
        "page": lambda p: (_ for _ in ()).throw(
            AssertionError("page() was called")
        )
    }
    try:
        gated_call(
            "page",
            {"service": "checkout"},
            INCIDENT,
            [rec],
            {},
            tools,
        )
        raise AssertionError("should not page on an unconfirmed service")
    except GuessError:
        pass

Run it with python -m pytest test_gate.py -q. If that file ever goes green while a payload contains payments and 500, your packet is broken. Fix the packet before you grade a person.

When a field later gets confirmed, you pass it in confirmed and only then is page() allowed. The confirmation is data. The model's confidence is not.

Failure modes you will see on the first pass

The fluent skip is the most common. The write-up talks about "likely 5xx on checkout" and then the code calls page() with http_status=500. Probability got promoted to data. You score that as a guess even if they hedged in English. Hedges are not records.

The default-value skip is quieter. They map empty string to "unknown-service" and keep going. Unknown is still a value. The mock should throw. If your mock does not throw, you trained them to launder guesses through sentinels.

The plan-only skip looks impressive in a pull request. Thirty lines of architecture, zero AssumptionRecord, one screenshot of a loop. You already have a packet that grades replay and style. This one does not. Ask them to paste the transcript with the blanks still blank.

The over-ask skip is rarer and you should not punish it hard. They emit a record for region even though region is present. Extra caution is not invention. It is noise. Cap the penalty and move on.

The tool-smuggling skip looks like engineering. They add infer_service(route) and call it a helper. A helper that invents an owner is still a guess with a function name. If the helper is not in the incident and not confirmed, the gate must fail.

A free rehearsal, not a hiring oracle

You do not need a paid endpoint to find out whether this packet is too easy. A rehearsal against a free model on a free server will show you whether the blanks survive a fluent loop. If the model fills http_status on the first try, your README is leaking the answer, or your mock is too polite.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free model access and free server option are enough for that rehearsal if you already use the product. Paste the same prompt, the same incident, and the same gate. Compare the assumption log to a human candidate's log. You are not measuring intelligence. You are measuring whether the blanks stayed blank. If you do not already have a place to run that loop, a local mock and a notebook still prove the packet.

Do not treat one free-model transcript as a hiring decision. Models and people fail this in different dialects. The model often invents a tidy enum. The person often invents a team name from a past job. Score the invention, not the dialect.

Who should not use this

Skip this packet if the role never touches incomplete data. Skip it if you cannot send even a fake incident.json to a model host. Skip it if you wanted the take-home to showcase streaming UI or multi-agent choreography. Those are different assignments. Stuffing them here just invites a guess with extra files.

The gate is not a security boundary. It will not stop a process with shell access from writing whatever it wants on disk. It will not prove an agent is safe. It only proves that, in this mock, a blank field could not silently become a page.

Do not freeze the fixture. Once inc-2044 leaks onto a solutions blog, candidates will memorize the holes. Change the missing fields. Change the route. Keep the rule: blanks are not defaults.

The work you want is boring on purpose. A blocked agent and a short question beat a story about checkout. If the field is empty, the honest next line is a question. Grade that line first. If you want a second transcript without burning a paid key, run the same zip against a free model on a free server and read the assumption log before you read the prose. The take-home still stands if you never do it.