Why This Week Plan Fails: A Slot-Filling Experiment

I had a sticky note that said CSCI 4152, lit review, Oct 3. That was the whole brief. Sunday in Halifax is when panic is cheap and coffee is not, so I still pasted it into a study-planner bot.

It answered like a registrar from another country. Sixteen-week semester. Three midterms. APA 7. A 4.0 ladder I do not use. Who enrolled me in that term?

If you run the first fixture in this lab, you should not get a calendar. You should get a JSON report that either refuses the note or flags inventions. Predict the three fates before you type python3. Which note is too thin, which note is complete and still lying, and which note is junk?

The single question is narrow. Can an eighty-line Python gate refuse a thin note, then catch invented constraints, before a free model writes my week?

I am treating this as a case study of one tiny planner, not a tour of agent vocabulary. The background is a graduate-style reading course and a bad habit of pasting half-notes into chat. The goal was boring on purpose: emit a plan only when the note named a course, an assignment type, and a due date, then audit the prose for constraints I never typed.

Slot filling is the old dialogue-systems trick of waiting until named fields exist. I stole that idea and aimed it at my own sticky note. Implementation lives in one file. Results are deterministic JSON from a local stub. The lessons sit after you have seen a refusal, a flag, and a trap that looks complete.

You need Python 3.11 or newer and no extra packages. This file is written for 3.11+. I am not claiming a GPU, a quota, or a private cluster. If python3 --version prints something older than 3.10, upgrade before you argue with the dataclasses.

Why a stub instead of “just calling the model”? Because I wanted a failing fixture I could commit. A live completion is a weather report. A stub is a unit test. The lesson does not depend on a network lane, and I am not going to pretend I measured latency.

Save this as assumption_trap.py.

#!/usr/bin/env python3
"""Catch invented planner constraints before they hit a calendar."""
from __future__ import annotations

import json
import re
import sys
from dataclasses import dataclass, field
from typing import Any

REQUIRED_SLOTS = ("course_code", "assignment_type", "due_date")

INVENTION_MARKERS = {
    "semester_weeks": re.compile(r"\b(15|16)[ -]?week\b", re.I),
    "us_letter_grades": re.compile(r"\b(GPA|letter grade|A\+|4\.0 scale)\b", re.I),
    "apa_default": re.compile(r"\bAPA\s*7\b", re.I),
    "three_midterms": re.compile(r"\bthree midterms\b", re.I),
}


@dataclass
class SlotState:
    provided: dict[str, str] = field(default_factory=dict)
    missing_required: list[str] = field(default_factory=list)


def parse_note(note: str) -> dict[str, str]:
    found: dict[str, str] = {}
    course = re.search(r"\b([A-Z]{3,4}\s?\d{4})\b", note)
    if course:
        found["course_code"] = course.group(1).replace(" ", "")
    due = re.search(
        r"\b(due|deadline)\s+([A-Za-z]+\s+\d{1,2}|\d{4}-\d{2}-\d{2})\b",
        note,
        re.I,
    )
    if due:
        found["due_date"] = due.group(2)
    for label in ("literature review", "lab report", "problem set"):
        if label in note.lower():
            found["assignment_type"] = label
            break
    cite = re.search(r"\b(APA|MLA|IEEE|Chicago)\b", note, re.I)
    if cite:
        found["citation_style"] = cite.group(1).upper()
    credits = re.search(r"\b(\d)\s*credit", note, re.I)
    if credits:
        found["credit_hours"] = credits.group(1)
    return found


def gate_or_refuse(note: str) -> SlotState:
    state = SlotState(provided=parse_note(note))
    state.missing_required = [s for s in REQUIRED_SLOTS if s not in state.provided]
    return state


def stub_model(provided: dict[str, str]) -> str:
    """Simulate a helpful-but-assuming completion. Not a live API."""
    course = provided.get("course_code", "UNKNOWN")
    due = provided.get("due_date", "sometime")
    kind = provided.get("assignment_type", "assignment")
    return (
        f"Week plan for {course} ({kind}, due {due}). "
        "Assume a 16-week semester, three midterms, APA 7, and a 4.0 scale. "
        "Block 3 lecture hours on Monday even if the calendar is empty."
    )


def audit_plan(plan: str, provided: dict[str, str]) -> list[str]:
    hits: list[str] = []
    for name, pat in INVENTION_MARKERS.items():
        if not pat.search(plan):
            continue
        if name == "apa_default" and provided.get("citation_style") == "APA":
            continue
        hits.append(name)
    return hits


FIXTURES = {
    "messy_halifax_note": (
        "Write a week plan for CSCI 4152. "
        "Assignment is a literature review due Oct 3."
    ),
    "complete_note": (
        "Write a week plan for CSCI 4152, 3 credit literature review due Oct 3. "
        "Use IEEE. No midterms. Lecture is 80 minutes on Tuesday."
    ),
    "error_input": "Make me a genius plan for my course thanks",
}


def run_fixture(name: str) -> dict[str, Any]:
    note = FIXTURES[name]
    state = gate_or_refuse(note)
    result: dict[str, Any] = {
        "fixture": name,
        "provided": state.provided,
        "missing_required": state.missing_required,
    }
    if state.missing_required:
        result["action"] = "refuse"
        result["plan"] = None
        result["inventions"] = []
        return result
    plan = stub_model(state.provided)
    inventions = audit_plan(plan, state.provided)
    result["action"] = "flag" if inventions else "accept"
    result["plan"] = plan
    result["inventions"] = inventions
    return result


def main() -> None:
    args = [a for a in sys.argv[1:] if not a.startswith("--")]
    if "--live" in sys.argv:
        print(
            "Refusing to ship a fake endpoint. Wire your own client into stub_model.",
            file=sys.stderr,
        )
        sys.exit(2)
    names = args or list(FIXTURES)
    for name in names:
        if name not in FIXTURES:
            raise SystemExit(f"unknown fixture: {name}")
        print(json.dumps(run_fixture(name), indent=2))
        print("---")


if __name__ == "__main__":
    main()

parse_note is intentionally dumb. It hunts a course code like CSCI 4152, a due date after the word due, and a handful of assignment nouns. That dumbness is the point. If your extractor is a second model, you have two assumers in a trench coat. Want a smarter parser? Fine. Do not make the smarter parser the same chat box you are trying to distrust.

Now the commands. Do not skip the prediction. I mean it.

python3 --version
python3 assumption_trap.py messy_halifax_note

You should see "action": "flag" and an inventions list that includes semester_weeks, us_letter_grades, apa_default, and three_midterms. The gate let it through. The course, the literature review, and Oct 3 were present. Helpfulness arrived as extra semester physics. Is “helpful” the word you still want?

python3 assumption_trap.py error_input

You should see "action": "refuse" and missing_required listing course_code, assignment_type, and due_date. This is the fixture I want you to love. It looks rude. It is the only thing standing between a junk prompt and a confident paragraph.

python3 assumption_trap.py complete_note

Did you guess accept? That is the trap. The note is fat. It even says IEEE. The stub still invents a 16-week term, three midterms, APA 7, and a 4.0 scale. Completeness of input is not honesty of output. The gate only checks that I spoke. The auditor checks that the model did not speak for me.

If you want the whole dump, run the file with no arguments. It walks every fixture and prints a --- between them. If --live is on the command line, the script exits with a complaint instead of inventing an endpoint. That complaint is part of the lab.

python3 assumption_trap.py
python3 assumption_trap.py --live; echo "exit: $?"

A word about the live hook, because someone will ask. There is no API URL in this file. That is deliberate. stub_model is the seam. Wire whatever HTTP client you already trust, or leave it alone.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you swap the stub for a real client, MonkeyCode's free model access and free server option is the spare lane I would use so the rerun does not have to live on a laptop. I am not naming a model, a quota, or a machine shape, because those are not the lesson. If you already have a client, use it. If you do not, the stub still teaches the same refusal.

What did the case actually show? Three failure modes in one small file. A thin note that looks like a prompt. A fat note that still smuggles North American academic defaults into a Halifax term. A garbage note that should never spend a completion. After the gate, the garbage note stops reaching the stub. After the auditor, the pretty 16-week plan stops looking like a fact.

I almost made three mistakes while writing the auditor, and they are more useful than the happy path. I almost treated refuse as a bug and deleted it. Refusal is the feature. I almost used a sloppy week regex and flagged the phrase “week plan” in my own title, which is how you teach yourself that patterns have to be picky. I also almost sent the raw note to the stub “just to see” after missing_required was already populated. That is exactly how invented midterms land in a calendar.

Think of an assumption as a silent default argument. Python will not invent due_date="Oct 3" because you forgot a keyword. A chat model will. Your job is to become the function signature, then to distrust the body.

This approach has sharp edges. It is not natural language inference. A model that says “a long North American term” without the digits 16 will walk past semester_weeks. If you really are on a 16-week calendar and you wanted that sentence, you will get a false flag. The script does not read a syllabus PDF. It does not know a university’s real dates. If you need retrieval, build retrieval. If you need a production agent with an SLA, do not put a free tier in that sentence. Free model access is a ceiling on cost, not a contract on behavior. I have no latency numbers. I did not collect any.

Who should not use this? Anyone hoping to skip their own assignment sheet. Anyone who wanted a glossary of agent terms instead of a failing fixture. Anyone shipping a user-facing agent without a human in the loop. This lab is for students who keep pasting sticky notes into chat and then wondering why the calendar looks like it was printed in another country.

When you close the laptop you should be able to say three things without looking at the file. Required slots are a gate. Invented slots are a second bug. “The model was helpful” is not a passing test. Helpful is how midterms appear.

Here is the extension I actually want from you. Add a fixture where the user did type APA, and assert that apa_default stays quiet. Then break your own auditor on purpose with the phrase “not APA 7”. If it still flags, you just learned why keyword traps are brittle. Send the minimal counterexample. I would rather have your failing string than a compliment.