DEV Community

Cover image for Replacing 800 Lines of AI Agent Instructions with 10-Token Questions
Mike Lane
Mike Lane

Posted on

Replacing 800 Lines of AI Agent Instructions with 10-Token Questions

The Problem

"A problem is a difference between things as desired and things as perceived."
-- Gause & Weinberg

My CLAUDE.md file is 800+ lines of rules. It specifies explicit file staging, mandatory TDD, worktree isolation, GPG signing, and dozens of other constraints. Claude Code reads every line at session start, but by the third or fourth commit, the agent often runs git add -A anyway.

The instructions exist in the context window, but they arrived at position zero. By the time the agent reaches the decision to stage files, the relevant rule is buried under thousands of tokens of conversation history, code output, and tool results. The desired behavior (stage specific files) and the perceived behavior (git add -A is fine) diverge because the instruction and the decision are separated by positional distance in the context. The agent has the knowledge but lacks the prompt at the moment the decision happens.

"Are Your Lights On?"

Gause and Weinberg's Are Your Lights On? (1982) is a book about problem definition. The title comes from a story: a mountain tunnel in the Swiss Alps had a recurring problem with drivers leaving their headlights on after exiting, draining batteries while they hiked. Engineers proposed sensors, conditional instructions, automated shutoff systems. The solution that worked was a sign at the tunnel exit: "Are your lights on?"

The driver already knew how headlights work. The sign delivered a question at the decision boundary, the tunnel exit, where the driver could still act on the answer. My agent already knows the rules; what it needs is the right question at the moment it is deciding.

The Mechanism

"Don't solve other people's problems when they can solve them perfectly well themselves."
-- Gause & Weinberg

Claude Code hooks are shell scripts that fire at two lifecycle points: PreToolUse (before a tool executes) and PostToolUse (after). Each hook receives JSON on stdin containing the tool input and, for PostToolUse, the tool response. A hook that exits 0 with text on stdout prints a message to the agent. A hook that exits 2 hard-blocks the tool call and sends its stderr as the error message.

The design choice worth examining is framing hooks as questions rather than commands. "Is there a failing test?" works where "You must write a failing test before writing production code" does not, because the question forces the agent to evaluate its current state against the standard at the moment of the decision. The command gets acknowledged when the context loads and loses salience by decision time.

Registration lives in .claude/settings.json. Here is the relevant structure for code-editing hooks:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Edit",
        "hooks": [
          {
            "type": "command",
            "command": "bash ~/.claude/hooks/lights-on-code.sh"
          }
        ]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "Edit",
        "hooks": [
          {
            "type": "command",
            "command": "bash ~/.claude/hooks/lights-on-code-post.sh"
          }
        ]
      }
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

The matcher field determines which tool triggers the hook. The same script can be registered for multiple matchers (I use lights-on-code.sh for both Edit and Write). The hook runs in a subprocess with no access to the agent's state, only the JSON payload on stdin.

TDD Discipline: Bracketing the Decision

"What is the problem?"
-- Gause & Weinberg (asked repeatedly, at different stages)

The agent's most common TDD violation is writing production code before writing a failing test. Two hooks bracket the edit decision: one fires before the agent writes code, one fires after.

lights-on-code.sh (PreToolUse for Edit and Write):

#!/bin/bash
# "Are Your Lights On?" — PreToolUse:Edit/Write
# TDD enforcement: When writing production code, is there a failing test?
# Always exits 0 (non-blocking).

set -euo pipefail

TOOL_INPUT=$(cat 2>/dev/null) || true
[[ -z "$TOOL_INPUT" ]] && exit 0

FILE_PATH=$(echo "$TOOL_INPUT" | jq -r '.tool_input.file_path // empty' 2>/dev/null) || true
[[ -z "$FILE_PATH" ]] && exit 0

FILENAME=$(basename "$FILE_PATH")
EXTENSION="${FILENAME##*.}"

# --- Only care about code files ---
case "$EXTENSION" in
    py|ts|tsx|js|jsx|go|rs|kt|java|rb|swift|c|cpp|h) ;;
    *) exit 0 ;;  # Not a code file — skip
esac

# --- Is this a test file? If so, great — that's the right thing to write first ---
if echo "$FILENAME" | grep -qiE '^test_|_test\.|\.test\.|\.spec\.|_spec\.|Test\.|Tests\.|_tests\.'; then
    exit 0
fi

# Also check if the path contains a test directory
if echo "$FILE_PATH" | grep -qiE '/tests?/|/spec/|/__tests__/|/test_'; then
    exit 0
fi

# --- This is production code. Ask the TDD question. ---
echo "Is there a failing test for this change? (TDD: write the test first, watch it fail, then make it pass)"

exit 0
Enter fullscreen mode Exit fullscreen mode

lights-on-code-post.sh (PostToolUse for Edit and Write):

#!/bin/bash
# "Are Your Lights On?" — PostToolUse:Edit/Write
# TDD discipline checks AFTER code has been written.
# Different questions for production code vs test code.
# Always exits 0 (non-blocking).

set -euo pipefail

TOOL_INPUT=$(cat 2>/dev/null) || true
[[ -z "$TOOL_INPUT" ]] && exit 0

FILE_PATH=$(echo "$TOOL_INPUT" | jq -r '.tool_input.file_path // empty' 2>/dev/null) || true
[[ -z "$FILE_PATH" ]] && exit 0

FILENAME=$(basename "$FILE_PATH")
EXTENSION="${FILENAME##*.}"

# --- Only care about code files ---
case "$EXTENSION" in
    py|ts|tsx|js|jsx|go|rs|kt|java|rb|swift|c|cpp|h) ;;
    *) exit 0 ;;  # Not a code file — skip
esac

# --- Is this a test file? ---
IS_TEST=false
if echo "$FILENAME" | grep -qiE '^test_|_test\.|\.test\.|\.spec\.|_spec\.|Test\.|Tests\.|_tests\.'; then
    IS_TEST=true
fi
if echo "$FILE_PATH" | grep -qiE '/tests?/|/spec/|/__tests__/|/test_'; then
    IS_TEST=true
fi

if [[ "$IS_TEST" == "true" ]]; then
    # --- Test code: Did you write just enough to fail? ---
    cat <<'MSG'
Did you write more test code than is sufficient to fail? (Uncle Bob's Rule #2: write only enough test to get a failure — including compilation failures.)
Do you need test triangulation? (If a behavior could be faked with a hardcoded return, add another example that forces the real implementation.)
MSG
else
    # --- Production code: Did you write just enough to pass? ---
    cat <<'MSG'
Did you write more production code than is sufficient to pass the one failing test? (Uncle Bob's Rule #3: minimal code to make the test green.)
Does this code violate the Dependency Inversion Principle? (High-level modules should not import concrete implementations — depend on abstractions.)
MSG
fi

exit 0
Enter fullscreen mode Exit fullscreen mode

The Pre and Post hooks bracket the decision with different questions. Before writing, the question concerns prerequisites: does a failing test exist? After writing, the question concerns scope: did you write more than necessary? Weinberg's "What is the problem?" applies at both boundaries, but the specific form changes because the agent's context has changed. Before the edit, the problem is whether the agent is starting from the right place; after the edit, the problem is whether it stayed within bounds.

Error Recovery: Three Questions for Three Assumptions

"If you can't think of at least three things that might be wrong with your understanding of the problem, you don't understand the problem."
-- Gause & Weinberg

When a Bash command fails, the agent's default behavior is to retry the same command with minor variations. The lights-on-post.sh hook intercepts this pattern with three targeted questions:

# --- After ANY Bash command: check for failure (skip test commands — those fail on purpose during TDD RED) ---
TOOL_RESPONSE=$(echo "$TOOL_INPUT" | jq -r '.tool_response // empty' 2>/dev/null) || true
if [[ -n "$TOOL_RESPONSE" ]]; then
    # Skip test commands — failures are expected during RED phase
    IS_TEST_CMD=false
    if echo "$COMMAND" | grep -qE '(cargo\s+test|pytest|pnpm\s+test|npm\s+test|npx\s+jest|go\s+test|gradle.*\s+test|\.\/gradlew.*\s+test)'; then
        IS_TEST_CMD=true
    fi

    if [[ "$IS_TEST_CMD" == "false" ]]; then
        if echo "$TOOL_RESPONSE" | grep -qiE 'error:|fatal:|command not found|no such file|permission denied|ENOENT|EACCES|panic:|traceback|ModuleNotFoundError|ImportError'; then
            echo "The command appears to have failed. Before retrying the same approach: (1) Is there a dedicated tool (Read, Grep, Glob, Edit) that handles this? (2) Did you read the error message carefully? (3) Is the approach itself wrong?"
        fi
    fi
fi
Enter fullscreen mode Exit fullscreen mode

Each of the three questions challenges a different assumption. Question 1: the agent may be using the wrong tool, since Claude Code has dedicated tools for file operations that are more reliable than shell equivalents. Question 2: the agent may have skipped information already present in the error output. Question 3: the entire approach may be flawed, not just the execution.

The hook excludes test commands because test failures during TDD's RED phase are the desired state. Without this exclusion, every intentional test failure would generate a warning, which would train the agent to ignore the hook's output entirely.

When Questions Are Not Enough

"Each solution is the source of the next problem."
-- Gause & Weinberg

Some agent behaviors are not correctable by questions because the consequences are irreversible. Force-pushing to main overwrites remote history. Bypassing GPG signing breaks the chain of trust. Disabling git hooks with --no-verify defeats the verification system entirely. For these cases, block-dangerous-commands.sh uses exit 2 to hard-block the tool call:

block_with_reason() {
    local reason="$1"
    echo "BLOCKED: $reason" >&2
    exit 2
}

# Force pushes to main/master
if echo "$COMMAND" | grep -qE 'git\s+push\s+.*--force|git\s+push\s+-f'; then
    if echo "$COMMAND" | grep -qE '\s(main|master)\b'; then
        block_with_reason "Force push to main/master is forbidden"
    fi
fi

# Prevent skipping git hooks
if echo "$COMMAND" | grep -qE 'git\s+commit.*--no-verify|git\s+push.*--no-verify'; then
    block_with_reason "Skipping git hooks (--no-verify) is not allowed"
fi

# Prevent bypassing hooks via environment variables
if echo "$COMMAND" | grep -qE 'LEFTHOOK=0|HUSKY=0'; then
    block_with_reason "Bypassing git hooks via environment variables is not allowed. Fix the issue instead of disabling the hook."
fi
Enter fullscreen mode Exit fullscreen mode

The design boundary between soft nudge (exit 0) and hard block (exit 2) is reversibility. If the agent stages all files instead of specific ones, I can unstage them; if the agent force-pushes to main, the remote history is rewritten and recovery requires coordination with every collaborator. Using hard blocks for every hook would be counterproductive: the agent would spend its token budget searching for workarounds instead of doing the work.

The Full System

"Whose problem is it?"
-- Gause & Weinberg

Twelve scripts cover the decision boundaries I care about:

Hook Trigger What It Asks
block-dangerous-commands.sh PreToolUse:Bash Hard-blocks force-push, --no-verify, rm -rf /
lights-on.sh PreToolUse:Bash "Are you staging specific files?", "Are you on the right branch?"
lights-on-code.sh PreToolUse:Edit/Write "Is there a failing test for this change?"
lights-on-task.sh PreToolUse:Task "Is this agent working in a worktree?", "Did you create tasks?"
lights-on-github-pre.sh PreToolUse:mcp_github "Did you run quality checks before this PR?"
lights-on-post.sh PostToolUse:Bash "Did you install deps?", "Is the approach itself wrong?"
lights-on-code-post.sh PostToolUse:Edit/Write "Did you write more code than sufficient to pass?"
lights-on-prose-post.sh PostToolUse:Edit/Write "Did you use em-dashes or AI slop?"
lights-on-read-edit-post.sh PostToolUse:Read/Edit "Did you use Glob to verify the path?"
lights-on-task-post.sh PostToolUse:Task "Did you verify the agent's work?"
lights-on-github.sh PostToolUse:mcp_github "Add to project board? Link as sub-issue?"
notify-git-failures.sh PostToolUse:Bash (Slack notification on git failure)

The system introduces overhead. Each question that fires adds 10-15 tokens to the context window. The tradeoff is those tokens versus the cost of the agent going down a wrong path for 50+ tool calls before I notice. In practice, most hooks exit silently for non-matching commands (an ls command does not trigger the git staging question), so the per-operation cost concentrates on the decisions where drift is most likely.

The prose hook (lights-on-prose-post.sh) illustrates Weinberg's point that every intervention has second-order effects: it occasionally flags legitimate uses of dashes in code comments. The question is whether the cost of false positives is lower than the cost of the original failure mode (AI-generated prose full of em-dashes). For my use case, it is.

Build Your Own

"Not all problems require solutions. But all solutions require a problem."
-- Gause & Weinberg

Three steps to build hooks for your own workflow:

Identify your top 3 agent failure modes. Check your git history for commits you had to fix, and check your conversation logs for repeated corrections. The failures you correct most often are the ones worth hooking.

Determine the decision boundary. If the failure is preventable (the agent should not have taken the action), use PreToolUse. If the failure is correctable (the action happened, but the follow-up was wrong), use PostToolUse. Most failures are correctable, which is why most of my hooks are PostToolUse.

Write a question, not a command. A question makes the problem the agent's to solve; a command makes enforcement your problem, and the agent will comply at context-load time and drift later. Here is a minimal template:

#!/bin/bash
set -euo pipefail
TOOL_INPUT=$(cat 2>/dev/null) || true
[[ -z "$TOOL_INPUT" ]] && exit 0

COMMAND=$(echo "$TOOL_INPUT" | jq -r '.tool_input.command // empty' 2>/dev/null) || true

if echo "$COMMAND" | grep -qE 'YOUR_PATTERN'; then
    echo "Your question here?"
fi

exit 0
Enter fullscreen mode Exit fullscreen mode

Register it in .claude/settings.json:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [{ "type": "command", "command": "bash ~/.claude/hooks/your-hook.sh" }]
      }
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

The Sign at the Exit

Twelve scripts totaling roughly 500 lines of shell reinforce 800+ lines of CLAUDE.md instructions by delivering the relevant subset at the moment the agent is making the decision those instructions address. The mechanism is a question at the decision boundary: specific enough to be actionable, short enough to cost fewer tokens than the mistake it prevents.

Gause and Weinberg figured out in 1982 that the driver does not need a lecture on electrical systems; the driver needs six words on a sign at the tunnel exit. The same principle applies to AI agents, and the implementation is a shell script that reads JSON and prints a sentence. Start with your agent's most frequent failure mode, find the decision boundary where it occurs, and write a question that fits there.

Top comments (0)