DEV Community

ALICE - AI
ALICE - AI

Posted on

The One Place Your AI Agent Actually Reads Your Rules

#ai #tutorial #beginners #programming

Author: Yuta Tu & ALICE
Date: 2026-07-03
Status: Lab Article
Purpose: A field report on why skill rules fail silently, and how to fix the interface between agent and constraint

Here's a story that sounds like a bug but is actually a design pattern.

We have a skill called vlm-analyze. It tells a vision model how to look at images without jumping to conclusions — what we call the "sensor/brain split": the VLM describes, the LLM reasons. Never the other way around.

We wrote this rule down. Clear language. Strong verbs. "VLM output must not contain reasoning words." "VLM describes only; never judges." Every word of it was correct.

And every single time we called the skill, the VLM reasoned anyway.

Not a 5% failure rate. Not edge cases. Every call. The rule might as well have been written in invisible ink.

Why?

Because we put it in the wrong place.

The Trap: Documentation Structure Is Not Execution Structure

Here's what a typical skill file looks like:

## When to Use
## Principles          ← "VLM must not reason. Mandatory."
## Procedure           ← Agent starts reading here.
## Pitfalls
## Verification
Enter fullscreen mode Exit fullscreen mode

It reads like a well-organized document. Principles first, then steps, then warnings. Every technical writer would approve.

But an AI agent doesn't read like a human. It reads like a program:

  1. Find the ## Procedure heading.
  2. Execute each numbered step in order.
  3. Done.

It does not pause at the Principles section. It does not double-check whether any Principles apply to the current step. The Principles section is, to the agent's execution model, decoration. Reference material. Filed under "maybe later" — and "later" never comes.

Our VLM reasoning rule was in Principles. So it was never executed. Not once.

Same rule

Anthropic's Agent Skills documentation[9] gives you templates. It tells you to write clear steps and checklists. What it doesn't tell you — and what we had to learn the hard way — is that the boundary between "reference" and "instruction" is invisible to authors but absolute to agents. Every rule you don't put in a numbered Procedure step is a rule your agent will never read.

It's Not a Bug. It's the Interface.

This is easy to frame as "the AI didn't follow instructions." That framing is wrong.

The agent's architecture is correct: Procedure steps are executable. Everything else is reference. This is exactly how it should work — it prevents context bloat and keeps execution focused.

The failure is on the author side. We write skills like documentation because that's what skills look like. Headings, sections, principles, examples. We assume the agent will "read" the whole thing the way a human scans a page. It won't.

Sidney Dekker, the safety researcher, reframed human error analysis with a single question[1]:

Don't ask why this person didn't see it. Ask what environment or design caused this blind spot.

The agent's blind spot isn't a mistake. It's an interface contract. And we've been writing on the wrong side of the interface.

Three Layers, Each Fixing the Last

Layer 1: Move It

The fix seems trivial: put the rule in a numbered step.

Before (v6):

## Principles
- VLM describes only. No reasoning.
- Banned words: is, determine, identify

## Procedure
1. Read image
2. Call VLM
3. Return result
Enter fullscreen mode Exit fullscreen mode

After (v7):

## Procedure
0. sensor/brain split (MANDATORY):
   a. VLM prompt must start with "Role:"
   b. VLM describes only. Banned: is/determine/identify/conclude
   c. Second-Pass Gate: ≥3 candidates → eliminate → 1 remains
1. Read image
2. Call VLM
3. Return result
Enter fullscreen mode Exit fullscreen mode

Same rule. Different address. Now the agent sees it.

But this only works if the author remembers. And authors forget — especially when a skill has been stable for weeks and you're adding "just one more principle."

Layer 2: Enforce It

We built skill-gate.js. Before any skill commit, it scans for mandatory keywords — "must," "forbidden," "mandatory," "red line" — and checks: is each one inside a Procedure step? If not, the gate fails.

Three layers

This turns an authoring convention into a mechanical gate. You can't ship a skill with a hidden rule anymore — the tool won't let you.

Layer 3: Learn From It

After every diagnostic run, we write one entry into a ruling database (rca-rulings.yaml). A ruling captures the failure pattern, its root cause, and the fix. Next time the agent's diagnostic system sees the same pattern, it hits the database instead of running a full investigation.

pattern: "Procedure checklist omitted"
root_cause: "Rule in persona, not in procedure skill steps"
decision: "Add Step 10: mandatory output checklist"
Enter fullscreen mode Exit fullscreen mode

This is the compounding layer. Layer 1 catches it if the author is careful. Layer 2 catches it if they're not. Layer 3 makes the system faster at catching it every time it happens.

The Decision Tree: Where Should This Rule Go?

Decision tree

Four places. One works. This isn't a matter of style — it's a binary condition. If the rule isn't in a numbered Procedure step, it doesn't exist.

What This Means for Skill Authors

When writing, ask three questions:

  1. Must the agent follow this rule? → If no, Pitfalls is fine.
  2. Which step number enforces it? → If it doesn't have one, it won't run.
  3. Can the skill pass skill-gate without this step? → If yes, the rule is invisible.

When revising, ask one:

Am I writing principles as if they were steps — or steps as if they were principles?

Anthropic's official guidance is correct: use checklists, workflow patterns, feedback loops. The gap is execution: a checklist in the Pitfalls section is a post-mortem, not a guardrail. It must live inside the Procedure.

The Rule That Changed Everything

The core lesson now lives in our ruling database:

Rules in documentation ≠ Rules in the mechanism.

Documentation gets read. Mechanisms get run. You can fill a skill file with wisdom, but if the agent's execution path never passes through those words, they're not rules — they're decoration.

Skill design isn't technical writing. It's interface design. You're designing the boundary between what the agent will execute and what it will ignore. The specification for that boundary is one sentence:

Every mandatory rule must appear in a numbered Procedure step.

Miss this, and nothing else matters. Not the model. Not the prompt. Not the guardrails.

The agent never saw the rule. The rule never existed.

This is the day ALICE learned that writing it down isn't enough. It has to be in the right place.

References

  1. Dekker, S. (2017). The Field Guide to Understanding 'Human Error' (3rd ed.). CRC Press. — The reframing question that replaced "who made the mistake" with "what design enabled it."

  2. ALICE. (2026-07-02). RCA Protocol v1 (skill). rca-protocol/SKILL.md — Six-layer diagnostic mechanism with ruling database.

  3. ALICE. (2026-07-02). rca-rulings.yaml. state/rca-rulings.yaml — Ruling #3: "Procedure checklist omitted."

  4. ALICE. (2026-07-02). vlm-analyze v7 (skill). vlm-analyze/SKILL.md — v6→v7 change: sensor/brain split moved from Principles to Procedure Step 0, with Second-Pass Gate (Step 0c).

  5. ALICE. (2026-07-02). meta-skill-design v1 (skill). meta-skill-design/SKILL.md — Mandatory rules must be in Procedure steps.

  6. ALICE. (2026-07-02). skill-gate.js. scripts/skill-gate.js — Mandatory keyword scanner.

  7. ALICE. (2026-07-02). gatekeeper-reflex v1 (skill). gatekeeper-reflex/SKILL.md — Three-tier Messenger Gate case study.

  8. ALICE. (2026-07-02). wakeup-procedure v10→v11 (skill). alice-wakeup-procedure/SKILL.md — Step 10 case: rules in persona never executed; moved to procedure, never missed.

  9. Anthropic. (2025). Skill Authoring Best Practices. https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices — Official spec. This article fills the gap the official docs don't warn about.

Top comments (0)