DEV Community

Devflow AI
Devflow AI

Posted on

SKILL.md is not a compiler

If you use Cursor long enough, you will watch it ignore a rule you wrote down.

You add a SKILL.md. You tell it "no explicit any". You maybe add a hook. It agrees. Then it ships id: any anyway.

A comment on a thread about skill files put it cleanly: the model can work around hooks too. That is not a Cursor bug. It is what "the instruction is text" means.

What a Skill File Actually Is
A SKILL.md is a document the agent is supposed to read when the task matches. .cursor/rules and hooks are the same family: they change the prompt, not the type system.

Attention is limited. Mid-thread, the ratio of "your rules" to "the last twenty tool results" gets worse. The model locally prefers completing the user's latest sentence over satisfying a file it saw at the start of the turn.

So:

Skills are good at default shape: interfaces, try/catch, test file next to the change.

Skills are bad at guarantees.

If your workflow assumes the markdown file is a compiler, you will keep getting surprised.

The Setup That Survives a Skip
Treat generation and verification as different jobs.

Layer 1 โ€” The Skill
Tell the agent how you want code to look. Keep it short. Long skill files are just more tokens to ignore.

Layer 2 โ€” A Command That Can Fail
After the edit, run whatever this repo already uses:

Bash
npx tsc --noEmit
npm test -- --run path/to/changed-file
Use the scripts in package.json if they exist. Do not invent a parallel toolchain so the agent can look thorough.

The skill's job is to remember to run Layer 2, and to refuse the sentence "this is production-ready" when Layer 2 did not run.

That is the whole product. Not a smarter prompt. A prompt that stops lying when tsc is red.

A Small Example
Layer 1 alone often still emits this:

TypeScript
async function getUser(id: any) {
const res = await fetch('/api/user/' + id)
return res.json()
}
You can write a skill that forbids any. Sometimes it works. Sometimes the chat is long and it does not.

Layer 2 does not care about the chat. tsc --noEmit fails until id is typed and res.ok is handled. Tests fail until a non-200 is covered. The agent then has to edit against a failing command, which is a much stronger signal than a paragraph in SKILL.md.

What to Do in a Live Session
Keep skills tiny: Four rules beat forty.

End requests cleanly: End every "make this production-ready" request with "run typecheck and tests, paste the output."

Reset threads: Start a new chat when the thread is doing two features. Skills do not reset attention math.

Use hooks strictly: If you use hooks, keep them for formatters and secret scans โ€” deterministic tools. Do not expect a hook to be a second personality that the model cannot dodge.

If You Want the File
The SKILL.md I use for this is a two-layer review gate: read the diff, then run the repo's typecheck/lint/test and print the commands. It lives in this studio as two-layer-review-gate. Use it as a template; the valuable part is Layer 2, which you already have if your repo has tsc.

I will put the listing on PromptBase after the current reviews clear. Until then, the article is the whole method.

Subscribe if you want the next one: how to hand a session to a fresh chat without dumping 40k tokens of history into the new thread.

Originally published on Substack: https://apparatusappello861559.substack.com/p/skillmd-is-not-a-compiler

Top comments (4)

Collapse
 
mk023 profile image
Marco

This is a really good distinction between instruction and enforcement. ๐Ÿ‘

I especially like the separation between the Skill as the desired behavior and the repo tooling as the actual gate. A markdown rule can influence what the agent does, but tsc, tests, schema validation, or security scanners can independently tell us whether the resulting artifact satisfies a property.

There is also a security parallel here ๐Ÿ”: the closer a requirement is to an actual capability boundary, the less I want it to exist only as agent-readable text. If "don't do X" matters, the strongest version is usually a control that makes X fail even when the model ignores the instruction.

In other words: the skill can tell the agent what to do; the verification layer should be able to prove what actually happened.

That's a much more useful mental model than treating SKILL.md as a compiler. ๐Ÿง 

Collapse
 
rulestack profile image
Rulestack

The two-layer split matches what I've seen with Claude Code skills too: the skill's most reliable job is remembering to run the command, not enforcing the rule itself. One nuance on hooks โ€” in Claude Code a PostToolUse hook that exits 2 pushes its stderr back into the model's context, so it sits closer to your Layer 2 than to 'a second personality'; it still can't undo the edit, but the model can't claim it didn't see the failure. Does Cursor have an equivalent return path, or do hooks there only fire and log?

Collapse
 
citedy profile image
Dmitry Sergeev

finally someone said it. cursor definitely starts ignoring rules once the context window gets too cluttered lol

Collapse
 
citedy profile image
Dmitry Sergeev

ngl i've been fighting with.cursorrules for weeks, gonna try the SKILL.md approach and see if it actually sticks