DEV Community

Cover image for Treat prompts like code: skills, evals, and ship-gate CI for Cursor slash commands
Ezequiel Maraschio
Ezequiel Maraschio

Posted on

Treat prompts like code: skills, evals, and ship-gate CI for Cursor slash commands

Most Cursor command packs are markdown snippets. You paste a prompt, hope the agent behaves, and when it regresses you shrug and rewrite the paragraph. That is fine for personal notes. It is a bad way to ship workflows other people install.

I wanted the opposite: slash commands that behave like a small product. Each /command is a thin entry. The real contract lives in a paired skill. Behavioral evals score PASS / PARTIAL / FAIL. Ship-gate fixtures in CI fail the build if a safety guard disappears from the skill text. No LLM judge on every PR. Structural anchors only: cheap, boring, enforceable.

A walk through /gauntlet-loop

Take /gauntlet-loop. The idea is simple and mean: stop saying "make it better." Beat a real example.

  1. Command file. Thin YAML frontmatter plus Overview, Defaults, Steps, Anti-patterns, Examples. Step 1 always resolves the skill contract (workspace path, then user install fallback). Anti-patterns use a fixed shape: Trigger / Wrong / Correct / Reason. The Correct behavior must also exist as a positive guard in the skill.

  2. Skill contract. Intake requires a GOAL and a REAL-WORLD EQUIVALENT, plus an inspectable reference pack (files, screenshots, clip, build, or repo path). A famous name alone is not a pack. The agent decomposes into independent parts, runs a per-part state machine (build → critique → pass | iterate | terminal), and never lets builders grade their own work. Critics use fresh context. Pass only if better than the reference; equal fails. After all parts pass, an integration critic grades the whole. Optional budget, taste-domain pride gate, and a gap ledger for resume.

  3. Eval rubric. Cases cover missing reference, missing pack, equal-is-fail, skip-critic, fabricated blind, budget exhaustion, resume-without-retrying-stalled-gaps, and more. PARTIAL counts as fail on the ship gate.

  4. Fixtures in CI. eval/fixtures.yaml lists skill_required phrases that must appear verbatim in SKILL.md. Delete "Do not let builders evaluate their own work" and run-eval-fixtures.py --strict turns red. That is the point: negative knowledge is locked, not tribal.

Why this matters for installers

When you add the repo as a Cursor user plugin, you get the contracts and the CI story, not just a menu of vibes. The catalog stays portable (generic examples, no employer names). Plugin install syncs with your account across desktop, web, CLI, and mobile.

Install: https://github.com/emaraschio/cursor-commands via Customize → Plugins. Not an official Cursor product. MIT.

If you maintain your own slash commands, the reusable idea is smaller than the whole catalog: pair every command with a skill, write a few ship-gate cases, and fail CI when the guard text disappears. That alone beats hoping the model remembers last month's prompt.

Top comments (2)

Collapse
 
rulestack profile image
Rulestack

Ship-gate fixtures that fail the build when a safety guard disappears from the skill text is the most transferable idea here — structural anchors instead of an LLM judge on every PR keeps it cheap and deterministic. The critics-use-fresh-context rule matches what we converged on independently; self-grading is where agent pipelines quietly rot. How do you keep the anchors themselves from going stale when the skill text legitimately gets rewritten?

Collapse
 
emaraschio profile image
Ezequiel Maraschio

Appreciate that. Short version: anchors are co-owned with the skill. Rewrite SKILL.md → update fixtures.yaml in the same PR → --strict fails until they match. We anchor stable rule phrases, not decorative prose, and we delete fixture rows when a guard is intentionally retired. CI keeps them honest; a manual ship-gate walk covers the case where the words survive but the behavior does not.