DEV Community

Cover image for I built a package manager for AI agent best practices — because knowing them and applying them are completely different problems
jeffreytse
jeffreytse

Posted on • Edited on

I built a package manager for AI agent best practices — because knowing them and applying them are completely different problems

#claude #ai #devtools #opensource

A few years ago I started doing something embarrassing: making a list of software engineering best practices I actually knew well enough to apply.

SOLID principles. Conventional commits. Blameless retrospectives. The Google SRE handbook. A handful of others.

The list was shorter than I expected.

Not because I was lazy. I'd read plenty. But reading about a practice and having it available when you need it — at 11pm debugging a production incident, or mid-PR-review when you're already context-switched — are completely different things. The gap between knowing a practice exists and applying it consistently in daily work is enormous, and most of us just quietly accept it.

Then I started wondering how many practices I'd never even heard of. Architectural decision records. Pre-mortems. The Five Whys. The ABA Model Rules for legal review. Different domains — engineering, finance, law, medicine, leadership — each have decades of accumulated wisdom. Most of it is documented, cited, and public. None of it is particularly accessible.

What I wanted was a master. Someone who knew every relevant practice, could surface the right one for my exact situation, and would hold me accountable for actually following it. That used to cost $800/hr. I decided to build it instead.

I called it grimoire 🧙 — a wizard's book of spells.

📦 What is grimoire?

Think npm, but for AI agent best practices.

You declare which skill libraries you want in a grimoire.toml. Grimoire clones them from a git registry and symlinks the individual skill files into each AI agent's context directory. Skills show up as slash commands in Claude, tab-completable commands in Codex, and equivalently in other agents.

Then grimoire check audits your project against those installed practices — like ESLint, but for how you work, not just how you code.

# Install grimoire
curl -fsSL https://raw.githubusercontent.com/jeffreytse/grimoire/main/scripts/install.sh | bash

# Guided setup
grimoire wizard

# Install skills to all detected agents
grimoire install

# Run your first compliance check
grimoire check
Enter fullscreen mode Exit fullscreen mode

That's the full loop.

🚀 From install to first wow

After grimoire install, skills appear immediately as slash commands. Here's what code review looks like after installing the engineering skill library:

User: /conduct-code-review

Claude: Reviewing staged changes against declared engineering standards...

  VERDICT: REQUEST CHANGES

  1. Missing test for the new token refresh path (auth/refresh.go:34)
     → coverage drops to 61%; your threshold is 80%

  2. Secret potentially exposed in error message (api/handler.go:89)
     → raw token value in error string; wrap before logging

  3. Breaking change undocumented
     → removeImplicitGrant() is public; needs BREAKING CHANGE footer

  File a review comment? [y/n]
Enter fullscreen mode Exit fullscreen mode

Not five bullet points. A structured verdict, with file locations and criteria, every time.

🧠 You don't need to know which skill to invoke

This is my favorite part. You don't need to memorize skill names. Just describe the situation and grimoire routes to the right practice:

User: My team keeps missing sprint goals and I don't know why.

Claude: Situation matches: plan-retrospective (engineering/project-management)

        Run a blameless retrospective (Amazon format):
        1. Timeline reconstruction — what actually happened
        2. Five whys — root causes, not symptoms
        3. Action items with owners and deadlines
        → /plan-retrospective to run the session now
Enter fullscreen mode Exit fullscreen mode 
User: We're launching in 48 hours and I'm terrified something will break.

Claude: Situation matches: apply-premortem (engineering/reliability)
        Followed by: design-slo → plan-incident-response

        → /apply-premortem to begin
Enter fullscreen mode Exit fullscreen mode

suggest-best-practice auto-classifies any situation and routes it. If the relevant skill isn't installed, it tells you exactly what to add to your grimoire.toml.

🗂️ How skills actually work

A skill is a single markdown file — not a prompt, more like a runbook:

---
name: conduct-code-review
tags: [engineering, review]
description: Run a structured code review against declared standards.
source: https://google.github.io/eng-practices/review/reviewer/
---

## Steps

1. Identify the scope — what problem does this change solve?
2. Check test coverage: new code paths must have tests.
3. Review naming: does it match established project conventions?
4. Check for security implications: auth, input validation, secrets.
5. Identify breaking changes — flag with BREAKING CHANGE if present.
6. Issue a verdict: APPROVE / REQUEST CHANGES / COMMENT.
Enter fullscreen mode Exit fullscreen mode

Every skill has a triggering condition, a cited source, numbered steps, and a verdict form. Not a suggestion — a procedure.

Here's the part I think is clever: the same file that teaches the AI how to do a code review is what grimoire check reads when auditing whether your project actually ran one. No separate compliance schema to maintain. Update the skill and both the guidance and the compliance criteria update in one step.

🔍 The compliance engine — grimoire check

grimoire check is where things get interesting.

grimoire check                     # full project audit
grimoire check --live              # watch mode: re-checks on every file save
grimoire check --scope changed     # incremental — changed files only (great for large repos)
grimoire check --junit report.xml  # JUnit XML for GitHub Actions / GitLab CI
Enter fullscreen mode Exit fullscreen mode

Result page:

Sample output:

$ grimoire check

  ✓  propose-conventional-commit   100%  all 4 criteria passing
  ✓  apply-solid                    88%  7/8 criteria passing
  ✗  conduct-code-review            62%  below threshold (80%)
     └─ missing: security-checklist, breaking-change-annotation

2 passed · 1 failed · exit 1
Enter fullscreen mode Exit fullscreen mode

You can gate CI on this. Declare a threshold in grimoire.toml:

[standards]
threshold = 80  # fail CI below 80% compliance
Enter fullscreen mode Exit fullscreen mode

The difference from other linters: ESLint checks whether your code is valid JavaScript. grimoire check checks whether your project actually follows the practices you declared. Did the last five commits use conventional commit format? Does the new auth module have adequate test coverage against your standards? Did the PR include a security checklist? Those are semantic questions — existing linters don't answer them.

🖥️ Editor integration (LSP)

grimoire lsp implements the Language Server Protocol. Point any LSP-capable editor at it and you get compliance diagnostics in the gutter on every file save.

Neovim:

if not configs.grimoire then
  configs.grimoire = {
    default_config = {
      cmd      = { 'grimoire', 'lsp' },
      filetypes = { 'go', 'python', 'javascript', 'typescript', 'rust' },
      root_dir  = lspconfig.util.root_pattern('grimoire.toml', '.git'),
    },
  }
end
lspconfig.grimoire.setup {}
Enter fullscreen mode Exit fullscreen mode

Helix:

[[language-server]]
name = "grimoire"
command = "grimoire"
args = ["lsp"]
Enter fullscreen mode Exit fullscreen mode

No plugin required beyond a one-time language client config. VSCode is supported too.

🤖 MCP server — let the AI manage itself

grimoire mcp serve exposes all grimoire operations as MCP tools. Claude Desktop, Cursor, or Windsurf can manage their own skill library mid-conversation:

grimoire mcp serve   # start the MCP server
grimoire mcp config  # write config file to your editor
Enter fullscreen mode Exit fullscreen mode

The AI can call grimoire_install, grimoire_update, grimoire_check as tool calls — detect a missing skill, install it, and immediately apply it, without you touching the terminal.

🌍 It's not just for engineering

This is the thing I keep having to say out loud: grimoire-core ships 1000+ skills across 27 domains.

Engineering. Architecture. Product management. Technical writing. Legal review. Financial analysis. Health. Leadership. Sales. Cooking.

The same infrastructure that enforces code review standards can enforce legal document review, financial due diligence, clinical protocol compliance, or onboarding documentation quality. The mechanism is identical — only the skills change.

That's the bigger bet: AI gives everyone comprehension. Grimoire gives everyone practice. The senior lawyer's review checklist, the experienced engineer's PR process, the McKinsey consultant's structuring framework — not proprietary, just previously inaccessible. Grimoire makes them installable.

📦 Declaring dependencies 

# grimoire.toml — commit this to your repo
[package]
name    = "my-project"
version = "0.1.0"

[dependencies]
"jeffreytse/grimoire-core"             = "*"   # all 1000+ skills
"jeffreytse/grimoire-core:engineering" = "*"   # one domain only
"jeffreytse/grimoire-core@0.1.0"       = "*"   # pinned release
"mycompany/internal-skills"            = "*"   # your team's private practices
Enter fullscreen mode Exit fullscreen mode

Any git repo is a valid package. Your company's internal engineering standards, a fintech-specific library, a jurisdiction-specific legal practices set — all install identically. Versions are locked in grimoire.lock (yes, it's exactly what you think it is).

✍️ Publish your own skills

If you've spent years mastering something, your practice belongs here.

# Validate your skill files
grimoire validate

# AI-powered auto-fix for conformance issues
grimoire validate --fix

# Preview before writing
grimoire validate --fix --dry-run
Enter fullscreen mode Exit fullscreen mode
  1. Write skill markdown files (name, tags, description, source, steps)
  2. Push to any git repo
  3. Others install with grimoire install yourgithub/yourpackage

Naming convention: yourorg/grimoire-fintech, yourorg/grimoire-legal-us, yourorg/grimoire-medical. Private repos work fine — install is identical.

🏁 Try it 

curl -fsSL https://raw.githubusercontent.com/jeffreytse/grimoire/main/scripts/install.sh | bash
grimoire wizard
grimoire check
Enter fullscreen mode Exit fullscreen mode

It's free, open source (MIT), and the wizard gets you to a first compliance check in about two minutes.

I'd love to know: what practices does your team currently enforce through system prompts or manual checklists? That's exactly the kind of thing that should probably be a grimoire skill.

Drop a comment or open an issue — especially if you want to contribute skills from your domain. 👇

Top comments (6)

Collapse
 
alexshev profile image
Alex Shev •

Turning agent practices into installable packages makes sense because “remember to do X” does not scale. The interesting part is versioning the practice itself: when a workflow changes, every agent using it should inherit the updated boundary.

Collapse
 
jeffreytse profile image
jeffreytse •

Spot on. The "versioning of practices" is actually where standard system prompts completely fail compared to a package manager approach.

In Grimoire, because practices are declared as dependencies in grimoire.toml and pinned in grimoire.lock, updating a boundary across a team or an fleet of agents is just a grimoire update away. If a core engineering standard or security checklist updates upstream, every agent inheriting that package gets the new guardrails instantly without needing to manually tweak system prompts across different IDEs or agent configs.

Glad that resonance hit—that exact problem was a huge driver for building it as a package ecosystem rather than static prompt files!

Collapse
 
alexshev profile image
Alex Shev •

The lockfile angle is what makes this feel real. Prompts drift invisibly, but a pinned practice can be reviewed, upgraded, rolled back, and audited. That makes the team boundary much clearer than “everyone remember to paste the new instructions.”

Thread Thread
 
jeffreytse profile image
jeffreytse •

That's exactly the frame. A practice upgrade becomes a PR: grimoire update --dry-run shows the diff before you commit, the lockfile change is reviewable, and if something breaks you can pin back to the previous version in grimoire.toml. Compare that to "I updated the system prompt, slack me if anything feels off."

The other thing it fixes is onboarding. New engineer clones the repo, runs grimoire install, and gets the same practice set everyone else has - not whatever version of the instructions was in someone's clipboard last Tuesday.

Collapse
 
eduzsh profile image
Edu Peralta •

I like the idea a lot, mainly because I have watched the opposite problem for months. A CLAUDE.md rule sits right there in the repo and the agent still skips it under time pressure, then writes a summary claiming it followed it anyway. My worry with grimoire check reusing the same file that teaches the practice is that exact gap, an agent can satisfy the letter of a markdown checklist without the diff actually reflecting it, so the audit needs to look at what changed in the code, not whether the right words got emitted somewhere in the session. Have you tested check against a case where the agent claims compliance but the actual diff does not back it up? That is the failure mode I would worry about most once you are past a thousand skills, where nobody is reading each one closely enough to notice the audit is checking vibes instead of code.

Collapse
 
jeffreytse profile image
jeffreytse • • Edited

You're describing the exact failure mode I've thought about most, and you're right that the body-as-rubric design doesn't solve it - it just shifts who's doing the checking.

The key structural difference from a CLAUDE.md rule: grimoire check is a separate AI invocation with no session memory of what the working agent claimed. It evaluates the current project state directly - not the session log, not the agent's self-report. So the "I followed it anyway" problem that plagues inline rules is partially addressed. But "partially" is doing real work in that sentence. An independent AI evaluating a diff can still be satisfied by good-looking code comments without verifying the actual behavior. It's probabilistic judgment, not a rule engine, and I try to be honest about that distinction even if "ESLint for practices" is a useful shorthand.

--scope changed helps somewhat - it grounds the check in the actual diff rather than ambient project state, which is closer to your "what changed in the code" framing.

The scaling concern is the one I don't have a clean answer to. At 1000+ skills, audit quality depends on the AI taking each criterion seriously, and that degrades. The honest direction is a deterministic layer for codifiable criteria - test coverage thresholds from CI, structural checks, commit format validation - layered under the AI judgment for the parts that genuinely require it. That's on the roadmap but not shipped yet.

Your instinct about diffs over words is the right one. If you want to dig into what a harder check mode would look like, open an issue - that's exactly the kind of input that shapes where this goes.