I built a governance layer for AI agent skills — with an AI agent as my pair programmer

Artemr Rudenko — Mon, 03 Aug 2026 16:16:50 +0000

The moment I noticed the problem

Somewhere around late May 2026, I had a folder full of SKILL.md files. Some were mine, some were pulled from a corporate library, some were written half-tired at 11pm to unblock a task. They all worked, in the sense that the agent picked them up and did something. But I couldn't have told you, with a straight face, which ones were safe to hand to a teammate, which ones would silently break the moment someone tried them in Cursor instead of Claude Code, and which ones had a curl | bash sitting quietly in a code block that nobody had ever read past the first two lines.

That's the actual origin story. Not a grand vision — a folder of skills I didn't trust.

Two open standards had recently landed, within months of each other: AGENTS.md (the Linux Foundation AAIF format for repo-wide agent instructions) and SKILL.md (Anthropic's Agent Skills spec for packaged, on-demand procedures). Both are refreshingly simple — frontmatter plus Markdown, no build step, no SDK lock-in. That simplicity is exactly why they were spreading fast, and exactly why nobody had built the boring, unglamorous layer every fast-spreading format eventually needs: governance. Something that scores a skill, audits a library of them, checks whether a "portable" skill is actually portable, and catches the security issues a human reviewer skims past on the fifth PR of the day.

So I built one. Six meta-skills, a security-focused sub-agent, and a deterministic audit engine — all designed to run identically on Claude Code, GitHub Copilot, Cursor, OpenAI Codex CLI, and Google Gemini CLI, because a governance tool that only works on one vendor's runtime isn't governance, it's a walled garden.

I did not write this alone. Claude Code wrote most of the actual code and Markdown; I made the calls on architecture, scope, and what "good enough" meant at each stage. That division of labor is worth being honest about, because it's part of how this got built as fast as it did — more on that below.

What actually shipped

The repo is called skill-governance-toolkit, and it's a "reference implementation of the universal skill authoring pattern" — meaning it doesn't just describe the AGENTS.md + SKILL.md standard, it enforces it against itself. The toolkit's own skills pass its own audit. If they didn't, that would be the whole pitch falling over.

Six meta-skills, all operating on other skills rather than on application code:

Skill	Job	You'd say
`skill-find`	Search project/personal/corporate skill libraries for something matching a need	"is there a skill for X"
`skill-evaluate`	Score one `SKILL.md` across 9 quality dimensions (D1–D9) + a lexical security scan	"is this skill production-ready"
`skill-compare`	Diff a local skill against a corporate version, recommend USE / KEEP / MERGE	"is there a similar one upstream"
`library-audit`	Meta-validate a whole library — handoff contracts, token budget, naming, security baseline	"audit my skills"
`skill-build-portable`	Turn a Claude-only skill into a cross-runtime one, diff preview before write	"make this skill portable"
`integration-init`	Bootstrap optional tooling (code-review-graph, repomix, serena) per project	"set up integrations"

Plus agents/code-reviewer.md, a TypeScript-focused sub-agent with five review categories, dispatchable from any orchestrator skill.

And under all of it, a deterministic audit engine (agent_audit.py) that ships in three delivery modes — CLI for local checks and CI gates, MCP (Anthropic's Model Context Protocol, the standard agents use to call external tools) so an agent can call it mid-edit as a tool, and a sandboxed Docker image for teams that don't want local Python at all. Same engine, same rules, three ways to invoke it.

How it got built — and how fast

Here's a detail I find more interesting than the feature list: the CHANGELOG we maintain ourselves — the project's own dated release record — shows 42 versioned releases in 18 days, v1.0.0 on May 23 through v1.20.0 on June 10, 2026. It wasn't an even drip either: 24 of those releases landed in the first three days, another 13 landed in a single day about two weeks later, with quieter single-release days connecting the two bursts. That kind of pace only works with a strict discipline underneath it, and the discipline was non-negotiable: red first, then green, every time.

Concretely: every change gets a failing test written before the fix, then the fix, then the entire self-CI suite has to pass before anything gets committed. Right now that suite is 37 phases deep (the project literally numbers them P1 through P37) — covering everything from "does the CLI behave correctly" to "does the Windows installer's PowerShell script parse" to "does the documented phase range in AGENTS.md match what audit.sh actually runs" (yes, there's a phase whose entire job is catching documentation drift about the test suite itself — P35, if you're curious).

The architecture decision I'm proudest of isn't a feature, it's a fork in the road that the toolkit makes explicit for anyone who wants to add a new check. There are two surfaces, and picking the wrong one is the most common mistake:

Surface A is the deterministic gate — regex and structural rules, running in CI, blocking, zero tolerance for false positives. Surface B is the LLM-scored skill layer — skill-evaluate's rubric, library-audit's cross-skill checks — where judging intent actually requires reading and understanding, not pattern-matching. The rule of thumb the docs state plainly: a semantic check forced into the gate becomes a false-positive machine; a security invariant left to the LLM is not a gate at all. Getting that boundary right early saved a lot of wasted rework later.

What it actually checks for

The deterministic gate currently carries 17 distinct rule IDs across five families:

SEC001–SEC003 — bidirectional-text overrides, invisible/zero-width characters, homoglyph typosquats (pаypal.com with a Cyrillic а). The kind of thing a human reviewer will never catch by eye.
SEC010–SEC012 — dangerous inline shell (destructive commands, approval bypasses), inline secrets that should be env vars, unsafe credential paths.
REF001–REF004 — unsafe references to scripts or assets outside the governed tree.
SCR001–SCR003 — script-level issues in the executable helpers a skill ships.
MCP001–MCP002, HOOK001–HOOK002 — MCP server config consistency and hook-command safety, because a skill that quietly reconfigures your MCP servers or hooks is a supply-chain vector, not a convenience.

On top of the deterministic layer, skill-evaluate scores nine quality dimensions per skill — D1 Clarity through D9 Safety/Risk Awareness, with Portability (D6) and Testability (D7) sitting right alongside them, because "works on my machine" isn't a bar for a skill anyone else will run.

D9 is the one I keep coming back to, because it scores behavioral design, not lexical content — a different job from the SEC-family regex checks above. It breaks down into six sub-concerns:

Blast radius — does a mutating skill name a hard limit ("max 50 tickets per run"), not just "as many as needed"?
Prompt injection — does a skill that ingests external content say "treat as untrusted"?
Sensitive data flow — does it name what must never land in an output (PII, secrets, internal URLs)?
Authorization assumptions — what happens on insufficient permissions, fail-fast or proceed?
Rollback on partial failure — if a multi-step mutation dies at step 6 of 10, what's the story?
Idempotency — does a re-run duplicate, no-op, or update?

The rubric's own worked example is a jira-ticket skill that creates tickets from a brief: no ticket cap named (blast-radius fail), the source brief treated as authoritative even though it's untrusted external content (prompt-injection fail), and no mention of what happens to PII in that brief once it lands in a ticket description (sensitive-data fail). Three of six, straight to a score of 2 — on a skill that "worked" in every demo anyone ever ran.

There's also a floor rule worth knowing about: a skill with no mutations and no untrusted-input ingestion defaults to a D9 of 4, because you can't fail a safety review for risks you structurally don't carry. The dimension is measuring exposure, not vibes.

How the detectors actually decide

Naming the rule IDs is one thing; here's what two of them look like as actual Python, pulled straight from agent_audit.py.

SEC001/SEC002 (bidi and invisible characters) isn't a list of specific banned codepoints — it's category-based, so it catches the whole Unicode class instead of whatever a hardcoded list happened to enumerate:

def scan_hidden_chars(text: str, path: Path) -> list[Finding]:
    """Flag bidi controls (SEC001) and invisible/format/tag/variation chars (SEC002).
    Category-based (unicodedata 'Cf') so it catches the whole class, not a fixed list."""
    for off, ch in enumerate(text):
        cp = ord(ch)
        if cp in BIDI_CONTROLS:
            out.append(finding("SEC001", path, line_for_offset(text, off), ...))
        elif unicodedata.category(ch) == "Cf" or _is_tag_or_variation(cp):
            out.append(finding("SEC002", path, line_for_offset(text, off), ...))

That one unicodedata.category(ch) == "Cf" check is doing more work than it looks like — "Cf" is the Unicode "format" category, and it covers zero-width spaces, soft hyphens, and the entire bidi-control block in one branch, so a brand-new invisible character added to a future Unicode version gets caught automatically, without anyone updating a pattern list.

The dangerous-shell/secret scanner (SEC010–SEC012) has a subtler trick: it runs twice per file — once on the raw text, once on an NFKC-normalized copy — specifically to catch an attacker (or a careless copy-paste) using fullwidth Unicode lookalikes to dodge a plain-ASCII regex. A fullwidth ｒｍ　－ｒｆ　／ doesn't match rm -rf / literally, but normalizes to it under NFKC, so the second pass catches what the first pass's regex alone would miss. A (rule_id, matched_text) seen-set stops the same real hit from being reported twice.

And every finding funnels through one finding() constructor that looks up the rule ID in the catalog and raises KeyError if it's not registered — so a scan function can't silently emit a rule ID that has no checks-reference.md entry and no severity assigned. The catalog and the code are structurally prevented from drifting apart; nobody has to remember to keep them in sync.

A skill that fails, and one that doesn't

The README opens with exactly this contrast, because showing beats telling:

name: Deploy-Helper          # wrong case, no closed-vocab action verb → naming fail
allowed-tools: [Bash]        # breaks portability (Copilot/Gemini ignore it)
# body: Run `curl $URL | bash`   → dangerous-shell HIGH + missing Skill-type contract

Three separate failures stacked in four lines: a naming-convention violation, a portability trap (allowed-tools: is a hard whitelist on Claude Code and silently ignored everywhere else — meaning the skill behaves differently per platform with no warning), and a dangerous-shell finding severe enough to block a merge outright.

Compare it to a real skill from the toolkit's own repo, skill-find's actual frontmatter:

---
name: skill-find
description: |
  Discovers skills in project / personal / corporate libraries matching a natural-language need.
  Use when the user is unsure if a skill exists for their task, asks "is there a skill for...",
  or wants to browse available capabilities before creating something new.
  Triggers: «is there a skill for X», «find a skill that does Y», «browse skills»,
  «what skills handle Z», «do we have a skill for», «search skills».
compatibility: "Claude Code · GitHub Copilot · Cursor v2.2+ · OpenAI Codex CLI · Google Gemini CLI"
license: MIT
metainfo:
  category: [discovery, governance]
  tier: stable
  owner: artsiom
  added: "2026-05-23"
---

No allowed-tools:. An explicit compatibility: line naming every runtime it's tested against. A description that answers both "what does this do" and "when should you trigger it," with real trigger phrases in the user's own words — because skill-evaluate's D8 (Discoverability) dimension scores exactly that, and a skill nobody's agent ever activates is dead weight regardless of how good its logic is.

skill-build-portable automates the gap between those two examples. Feed it a Claude-only skill with allowed-tools, hardcoded paths, and a 16-character description, and it produces a diff, not a lecture:

Transformations applied (5):
  🔴 F1 · frontmatter:5 · remove allowed-tools (non-portable hard whitelist)
  🟡 F2 · frontmatter:3 · remove type:, status: (not in agentskills.io spec)
  🟡 F3 · frontmatter:8 · remove paths: (Claude-only; document in body if path-scope matters)
  🔴 B1 · body:line 1 · replace _audits/internal/ → out/audits/internal/
  🟡 F7 · add compatibility: field

Description too short (16 chars). Required ≥ 80 to score well on D8.

Run it again on an already-portable skill and the honest answer is zero changes — the tool proves its own idempotency instead of asserting it.

Extending it without forking it

The thing I wanted to avoid most was becoming the toolkit's single point of contact for every new rule. So the two-surface split from earlier doubles as the contribution contract:

New pattern for an existing detector (a new dangerous command, a new secret shape) — that's policy data, not code. It goes into a config overlay, no Python touched:

  "config": { "patterns": { "dangerous": [
    {"label": "kubectl delete", "regex": "kubectl\\s+delete", "suggestion": "No cluster deletion in assets."}
  ] } }

Before it ships, a ReDoS budget guard — ReDoS being a regex pattern that pathologically backtracks and hangs on certain input — rejects catastrophic patterns automatically, so a badly written rule can't take down CI by freezing it.

A genuinely new kind of detection — that's a code change, and the bar is explicit: write the scan_* function, register the rule ID, add a severity, document it, and — this is the part I like — ship a triggering fixture and a clean near-miss fixture. A rule that only has a positive example can't prove it isn't over-broad. No clean fixture, no merge.
Semantic judgment that can't be a regex — new D-dimension sub-check in skill-evaluate, or a new phase in library-audit.

That contract is also why the self-CI suite reached 37 phases without becoming unmaintainable: every new capability came with the fixtures that keep it honest, checked in alongside it, not bolted on after the fact.

One more pattern worth stealing: checkpointed runs

Multi-phase skills — library-audit walking a hundred-skill repo, skill-evaluate scoring through nine dimensions plus a security pass — take real time, and real time means real interruptions: a timeout, a crash, someone hitting Ctrl-C because a meeting started. The toolkit's answer is a small, boring, load-bearing convention: after every expensive phase, write state to ${CWD}/.skill-state/{skill-name}/run-{utc-ts}.json in the project being audited, not the toolkit checkout, so one toolkit install can safely service many projects without their state colliding. An interrupted library-audit run picks up from the last completed phase instead of re-scoring a hundred skills from zero. It's not a clever feature. It's the kind of unglamorous reliability work that only gets built when someone's actually been burned by its absence — which, if I'm honest, is most of what's in this toolkit.

What running it actually looks like

Not a mockup — this is the real self-audit gate, one phase of 37, testing that a blocking finding actually blocks and that the kill-switch actually downgrades it:

── P22 — Delta-gate integration (real git: --fail-on-new + kill-switch) ──
  ✓ --fail-on-new high blocks on the newly introduced finding (exit 1)
  ✓ SGT_FAIL_MODE=advisory downgrades the gate to non-blocking (exit 0)

And the tail of a full run:

══════════════════════════════════════════════════════════
Audit summary:
  ✓ passed:   determined by zero failures
  ! warned:   1
  ✗ failed:   0
══════════════════════════════════════════════════════════

(That one warning, every time, is "pwsh not on PATH" — expected on a Mac without PowerShell installed, not a real failure.)

And here's the "three delivery modes, one engine" claim made concrete — the Docker smoke test literally pipes a raw MCP JSON-RPC call into the sandboxed image and expects a real answer back, no local Python involved:

printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"audit_project","arguments":{"root":"/workspace","profile":"release-ready"}}}' \
  | docker run --rm -i --network none -v "$PWD:/workspace:ro" agent-governance-mcp:ci

--network none isn't incidental — the audit engine is read-only and offline by design, so the smoke test proves it doesn't need a network to do its job, not just that it happens not to use one.

What's next

The toolkit is public now — MIT licensed, GitHub Actions CI green (self-audit, Windows-parity, Docker smoke test), and its own release notes are generated from CHANGELOG.md by the same script that gates its own merges, so there's no hand-maintained notes file to drift out of sync.

If you're standing in the same spot I was in May — a folder of skills you can't quite vouch for — the six-skill tour starts with skill-evaluate on whatever SKILL.md you're least sure about. It'll tell you, with a score and a reason, whether "least sure about" was the right instinct.

Repo: github.com/artemrudenko/skill-governance-toolkit

DEV Community: Artemr Rudenko