DEV Community

Cover image for I tried to trick my own AI-skill signing tool. Here's what happened.
Bharat Dudeja
Bharat Dudeja

Posted on

I tried to trick my own AI-skill signing tool. Here's what happened.

Over the last few months I’ve noticed a pattern emerging across AI tools.

Whether it’s Claude Skills, Cursor, Codex, or custom agent frameworks, we’re increasingly giving AI agents “skills”—packages containing instructions, documentation, and sometimes scripts.

The problem is…

A skill is usually just a Markdown file (plus some assets).

Nothing tells you:

  • Who created it.
  • Whether it has been modified.
  • Whether the version your AI is executing is the same one you reviewed yesterday.
  • Whether someone quietly injected new instructions into it.

As AI agents become capable of executing increasingly powerful workflows, that becomes a real supply-chain problem.

So I built Skillerr.

⸻

What is Skillerr?

Skillerr is an open-source protocol and CLI that adds trust and verification to AI skills before they’re executed.

Instead of treating a skill as “just another folder,” Skillerr treats it as a verifiable package.

It focuses on three things.

  1. Package Integrity

Every packaged skill receives a unique content-derived identifier along with cryptographic SHA-256 hashes.

If any file changes after packaging—even a single character—Skillerr detects it immediately.

No silent modifications.

⸻

  1. Structured Contracts

Instead of relying on long paragraphs that an AI has to interpret, a Skill contains a structured contract describing:

  • required inputs
  • permissions
  • forbidden actions
  • expected outputs
  • whether a human has actually reviewed it

This makes skills easier for both humans and AI agents to reason about.

⸻

  1. Optional Public Provenance

Authors can cryptographically sign their skills.

Optionally, the package digest can also be anchored into Sigstore’s transparency log, making it independently verifiable without trusting Skillerr itself.

Importantly:

Only cryptographic identifiers are published.

No prompts.
No documentation.
No knowledge base.
No proprietary content.

⸻

I tried to break my own tool

Before releasing it, I intentionally attacked it.

First I packaged and signed a simple CSV processing skill.

Then I:

  • unpacked the archive
  • edited one of the packaged files
  • inserted hidden instructions
  • repackaged everything

Skillerr immediately rejected it because the package hashes no longer matched.

Next I tried executing an unsigned package.

It refused by default.

Running untrusted skills requires an explicit opt-in rather than being the default behavior.

That felt like the right security model.

⸻

Why I built this

I don’t think AI agents should execute arbitrary instructions simply because they happen to live inside a Markdown file.

If we’re going to build ecosystems around reusable AI skills, we also need ways to answer questions like:

  • Can I trust this?
  • Who created it?
  • Has it changed?
  • Was it actually reviewed?
  • Can someone independently verify what I’m about to run?

That’s the problem Skillerr is trying to solve.

⸻

It’s open source

I’d genuinely love people to try breaking it.

If you’re already building AI skills—for Claude, Cursor, Codex, or your own framework—I’d love to know:

  • What does Skillerr miss?
  • What assumptions are wrong?
  • What security holes can you find?
  • What would make you trust it more?

Issues and pull requests are very welcome.

GitHub: https://github.com/dot-skill/skillerr

Protocol Package: @skillerr/protocol

Documentation: https://skillerr.com/docs

If you find a bug, please open an issue.

If you can’t break it, I’d like to hear that too.

Top comments (0)