How to Create Agent Skills: Tools, Testing, and Installation

Maksym Danylenko — Thu, 01 Oct 2026 11:56:50 +0000

In short. Start with a task that keeps coming up and turn it into a Skill with superpowers:writing-skills. While creating it, test scenarios with and without the Skill, and validate the structure of SKILL.md. Then use the Skill on several real tasks. If it consistently helps, share it with your team through a Skills repository or a Plugin.

Creating your first Agent Skill is easy. Make a folder, put a SKILL.md file in it, and write a few instructions.

The harder part is knowing whether that Skill actually works.

The agent has to find it among dozens of other instructions, invoke it for the right request, ignore similar but irrelevant requests, follow its instructions, and produce a better result than it would without the Skill.

This guide covers tools for the full cycle: creation, improvement, validation, testing, installation, and maintenance.

What a Skill is in practice

An Agent Skill is a folder containing instructions, reference material, and, if needed, executable scripts. Only one file is required: SKILL.md.

A minimal Skill looks like this:

folder structure

review-migration/
└── SKILL.md

SKILL.md

---
name: review-migration
description: "Reviews database migrations for compatibility and rollback risks. Use when a user creates or changes a database migration."
---

Review the migration:

1. Check backward compatibility.
2. Identify locks or long-running operations.
3. Verify the rollback or roll-forward path.
4. Report only risks introduced by this migration.

The open Agent Skills specification requires the name and description fields. The description tells the agent when to read the full Skill for the task at hand.

That matters because of how Skills are loaded:

The agent sees the metadata for all available Skills.
The description helps it decide whether to read a particular Skill.
Once the Skill is activated, it reads the full SKILL.md.
It opens additional references/, scripts/, and assets/ only when needed.

Anthropic calls this progressive disclosure. It manages context: the agent doesn't need the full text of every Skill in every request. It reads instructions for the relevant task and opens longer reference material when it needs it.

Skill, AGENTS.md, or Plugin?

Not every instruction should become a Skill.

Mechanism	When to use it
AGENTS.md	Short, always-on context: project structure, working commands, non-obvious architecture decisions, and links to detailed rules
Skill	A repeatable process or knowledge needed only for a particular type of task
Plugin	A collection of Skills with agents, Hooks, MCP servers, configuration, or its own versioning lifecycle
MCP server	Standardized access to an external system, data, or API
Hook	A predefined action triggered by an event, such as running a formatter after an edit

Why Skills once started replacing MCP integrations, and what changed

Large MCP integrations used to load descriptions for every tool into each request. A few servers could occupy a substantial share of the context window. That led people to replace narrow integrations with Skills containing CLI commands: the agent would read the instructions only when it needed them.

Modern tool search with deferred loading has changed that trade-off. Descriptions of unused MCP tools no longer have to be included in the initial request: the agent can find the tool it needs and load its description on demand. That makes discovery more similar to Skills, though their roles remain different: a Skill provides a process and knowledge, while an MCP server performs an external action or returns data.

A good sign that you need a new Skill: you're pasting the same checklist into a chat or explaining the same process to a new session for the third time.

Tools for creating Skills

1. The agent itself

You don't need a separate generator. Anthropic's official authoring guide recommends completing a real task from start to finish in a normal agent session, then asking the agent to extract the repeatable process into a Skill.

That's a better starting point than inventing a universal Skill from scratch. Real work quickly shows you:

which context you keep repeating;
where the agent makes the wrong decision;
what belongs in instructions and what would work better as a script;
which examples and edge cases you need.

Your initial request can be simple:

prompt to the agent

Turn the process we just followed into an Agent Skill.
Keep only the required process in SKILL.md.
Move long reference material to references/.
Add examples of requests that should and should not trigger the Skill.

2. skill-creator or superpowers:writing-skills

Once you have a draft, you can use the official skill-creator. It helps create and edit Skills, build test scenarios, compare results against an initial run without the Skill, and perform a blind A/B comparison of two versions.

Install it from the official Marketplace:

Claude Code

/plugin install skill-creator@claude-plugins-official

After installation, you can ask the agent:

prompt to the agent

Evaluate my review-migration Skill using skill-creator.

For creating Skills, I recommend Codex and superpowers:writing-skills. Its principle is stricter: first give the agent a real scenario without the Skill and record a specific failure; then write the smallest useful Skill, repeat the same scenario, and check whether its behavior changed. If the agent consistently completes the task correctly without the Skill, a new Skill may have no purpose.

That's enough for an early version. You don't need a CI system for an instruction you haven't even used twice.

Three different kinds of checks

The word "validation" often hides three different questions.

Superpowers:writing-skills adds a preliminary step: run a scenario without the Skill, put the agent under pressure, and observe how it behaves without extra instructions. This tests the premise itself. If the baseline run already succeeds, the Skill adds nothing; if it fails, you can see the specific behavior you need to change.

1. Is the structure valid?

For a portable Skill, use the validator for the open specification:

terminal

$skills-ref validate ./review-migration

It checks the frontmatter and naming rules.

In Claude Code, you can validate project Skills like this:

terminal

$claude plugin validate .claude/skills

Or validate all user-level customizations:

terminal

$claude plugin validate ~/.claude

According to the Claude Code documentation, the validator finds problems with YAML, metadata, and Plugin structure. This checks syntax and schema. A successful result doesn't mean the agent will choose the Skill at the right time.

2. Does the Skill trigger?

Give this check to superpowers:writing-skills while creating the Skill. Include real requests that should trigger it and similar-sounding requests that should not. superpowers:writing-skills will run the scenarios and show whether the agent finds the instructions at the right moment.

If the Skill doesn't trigger, or triggers when it shouldn't, refine its description and repeat the check. If it triggers but doesn't help, revisit the main instructions and supporting files.

3. Did the result improve?

For a Skill packaged as a Plugin, Claude Code has a separate testing tool:

terminal

$claude plugin eval init
$claude plugin eval .

Claude plugin eval runs each scenario in an isolated session several times with and without the Plugin. Graders can check text, tool calls, action order, created files, or criteria assessed by an LLM. The difference between WITH and W/OUT shows the Plugin's contribution, rather than just the base model's ability to complete the task.

This is useful for Skills that a team versions, shares, or uses in a critical process. The runs make real model calls. By default, each scenario runs three times with the Plugin and three times without it, so a suite costs both time and money.

One detail matters: skill-creator and claude plugin eval use different test formats. The former is useful for improving one Skill through a conversation. The latter is better suited to Plugins, regression suites, and CI.

What /skill-doctor tells you

/skill-doctor answers a different question: which Skills consume context but are rarely used?

It shows how much context descriptions occupy, which Skills have never been invoked, and which Plugins haven't been used for a while. That helps remove noise from a large library. But /skill-doctor doesn't assess instruction quality or replace behavior checks.

How to install a Skill in a project

The simplest option is to add it manually:

terminal

$mkdir -p .claude/skills/review-migration

Then create .claude/skills/review-migration/SKILL.md and commit the folder. The Skill will load in sessions for that repository.

For a personal Skill that you need across all your local projects:

path

~/.claude/skills/review-migration/SKILL.md

In a monorepo, you can put a Skill in <package>/.claude/skills/. It will apply to work in that subtree.

If a Skill is published in a Git repository and supports the open ecosystem:

terminal

$npx skills add owner/repository --skill review-migration -a claude-code

Add -g to install it globally instead of in the current project.

When you need a Plugin and Marketplace

A standalone Skill works well within one repository. Use a Plugin when you want to:

distribute several Skills as one package;
add agents, Hooks, or MCP configuration;
have a namespace such as /database-tools:review-migration;
version and update the package;
attach a formal test suite.

A minimal Plugin structure:

folder structure

database-tools/
├── .claude-plugin/
│   └── plugin.json
└── skills/
    └── review-migration/
        └── SKILL.md

You can test the Plugin locally with:

terminal

$claude --plugin-dir ./database-tools
$claude plugin validate ./database-tools

To share it, a team can create a private Marketplace, add it at the project level, and install the Plugin from there. The official Marketplace documentation supports a GitHub repository, Git URL, remote marketplace.json, and local path.

Claude Code

/plugin marketplace add acme/database-plugins
/plugin install database-tools@acme-database-plugins

The CLI equivalent supports user, project, and local scopes. At the project level, configuration is written to the repository so the team shares one Plugin source.

Check third-party Skills before installing them

There has never been an easier way for agent instructions to spread widely. One repository and one install command can put someone else's SKILL.md, scripts, or entire Plugin into hundreds of working environments.

A Skill gives instructions to an agent with access to files and tools. A Plugin can also contain Hooks, MCP servers, and executable scripts. Popularity in a catalog doesn't automatically make that code safe.

Before installing, check:

the contents of SKILL.md and every reference to scripts/;
allowed-tools and shell commands;
Hooks and MCP configuration inside the Plugin;
whether the Skill requests secrets, unnecessary permissions, or network access;
the repository owner, license, change history, and update process.

For a team library, pin a reviewed version, run validation in CI, and update Skills as deliberately as you update other dependencies.

A practical process without extra infrastructure

For your first Skill:

Complete a real task without the Skill.
Record the baseline result: exactly where the agent failed or behaved inconsistently.
Ask the agent to extract the repeatable process.
Write three requests that should trigger the Skill and three similar but irrelevant requests.
Put the Skill in .claude/skills.
Validate its structure.
Repeat the baseline scenario in a new session with the Skill.

For a Skill used by a team:

Add more real scenarios.
Move unstable steps into deterministic scripts.
Package the Skill as a Plugin.
Add claude plugin eval.
Run the suite after Skill changes and after switching to a new model.
Periodically check /skill-doctor and disable what you don't use.

The main mistake: testing the file instead of behavior

The weakest success criterion is: "The agent read the Skill, and its answer looks fine."

Better questions are:

Did the Skill trigger without its name being mentioned?
Did it stay quiet for a similar but irrelevant request?
Did it improve the result compared with the baseline run?
Did it reduce repeated explanations?
Can you reproduce the result in a new session?
Does the new version fix a specific failure without breaking existing scenarios?

A Skill isn't a document you write once and polish. It's an executable part of the agent workflow. Treat it like code: keep it short, test it on real scenarios, version it, and don't confuse valid syntax with the right behavior.

Try it in practice. We've published a set of Agent Skills for generating effective unit tests. Install it in your project and try it on real code.

Try the unit testing Skills

In the next article, I'll show how we tested these Skills, what baseline we compared them with, and what we found during testing.

Sources:

Originally published on Mavka.

Unit tests aren't useless. Bad unit tests are.

Maksym Danylenko — Tue, 29 Sep 2026 13:34:29 +0000

"Unit tests are worse than useless" got 327K views on X last week:

// Detect dark theme var iframe = document.getElementById('tweet-2102743849106014578-915'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2102743849106014578&theme=dark" }

I get it. Most AI-written unit tests restate the code, always pass, and catch nothing. They break on every refactor and never find a real bug.

But the problem is bad tests, not unit tests.

A study of 198 production failures in Cassandra, HBase, HDFS, MapReduce and Redis (Yuan et al., OSDI '14) found that 77% could be reproduced by a unit test. Dropping unit tests for E2E-only throws that away.

You can let the agent write junk tests and then pay again to delete them. Or teach it to write tests that catch bugs from the start. We use unit tests skills for this: rules the agent loads when it writes tests.

Here are the ones we built: https://github.com/mavka-ai/unit-tests-skills

DEV Community: Maksym Danylenko

How to Create Agent Skills: Tools, Testing, and Installation

What a Skill is in practice

Skill, AGENTS.md, or Plugin?

Tools for creating Skills

1. The agent itself

2. skill-creator or superpowers:writing-skills

Three different kinds of checks

1. Is the structure valid?

2. Does the Skill trigger?

3. Did the result improve?

What /skill-doctor tells you

How to install a Skill in a project

When you need a Plugin and Marketplace

Check third-party Skills before installing them

A practical process without extra infrastructure

The main mistake: testing the file instead of behavior

Unit tests aren't useless. Bad unit tests are.