Build a Tested Agent Skill with SKILL.md and Python Scripts

AI agents are good at interpreting goals, but prose instructions are a weak place to enforce exact rules. If a skill says "keep the commit subject short" or "never commit without approval," an agent can still misunderstand the boundary.

The open-source how-to-create-a-skill-tutorial shows a practical split: let the agent make judgments, and let small local scripts validate repeatable rules. This tutorial builds the smallest useful version of that pattern: a commit-crafter skill with a SKILL.md file, a Python validator, and tests that run with the Python standard library.

TL;DR

An Agent Skill is a directory containing at least SKILL.md. Put the workflow and safety boundaries in that file. Put exact validation in a script. Keep the script deterministic, return meaningful exit codes, and run it before presenting the result to a user.

The finished repository's example skill validates Conventional Commit messages. You can copy the same structure for release notes, config generation, research reports, or any other workflow with rules that can be checked mechanically.

Prerequisites

You need:

• Python 3.12 or newer for the repository's CI example.
• Git if you want the skill to inspect staged changes.
• An agent that supports the Agent Skills directory convention.
• A shell. The commands below use POSIX syntax; the files themselves are also designed for Windows.

The project has no stable release tag at the time of writing. The examples and commands below are checked against the current main branch. Read the Agent Skills specification if your client uses a different discovery directory.

1. Create the skill directory

The repository documents two useful scopes. A personal skill belongs in your user skills directory. A project skill belongs in the repository so a team can review and install it with the project.

mkdir -p .agents/skills/commit-crafter/scripts
mkdir -p .agents/skills/commit-crafter/references

The required layout is simple:

commit-crafter/
|-- SKILL.md
|-- scripts/
|   `-- check_message.py
`-- references/
    |-- conventional-commits.md
    `-- examples.md

Only SKILL.md is required. The scripts/, references/, and assets/ directories are conventions for executable code, on-demand documentation, and reusable resources.

2. Write the agent-facing contract

The frontmatter is the discovery contract. The name must match the directory name and use lowercase letters, numbers, and hyphens. The description should explain both the capability and the user vocabulary that should trigger it.

---
name: commit-crafter
description: Write git commit messages from staged changes following Conventional Commits. Use when the user asks to commit, asks for a commit message, wants to fix or improve a commit message, or mentions conventional commits.
---

# Commit Crafter

Write a commit message for the staged changes. The user reviews and approves
the message before anything is committed.

## Workflow

1. Run `git status` and `git diff --staged`.
2. If nothing is staged, report the changed files and ask what to include.
3. Classify the change and draft an imperative Conventional Commit message.
4. Run `python scripts/check_message.py --file message.txt`.
5. Fix violations and run the validator again.
6. Show the message. Commit only after explicit approval.

## Rules

- Never stage files yourself.
- Never commit without explicit approval.
- Never add an AI attribution footer unless asked.
- Read the linked reference files when the type or format is unclear.

Two boundaries are important here. First, the skill may inspect staged work but may not stage files. Second, drafting a message and executing git commit are separate actions. A good skill describes that separation directly.

3. Move exact rules into a script

The tutorial's design principle is: let the model decide, let the script verify, and let the script calculate. A model can choose whether a change is a feature or a fix. A script can consistently check the subject format, maximum length, blank-line separator, and breaking-change footer.

The example validator exposes a pure validate(message) function and a small CLI. It uses only the standard library, so there is no package installation step for the validation path.

def validate(message: str) -> list[Violation]:
    """Return deterministic violations for one commit message."""
    lines = message.splitlines()
    if not lines or not lines[0].strip():
        return [Violation("error", "empty", "Message has no subject line.")]

    subject, body = lines[0], lines[1:]
    violations = []
    match = SUBJECT_RE.match(subject)
    if not match:
        return [Violation("error", "format", "Invalid Conventional Commit subject.")]

    if len(match.group("subject")) > 72:
        violations.append(Violation("error", "subject-length", "Subject is too long."))
    if body and body[0].strip():
        violations.append(Violation("error", "body-separator", "Body needs a blank line."))
    return violations

The full implementation in check_message.py also checks allowed types, imperative mood warnings, breaking-change footers, and long body lines. The abbreviated code above illustrates the shape; use the repository file when you want the complete behavior.

Exit codes are part of the interface with the agent:

• 0 means the input passed.
• 1 means a validation or business-rule violation needs attention.
• 2 means there was no input.

Non-zero output should say what to fix. "Subject is 79 characters; maximum is 72" gives the agent a useful next action. "Validation failed" does not.

4. Test the script before installing the skill

The repository includes unit tests for the validator, including valid messages, malformed subjects, missing blank lines, breaking changes, warnings, and CLI exit codes.

python examples/commit-crafter/tests/test_check_message.py -v

On the current main checkout, this command completed with 16 tests passing. The repository's other deterministic examples also have standard-library test files for changelog generation and memory recording.

python examples/changelog-forge/tests/test_generate_changelog.py -v
python examples/lessons-keeper/tests/test_lessons.py -v

These tests are intentionally close to the scripts. A skill's instructions can drift, but a focused test keeps the mechanical contract visible in CI.

5. Install and exercise the trigger

Copy the skill directory into the discovery path used by your agent. For a personal installation, the repository documents this example:

cp -r examples/commit-crafter ~/.agents/skills/

Then try prompts that resemble real requests, not only the ideal wording:

commit this                         -> should trigger
write a good commit message         -> should trigger
reword my last commit               -> should trigger
what does the diff do?              -> should not trigger
create a git branch                 -> should not trigger

The description is the main trigger signal. If "reword my last commit" does not activate the skill, improve the description with the user's vocabulary before adding more body instructions.

Why this structure works

SKILL.md is loaded as instructions, so it is the right place for intent, sequencing, judgment, and safety boundaries. Reference files can hold detail that is only needed for a particular decision. Scripts are better for parsing, counting, rendering, and rejecting invalid output.

This is also a token-budget decision. A model does not need to reconstruct a line-count rule every time. It can run a small script and use the result. Deterministic scripts also make retries safer when they are idempotent and avoid network access unless the workflow truly requires it.

Failure modes and security boundaries

A skill is executable code with the permissions of the agent. Before installing a third-party skill, read its scripts. Treat curl | bash, unexpected credential-file reads, telemetry, and undeclared network calls as reasons to stop and investigate.

Common authoring failures include vague descriptions, a SKILL.md that is too large, scripts that silently succeed on invalid input, and instructions that blur drafting with side effects. Keep the main file focused, use actionable stderr messages, and require explicit approval before operations such as commits or publication.

The examples here are local and standard-library-first. They do not provide a security guarantee for the agent that executes them. Review compatibility requirements and permissions for your own client.

FAQ

Does every skill need a Python script?

No. Use a script when a stable rule is easier to enforce mechanically. A small skill that only provides judgment or reference material may need just SKILL.md.

Can I put all documentation in SKILL.md?

You can, but the specification recommends progressive disclosure. Keep the activated instructions concise and move detailed references into separate files that the agent reads when needed.

Does the repository ship a package?

No package installation is required for the example skills. They are folders that you copy into a supported skills directory. The tutorial site itself uses MkDocs Material, but that dependency is separate from running the example validators.

Takeaway

Start an Agent Skill with a narrow promise and an explicit boundary. Put the workflow in SKILL.md, put repeatable checks in deterministic scripts, and make tests exercise the same commands the agent will run. That gives you an installable skill that is easier to review, debug, and evolve.

What is one agent workflow in your project where a short validator would be more reliable than another paragraph of instructions?

AI assistance disclosure: This article was prepared with AI assistance using the repository's public documentation and source examples. The commands, project status, and test result were checked against the current public main branch before publication.