How to Create Claude Code Skills Automatically with Skill Creator

TL;DR

Claude Code Skills are custom capabilities that extend Claude’s functionality for specific workflows. Use the Skill Creator workflow to define a skill, write SKILL.md, create realistic test cases, benchmark results with and without the skill, iterate from feedback, optimize triggering, and package the result.

Try Apidog today

Introduction

If you repeatedly ask Claude Code to set up the same project structure, run the same test commands, or format reports the same way, turn that workflow into a Skill.

A Claude Code Skill is a reusable instruction package for a focused workflow. Skill Creator provides a systematic way to build one: define the outcome, write instructions, evaluate the result, improve the instructions, and package the final Skill.

This guide covers:

• The structure of a Claude Code Skill
• A repeatable Skill Creator workflow
• Test cases, assertions, and benchmarks
• Trigger-description optimization
• Packaging and sharing a Skill

If you are building API-related skills, Apidog fits naturally into the workflow for testing endpoints, validating responses, and generating documentation.

What Are Claude Code Skills?

Claude Code Skills are specialized instruction sets that extend Claude’s capabilities for specific domains or workflows. They are Markdown-based packages that can include instructions, scripts, references, and assets.

Skill system architecture

Skills use three loading levels:

1. Metadata (~100 words): name and description, always available in context.
2. SKILL.md body (<500 lines): core instructions, loaded when the Skill triggers.
3. Bundled resources (unlimited): scripts, references, and assets loaded only when needed.

skill-name/
├── SKILL.md
│   ├── YAML frontmatter (name, description)
│   └── Markdown instructions
└── Bundled Resources
    ├── scripts/    # Executable code for repeated tasks
    ├── references/ # Documentation loaded as needed
    └── assets/     # Templates, icons, fonts

When Skills trigger

Claude sees Skills through its available_skills list and uses the Skill description to decide whether a Skill applies.

A Skill is most reliable for complex, multi-step tasks. Simple requests such as “read this file” may not trigger a Skill even if its description contains matching words.

Write descriptions around:

• The workflow the Skill handles
• User phrases that indicate the workflow
• Expected inputs and outcomes
• Adjacent requests that should not trigger the Skill

Examples from Anthropic’s repository

Skill	Purpose	Key features
skill-creator	Create new Skills	Test-case generation, benchmark evaluation, description optimization
mcp-builder	Build MCP servers	Python/Node templates, evaluation framework, best practices
docx	Generate Word documents	python-docx scripts, templates, styling guidance
pdf	Extract and manipulate PDFs	Form handling, text extraction, reference documentation
frontend-design	Build web interfaces	Component libraries, Tailwind patterns, accessibility checks

The Skill Creation Workflow

Use this loop when creating or improving a Skill:

1. Capture intent
2. Draft SKILL.md
3. Create test cases
4. Run baseline and Skill-enabled evaluations
5. Review qualitative output and quantitative metrics
6. Improve the Skill
7. Optimize the description for triggering
8. Package the Skill as a .skill file

Step 1: Capture Intent

Start with a workflow, not a generic topic.

Ask these questions before writing instructions:

1. What should the Skill enable Claude to do? Define a concrete outcome.
2. When should it trigger? List user wording, contexts, and inputs.
3. What should it produce? Define files, code, reports, or other artifacts.
4. Can you verify the output objectively? If yes, create test cases and assertions.

Example: API testing Skill

Field	Definition
Intent	Help developers test REST APIs systematically
Trigger	API testing, endpoints, REST, GraphQL, response validation
Output	Test reports, pass/fail status, curl commands, response comparisons
Test cases	Yes, because the output is objectively verifiable

Keep the scope narrow enough that the Skill has a clear job. A Skill that “works with APIs” is ambiguous; a Skill that “generates a repeatable REST endpoint test plan and executable requests” is actionable.

Step 2: Write SKILL.md

Every Skill needs a SKILL.md file with YAML frontmatter followed by Markdown instructions.

Start with a minimal Skill

---
name: api-tester
description: How to test REST APIs systematically. Use when users mention API testing, endpoints, REST, GraphQL, or want to validate API responses. Make sure to suggest this skill whenever testing is involved.
compatibility: Requires curl or HTTP client tools
---

# API Tester Skill

## Core Workflow

When testing an API:

1. **Understand the endpoint**: Read the specification or request the schema.
2. **Design test cases**: Cover happy paths, edge cases, and error conditions.
3. **Execute tests**: Use `curl` or Apidog for requests.
4. **Validate responses**: Check status codes, headers, and body structure.
5. **Report results**: Summarize pass/fail status with evidence.

## Test Case Template

For each endpoint, test:

- Valid authentication with a correct payload
- Missing required fields
- Invalid authentication (`401` expected)
- Rate-limiting behavior
- Response time under load

## Output Format

# API Test Report

## Summary
- Tests run: X
- Passed: Y
- Failed: Z

## Failed Tests

### Test Name
**Expected:** 200 OK  
**Actual:** 400 Bad Request  
**Response:** `{...}`

## Recommendations
...

Keep SKILL.md focused

Use progressive disclosure:

api-tester/
├── SKILL.md
└── references/
    ├── authentication.md
    ├── rate-limiting.md
    └── response-codes.md

Keep the reusable workflow in SKILL.md. Move lengthy documentation, protocol details, and large examples to references/.

Write instructions that explain intent

Avoid instructions that only list rules. Explain why the rule exists so the model can adapt it correctly.

Instead of:

Always validate the status code first.

Use:

Validate the status code before analyzing the response body. A failed authentication or authorization response can make happy-path body validation meaningless.

Use imperative language, but avoid overly rigid wording when context may require adaptation.

Include concrete examples

## Commit message format

**Input:** Added user authentication with JWT tokens  
**Output:** `feat(auth): implement JWT-based authentication`

Examples make output expectations easier to follow than abstract instructions alone.

Step 3: Create Test Cases

Create two or three realistic prompts before running evaluations. These should resemble requests developers would actually send.

Save them in evals/evals.json:

{
  "skill_name": "api-tester",
  "evals": [
    {
      "id": 1,
      "prompt": "Test the /users endpoint on api.example.com - it needs a Bearer token and returns a list of users with id, name, email fields",
      "expected_output": "Test report with at least 5 test cases including auth failure, success, and pagination tests",
      "files": []
    },
    {
      "id": 2,
      "prompt": "I need to verify our new POST /orders endpoint handles invalid quantities correctly",
      "expected_output": "Test cases that send negative, zero, and non-numeric quantities with appropriate error responses",
      "files": ["openapi.yaml"]
    }
  ]
}

Write prompts with implementation details

Avoid vague test prompts:

Test this API

Use prompts that include endpoint context, a scenario, and expected behavior:

My team deployed a new payments endpoint at https://api.stripe.com/v1/charges.
I need to verify edge cases: negative amounts and invalid currency codes.
The documentation says both should return 400, but I want to inspect the actual error messages.

A strong evaluation prompt includes:

• A specific URL or endpoint
• A concrete failure mode or use case
• Expected behavior
• Realistic context and constraints

Before running the evaluation, share the test scenarios with the user:

Here are the test scenarios I plan to run. Do these match the workflow, or should we add another case?

Step 4: Run Evaluations

Run each test case twice:

• With the Skill
• Without the Skill, or with the previous version when iterating

This comparison shows whether the Skill improves output quality enough to justify its time and token cost.

Use a predictable workspace structure

Store results in a sibling workspace directory:

api-tester-workspace/
├── iteration-1/
│   ├── eval-0-auth-failure/
│   │   ├── with_skill/
│   │   │   ├── outputs/
│   │   │   └── timing.json
│   │   ├── without_skill/
│   │   │   ├── outputs/
│   │   │   └── timing.json
│   │   └── eval_metadata.json
│   ├── eval-1-pagination/
│   │   └── ...
│   ├── benchmark.json
│   └── benchmark.md
├── iteration-2/
└── feedback.json

Launch parallel runs

For each test case, run Skill-enabled and baseline executions in the same turn.

With-Skill run:

Execute this task:
- Skill path: /path/to/api-tester
- Task: Test the /users endpoint on api.example.com
- Input files: none
- Save outputs to: api-tester-workspace/iteration-1/eval-0/with_skill/outputs/

Baseline run:

Execute this task:
- Skill path: (none)
- Task: Test the /users endpoint on api.example.com
- Input files: none
- Save outputs to: api-tester-workspace/iteration-1/eval-0/without_skill/outputs/

Capture timing and token data

When each run completes, save the execution data immediately:

{
  "total_tokens": 84852,
  "duration_ms": 23332,
  "total_duration_seconds": 23.3
}

Write this to the run’s timing.json. This information is provided through task notifications, so capture it as each notification arrives.

Step 5: Draft Assertions While Evaluations Run

Do not wait for every run to finish before defining your success criteria.

Write assertions that are:

• Objective: unambiguous pass/fail criteria
• Descriptive: names explain what is checked
• Reusable: applicable across later iterations

Example assertions for an API testing Skill:

{
  "assertions": [
    {
      "name": "includes_auth_failure_test",
      "description": "Test report includes at least one authentication failure test case",
      "type": "contains",
      "value": "401"
    },
    {
      "name": "includes_success_test",
      "description": "Test report includes at least one successful request test",
      "type": "contains",
      "value": "200"
    },
    {
      "name": "includes_curl_commands",
      "description": "Each test case includes executable curl commands",
      "type": "regex",
      "value": "curl -"
    },
    {
      "name": "includes_response_validation",
      "description": "Report validates response structure against schema",
      "type": "contains",
      "value": "schema"
    }
  ]
}

Add these assertions to both eval_metadata.json and evals/evals.json.

Step 6: Grade and Aggregate Results

After every run completes, grade outputs against the assertions and aggregate the results.

Grade each run

Use a grader subagent that reads agents/grader.md and evaluates each assertion.

Save the output as grading.json in the run directory:

{
  "eval_id": 0,
  "grading": [
    {
      "text": "includes_auth_failure_test",
      "passed": true,
      "evidence": "Found 401 status code in test case 3"
    },
    {
      "text": "includes_curl_commands",
      "passed": true,
      "evidence": "Found 'curl -X POST' in test case 1"
    }
  ]
}

The grading.json entries must use the exact field names:

• text
• passed
• evidence

The evaluation viewer depends on those names.

Aggregate the benchmark

Run the aggregation script from the skill-creator directory:

python -m scripts.aggregate_benchmark api-tester-workspace/iteration-1 --skill-name api-tester

This produces:

• benchmark.json
• benchmark.md

The benchmark includes pass rates, timing, token usage, mean ± standard deviation, and deltas between configurations.

Analyze the benchmark

Read the benchmark instead of focusing only on the final outputs.

Look for:

• Non-discriminating assertions: checks that always pass with and without the Skill
• High-variance evaluations: potentially flaky test cases
• Time and token tradeoffs: quality gains that cost too much latency or context

Use agents/analyzer.md for the detailed analysis process.

Step 7: Launch the Eval Viewer

The eval viewer provides a browser interface for qualitative review and quantitative comparison.

Generate the viewer

nohup python /path/to/skill-creator/eval-viewer/generate_review.py \
  api-tester-workspace/iteration-1 \
  --skill-name "api-tester" \
  --benchmark api-tester-workspace/iteration-1/benchmark.json \
  > /dev/null 2>&1 &

VIEWER_PID=$!

For iteration two and later, provide the previous workspace:

--previous-workspace api-tester-workspace/iteration-1

What reviewers see

The Outputs tab provides:

• The original prompt
• Generated output files rendered inline
• Previous output for later iterations
• Formal assertion grades
• A feedback field that saves automatically
• Previous feedback for later iterations

The Benchmark tab provides:

• Pass rates for each configuration
• Timing comparisons
• Token usage
• Per-evaluation details
• Analyst observations

Tell the reviewer:

I opened the results in your browser. Use the Outputs tab to review each test case and leave feedback. Use the Benchmark tab to compare quality, timing, and token usage. When you are done, return here and let me know.

Headless environments

If webbrowser.open() is unavailable, generate a standalone HTML file:

--static /path/to/output/review.html

When the user selects Submit All Reviews, the viewer downloads feedback.json.

Step 8: Read Feedback and Iterate

After review, read feedback.json:

{
  "reviews": [
    {
      "run_id": "eval-0-with_skill",
      "feedback": "the chart is missing axis labels",
      "timestamp": "2026-03-23T10:30:00Z"
    },
    {
      "run_id": "eval-1-with_skill",
      "feedback": "",
      "timestamp": "2026-03-23T10:31:00Z"
    },
    {
      "run_id": "eval-2-with_skill",
      "feedback": "perfect, love this",
      "timestamp": "2026-03-23T10:32:00Z"
    }
  ],
  "status": "complete"
}

An empty feedback field means the reviewer found no issue. Prioritize test cases with specific complaints.

Improve the Skill without overfitting

Improve patterns, not only individual test prompts.

For example:

• If outputs repeatedly create the same helper script, bundle that script in scripts/.
• If the Skill wastes time on an unnecessary step, remove or simplify that instruction.
• If an instruction requires repeated ALWAYS or NEVER wording, explain the reason behind the behavior instead.
• If a fix only works for one test case, generalize the underlying rule before adding it.

Read execution transcripts as well as final outputs. A final response can look correct while the Skill still consumes unnecessary tokens or follows an inefficient process.

Repeat the iteration loop

1. Update the Skill.
2. Rerun every test case into iteration-<N+1>/.
3. Include baseline runs again.
4. Launch the viewer with --previous-workspace.
5. Collect feedback.
6. Repeat until output quality stabilizes.

Stop when:

• The user says they are happy
• All feedback is empty
• New iterations are no longer producing meaningful improvements

When finished, stop the viewer:

kill $VIEWER_PID 2>/dev/null

Step 9: Optimize the Skill Description

The description field in SKILL.md frontmatter is the main trigger mechanism. Optimize it after the core workflow works.

Create trigger evaluation queries

Create 20 queries split between should-trigger and should-not-trigger cases:

[
  {
    "query": "ok so my boss just sent me this xlsx file (its in my downloads, called something like 'Q4 sales final FINAL v2.xlsx') and she wants me to add a column that shows the profit margin as a percentage. The revenue is in column C and costs are in column D i think",
    "should_trigger": true
  },
  {
    "query": "I need to create a pivot table from this CSV and email it to the team",
    "should_trigger": false
  }
]

For the 8–10 should-trigger queries, include:

• Formal and casual phrasing
• Requests that do not explicitly name the Skill
• Common and uncommon use cases
• Edge cases

For the 8–10 should-not-trigger queries, include:

• Near-misses with similar keywords
• Adjacent tasks better handled by another Skill
• Ambiguous requests that naive keyword matching might incorrectly match

Avoid easy negative cases such as “Write a Fibonacci function” for a PDF-related Skill. Useful negatives should be close enough to expose incorrect triggering.

Review the eval set with the user

Use the provided HTML template:

1. Read assets/eval_review.html.
2. Replace placeholders with the Skill name, current description, and eval data.
3. Write the result to a temporary file.
4. Open it for review.

open /tmp/eval_review_api-tester.html

The user can edit queries, change trigger expectations, and add or remove cases. When they select Export Eval Set, the browser downloads:

~/Downloads/eval_set.json

This review matters because poor evaluation queries lead to poor trigger descriptions.

Run the optimization loop

python -m scripts.run_loop \
  --eval-set /path/to/trigger-eval.json \
  --skill-path /path/to/api-tester \
  --model claude-sonnet-4-6 \
  --max-iterations 5 \
  --verbose

Use the same model ID as the current session so trigger behavior matches what users experience.

The optimization script:

1. Splits the eval set into 60% training and 40% held-out testing.
2. Evaluates the current description three times for reliability.
3. Proposes description improvements from failures.
4. Re-evaluates training and test sets.
5. Iterates up to five times.
6. Returns best_description, selected from held-out test score rather than training score.

Apply the best description

Update the description value in SKILL.md frontmatter and show the user the change.

Before:

description: How to test REST APIs systematically

After:

description: How to test REST APIs systematically. Use when users mention API testing, endpoints, REST, GraphQL, or want to validate API responses. Make sure to suggest this skill whenever testing is involved, even if they don't explicitly mention 'testing'.

Step 10: Package and Distribute

When the Skill is ready, package it:

python -m scripts.package_skill /path/to/api-tester

The command creates a .skill file. Share the generated file path with the user.

Installation

Users can install a Skill by:

• Placing the .skill file in their Skills directory, or
• Using the Claude Code Skill installation command

Common Skill Creation Mistakes

Mistake 1: Vague descriptions

Too vague:

description: A skill for working with APIs

Actionable:

description: How to test REST APIs systematically. Use when users mention API testing, endpoints, REST, GraphQL, or want to validate API responses. Make sure to suggest this skill whenever testing is involved, even if they don't explicitly mention 'testing'.

The second version describes the task and the trigger conditions.

Mistake 2: Overly restrictive instructions

Avoid:

ALWAYS use this exact format. NEVER deviate. MUST include these sections.

Prefer:

Use this format because it helps stakeholders quickly find test status, evidence, and next steps. Adapt the structure when the audience needs a different presentation.

Explain why the format exists so the Skill can adapt without losing the goal.

Mistake 3: Skipping test cases

Test cases catch issues before users do. Even subjective Skills benefit from two or three sample prompts and qualitative review.

Mistake 4: Ignoring timing data

A Skill that takes 10× longer may not be practical. Track timing and token usage alongside output quality.

Mistake 5: Recreating repeated scripts

If every evaluation independently writes generate_report.py, bundle it:

api-tester/
├── SKILL.md
└── scripts/
    └── generate-report.py

Bundled scripts improve consistency and reduce repeated work.

Real-World Skill Examples

MCP Builder Skill

Anthropic’s MCP Builder Skill supports Model Context Protocol server development.

Key features:

• Python and Node.js templates
• MCP evaluation framework
• Best-practices reference documentation

mcp-builder/
├── SKILL.md
├── reference/
│   ├── mcp_best_practices.md
│   ├── python_mcp_server.md
│   └── node_mcp_server.md
└── evaluation/
    └── evaluation.md

Docx Skill

The Docx Skill generates Word documents programmatically.

Key features:

• Bundled python-docx scripts
• Templates for common documents
• Styling guidance for consistent output

Typical workflow:

1. Understand document requirements.
2. Select or create a template.
3. Generate the document with a python-docx script.
4. Validate the output structure.

Frontend Design Skill

The Frontend Design Skill builds web interfaces with modern implementation patterns.

Key features:

• Component libraries
• Tailwind CSS patterns
• Accessibility checks

It uses progressive disclosure: the core workflow is in SKILL.md, while component-specific documentation lives in references/.

Testing Your Skill with Apidog

If you are building API-related Skills, Apidog can be part of the workflow.

Example: API testing Skill integration

Add an implementation section to your Skill:

## Running API Tests

Use Apidog for systematic testing:

1. Import the OpenAPI specification into Apidog.
2. Generate test cases from the specification.
3. Run tests and export results as JSON.
4. Validate responses against expected schemas.

For custom assertions, use Apidog's scripting feature.

Bundle Apidog scripts

api-tester/
├── SKILL.md
└── scripts/
    ├── run-apidog-tests.py
    └── generate-report.py

Bundling scripts prevents future invocations from recreating the same implementation.

Once you can build your own Skills, design-focused Claude Code Skills show how the same mechanism can support layout generation, component scaffolding, and accessibility review.

If you work across multiple AI coding agents rather than only Claude Code, OpenClaw's tools and Skills system follows a similar modular philosophy.

Conclusion

Claude Code Skills let you capture repeatable development workflows as reusable capabilities.

Use this implementation process:

1. Define the Skill’s intent and output.
2. Write a focused SKILL.md.
3. Create realistic test prompts.
4. Run each prompt with and without the Skill.
5. Grade output with reusable assertions.
6. Compare pass rate, latency, and token usage.
7. Iterate from reviewer feedback.
8. Optimize the frontmatter description for trigger accuracy.
9. Package the finished Skill as a .skill file.

FAQ

How long does it take to create a Skill?

Simple Skills take 15–30 minutes. Complex Skills with multiple reference files, bundled scripts, and evaluation iterations can take two to three hours.

Do I need test cases for every Skill?

No. Skills with verifiable outputs, such as code generation, file transforms, and data extraction, benefit most from formal test cases. Subjective tasks such as writing style or design quality are often better evaluated qualitatively.

What if my Skill does not trigger reliably?

Improve the description field in SKILL.md. Add specific trigger phrases and relevant contexts, then run the description optimization loop with 20 trigger evaluation queries.

How do I share Skills with my team?

Package the Skill:

python -m scripts.package_skill <path>

Then distribute the generated .skill file. Team members place it in their Skills directory.

Can Skills call external APIs?

Yes. Bundle scripts that call external APIs and document when to run them. Store API keys in environment variables rather than in the Skill package.

What is the file size limit for Skills?

There is no hard limit, but keep SKILL.md under 500 lines. Move detailed documentation to reference files. Scripts and assets load on demand and do not count toward the core instruction limit.

How do I update an existing Skill?

Copy the installed Skill to a writable location, make changes there, and repackage it. Preserve the original name unless you are intentionally creating a separate variant.