Why Custom Skills Matter
Out-of-the-box AI coding agents are powerful, but they don't know your team's conventions, your deployment process, or your documentation style. Skills let you encode that knowledge so the agent follows your workflows every time.
But creating skills has been guesswork. You write a SKILL.md file, test it manually in a session, maybe tweak the description, and hope it works. There's no feedback loop, no measurement, no way to know if a change actually improved things.
opencode-skill-creator changes this by providing a structured workflow for the full skill lifecycle: create, evaluate, optimize, benchmark, and install.
Prerequisites
- OpenCode installed and configured
- Node.js 18+ (for the npm package)
- 5 minutes
Step 1: Install
One command:
npx opencode-skill-creator install --global
This adds the plugin to your global OpenCode config. Restart OpenCode to activate it.
Verify the install:
ls ~/.config/opencode/skills/skill-creator/SKILL.md
Then ask OpenCode: Create a skill that helps with Docker compose files
You should see it use the skill-creator workflow and tools.
Step 2: Describe What You Want
The skill-creator starts with an intake interview. It asks 3-5 targeted questions about what your skill should do:
- What should this skill enable OpenCode to do end-to-end?
- When should this skill trigger?
- What output format and quality bar are expected?
- What workflow steps must be preserved vs. where can the agent improvise?
Don't skip this. The interview captures your intent before any code is written. Think of it as shadowing a teammate — you're the domain expert, the agent is the new hire learning your workflow.
Step 3: Review the Skill Draft
Based on your interview, the skill-creator produces a draft SKILL.md with:
- Proper YAML frontmatter (name and description)
- Markdown instructions for the agent
- Optional supporting files (references, agents, templates)
The draft goes to a staging directory (outside your repo) so your project stays clean:
/tmp/opencode-skills/your-skill-name/
├── SKILL.md
├── agents/
├── references/
└── templates/
Review this draft. Make sure the description is accurate (it's the primary triggering mechanism) and the instructions reflect your actual workflow.
Step 4: Generate Eval Test Cases
The skill-creator automatically generates test cases — realistic prompts that an OpenCode user would actually type:
{
"skill_name": "docker-compose",
"evals": [
{
"id": 1,
"prompt": "help me set up a compose file for my Node app with a Postgres database",
"expected_output": "Skill triggers and provides Docker compose guidance",
"should_trigger": true
},
{
"id": 2,
"prompt": "explain how Kubernetes deployments work",
"should_trigger": false
}
]
}
Good eval queries are realistic and specific — not abstract like "help with containers" but concrete like "ok so my boss just sent me this xlsx file (its in my downloads, called something like 'Q4 sales final FINAL v2.xlsx')..."
Review the eval set. Add or modify test cases that reflect your real usage.
Step 5: Run Evals
The eval system runs each test case twice — once with the skill and once without (baseline). This measures whether the skill actually improves the output.
For each test case:
- OpenCode runs with the skill loaded
- OpenCode runs without the skill
- Both outputs are saved for comparison
Timing data (tokens used, duration) is captured automatically.
Step 6: Review Results Visually
The skill-creator launches an HTML eval viewer:
Call skill_serve_review with:
workspace: /tmp/opencode-skills/your-skill-name-workspace/iteration-1
skillName: "your-skill-name"
The viewer shows:
- Outputs tab: Each test case with with-skill and without-skill outputs side by side
- Benchmark tab: Quantitative metrics — pass rates, timing, token usage
- Feedback fields: Leave comments on each test case
Review the outputs. Give specific feedback on what's working and what's not. Empty feedback means "looks good."
Step 7: Iterate and Improve
Based on your feedback, the skill-creator improves the skill:
- Applies your feedback
- Reruns all test cases (new iteration)
- Launches the reviewer with previous iteration for comparison
- You review again
Repeat until you're satisfied or feedback is all empty.
Step 8: Optimize the Description
Even with perfect skill instructions, the skill won't trigger correctly if the description field isn't right. The description is what OpenCode reads to decide whether to load your skill.
The optimization loop:
- Generates 20 eval queries (should-trigger and should-not-trigger)
- Splits them 60/40 into train/test
- Evaluates each query 3 times for statistical reliability
- Analyzes failure patterns
- LLM proposes improved descriptions
- Re-evaluates on both train and test
- Selects the best description by test score
- Repeats up to 5 iterations
# Tell OpenCode:
"Optimize the description of my docker-compose skill"
This takes some time — grab a coffee while it runs.
Step 9: Install the Final Skill
Once you're satisfied with the skill and its description:
-
Project-level:
.opencode/skills/your-skill-name/SKILL.md— available only in this project -
Global:
~/.config/opencode/skills/your-skill-name/SKILL.md— available everywhere
# Project-level install
cp -r /tmp/opencode-skills/your-skill-name/ .opencode/skills/your-skill-name/
# Global install
cp -r /tmp/opencode-skills/your-skill-name/ ~/.config/opencode/skills/your-skill-name/
Only the final validated skill gets installed. All eval artifacts stay in the staging directory.
Real-World Example: Docker Compose Skill
Here's what the full workflow looks like in practice:
Ask OpenCode: "Create a skill that helps with Docker compose files"
Interview: The skill-creator asks about your conventions (multi-service vs. single container, development vs. production, preferred base images)
Draft: Produces a SKILL.md with Docker compose best practices, service configuration patterns, volume mount strategies
Eval: Generates test cases like "my api keeps crashing on startup, can you help me debug my compose file" (should trigger) and "what's the difference between Docker and Podman" (should not trigger)
Review: You look at the outputs, give feedback: "the skill should prioritize security configurations in production compose files"
Iterate: Improved skill draft, better outputs
Optimize: Description goes from "Help with Docker compose files" to something much more specific that triggers reliably
Install: Copy to
~/.config/opencode/skills/docker-compose/
Tips for Great Skills
- Be specific in the intake interview: The more context you give, the better the draft
- Don't skip evals: They catch triggering issues you'd never find manually
- Use realistic test prompts: Write them the way you'd actually type them, typos and all
- Iterate at least twice: First drafts are rarely perfect
- Optimize the description: It's the #1 factor in whether your skill triggers correctly
- Install globally for general skills, project-level for specific ones
Getting Started
npx opencode-skill-creator install --global
Then ask OpenCode to create a skill. That's it.
GitHub: https://github.com/antongulin/opencode-skill-creator
npm: https://www.npmjs.com/package/opencode-skill-creator
opencode-skill-creator is free and open source (Apache 2.0). Star it on GitHub. Install: npx opencode-skill-creator install --global
Top comments (0)