An agent skill can be easy to download and still be difficult to evaluate. Before giving it access to a repository or connecting an external service, check four things separately: the file's structure, the source version, the permissions it asks for, and the evidence for the task you actually need.
Disclosure: This article was drafted by an AI assistant for SkillManual. The examples are fictional. They illustrate file review, not completed runtime safety tests. SkillManual is the project associated with this account.
1. Start with the frontmatter
The Agent Skills specification describes a SKILL.md file with YAML frontmatter followed by Markdown instructions. The required fields include name and description.
Here is a small fictional example:
---
name: release-notes
description: "Draft concise release notes from a changelog."
---
Read the repository changelog and draft release notes.
Preserve version numbers and do not invent changes.
Use a simple lowercase name such as release-notes, and make the parent folder match the name. Keep the description useful: it should explain what the skill does and when to use it.
Now compare this incomplete file:
---
name: release-notes
---
Draft release notes from the repository changelog.
The missing description is a concrete problem to fix before evaluating the instructions. Add the field inside the YAML block, then check the corrected text again. A field mentioned only in the Markdown body does not fill the frontmatter requirement.
When a YAML value contains punctuation that could be interpreted as syntax, quoting the value can make the intended string clearer. Check the actual parser's diagnostic instead of changing unrelated instructions at random.
2. Review the whole source bundle
A valid file structure does not tell you what referenced scripts or dependencies do. Read the supporting files before running a command. Pay particular attention to instructions that download and execute code, request credentials, send data to an external URL, or change protective settings.
Those patterns need context. An external documentation link can be legitimate; a script reference can be necessary. Treat a static finding as a reason to inspect the relevant destination or source, rather than automatically labeling the whole skill malicious.
Write down the repository, directory path, and commit you reviewed. A later version can change its instructions or dependencies, so earlier evidence should not silently apply to the new version.
3. Separate file access from running costs
A freely available skill file can still depend on a paid agent, model plan, API, or other service. Read its license and installation instructions, then identify the services needed for your intended workflow.
This is especially useful when comparing development skills: two files aimed at the same task can require different tools and permissions. Record unknown costs as unknown, rather than interpreting a missing price as free execution.
4. Ask what the evidence actually covers
An installation command, a static scan, and a successful task run answer different questions.
- An installation check can show that files reached the intended location.
- A static check can identify configured structural errors or risk patterns in the material it inspected.
- A task run can provide evidence about a specific input, source version, host environment, and observed output.
For a task report, look for that scope and its limitations. Do not turn one successful example into a claim that every workflow or host is supported.
A small check you can try
The SkillManual SKILL.md Validator accepts one file's text and checks its structure and configured static risk patterns. You can use the fictional examples above to explore the difference between complete frontmatter and a missing required field. The site may require native human verification before accepting a check.
Its scope is deliberately limited: it does not inspect an entire folder or ZIP, execute the skill, test link reachability, or establish runtime compatibility. No configured findings does not mean the skill is safe.
After checking the text, return to the complete source and confirm the version, dependencies, permissions, and evidence for your task. The SkillManual directory presents pinned sources, cost information, and available review or test evidence to support that comparison; not every entry has been independently tested.
Top comments (0)