An agent skill can change upstream while the task you ask it to do stays the same. Keep a small run record so you can tell what changed before using the new version on real work.
- Record the source URL, commit or release, and the tool versions used for the run. If you cannot recover the exact skill version, record that gap.
- Keep one synthetic input and the expected result alongside the record. For a report-writing skill, that could be a short mock brief and a checklist of required headings.
- Run the old and new versions on the same input in separate scratch folders. Compare the generated files, paths, and any new network calls. Investigate unexplained changes before using the update on live files.
If the new run passes, update the version in your record and keep the sample for the next upgrade. Explore the Skillπ AI skill marketplace for skills to evaluate.
AI disclosure: This article was drafted with AI.
Top comments (0)