In this article, we review skills folder in DeepAgents codebase. You will learn:
eval-creator
langsmith-trace
Skills
I review skills in the OSS codebases to identify the best practices that go into defining the skills that are used in a codebase. When I write about skills, I tend to keep it to the meta and overview of the sections defined.
skills/eval-creator
The following is the meta info defined for the skills/eval-creator.
---
name: eval-writer
description: "Create new eval suites for the deepagentsjs monorepo. Handles dataset design, test case scaffolding, scoring logic, vitest configuration, and LangSmith integration. Use when the user asks to: (1) create an eval, (2) write an evaluation, (3) add a benchmark, (4) build an eval suite, (5) evaluate agent behaviour, (6) add test cases for a capability, or (7) implement an existing benchmark (e.g. oolong, AgentBench, SWE-bench). Trigger on phrases like 'create eval', 'new eval', 'add eval', 'benchmark', 'evaluate', 'eval suite', 'write evals for'."
---
I like this description. It lists out clearly when this skill should get triggered, depending on what user asks.
There is also Before you start section, that mentions what the Copilot should refer to, to understand existing patterns.
The following are the sections defined:
Understand the eval requirements
Create the eval package
Design test cases
Write scoring logic
Wire up the test file
Install and verify
Update documentation
Parity with Python deepagents evals
Reference: ls.test.each API
Reference: LangSmith integration
So I see that whatever is mentioned in the list about when this skill should triggered, is explained deeply with examples, code snippets etc.,
skills/langsmith-trace
The following is the meta info defined for the skills/langsmith-trace.
---
name: langsmith-trace
description: "INVOKE THIS SKILL when working with LangSmith tracing OR querying traces. Covers adding tracing to applications and querying/exporting trace data. Uses the langsmith CLI tool."
---
This one’s description is straight forward. This just defines when this skills is supposed to be triggered and what it covers and what it uses.
There is no table of content as such in this skill but this focuses mainly on:
Adding tracing
Querying traces
About me:
Hey, my name is Ramu Narasinga. Email: ramu.narasinga@gmail.com
Tired of AI slop?
I spent 3+ years studying OSS codebases and wrote 350+ articles on what makes them production-grade. I built an open source tool that reviews your PR against your existing codebase patterns.
Your codebase. Your patterns. Enforced.
Top comments (0)