How to turn domain knowledge, workflows, scripts, and engineering rules into reliable capabilities for AI agents.
AI agents are becoming part of everyday software development. They can explore codebases, implement features, run tests, investigate incidents, and interact with external systems.
But there is a problem that becomes increasingly obvious as projects grow:
Giving an agent access to a codebase does not mean giving it the knowledge required to work correctly within that codebase.
The agent might understand React, Go, REST APIs, databases, and distributed systems. Yet it may not know your application's business rules, your team's architectural decisions, the behavior of a particular WebSocket connection, or the mistakes that have already caused production incidents.
This is where agent skills become valuable.
I've been exploring ways to make AI agents more consistent across real engineering workflows. My biggest takeaway is that skills deserve much more attention than they usually receive.
A skill is not just a collection of instructions. It can become an executable part of your engineering process.
In this article, I'll explain how I approach skill design, how to organize domain knowledge, when to use scripts instead of repeatedly calling tools, and how to make sure agents actually follow the rules we define.
The examples use the open Agent Skills format as a reference.
1. Understand what a skill is supposed to do
An Agent Skill is a reusable package of instructions and resources that teaches an agent how to perform a particular class of tasks.
A typical skill looks like this:
fund-analysis/
├── SKILL.md
├── scripts/
│ └── fetch-fund-data.ts
├── references/
│ └── fund-domain.md
└── assets/
└── report-template.md
The SKILL.md file is the entry point. It describes when the skill should be used and provides the instructions the agent needs to follow. Supporting scripts, references, and assets can provide additional capabilities.
The important distinction is that a skill should not merely describe a subject. It should help the agent perform a task correctly and consistently.
For example, a weak skill might say:
Follow our fund analysis best practices.
A useful skill would explain:
- Which data sources are authoritative.
- How fund identifiers map to internal database records.
- Which fields are required before analysis can begin.
- How to handle missing or stale data.
- Which calculations are allowed and how to validate them.
- Which scripts the agent should execute.
- What the final output should contain.
The second version turns vague knowledge into an operational process.
2. Start with domain knowledge, not generic AI-generated instructions
One of the easiest mistakes is asking an LLM to create a skill from a title alone.
For example:
Create a skill for working with our backend.
The resulting instructions might contain generic recommendations about writing clean code, handling errors, and following REST conventions. These are not necessarily wrong, but they may contribute very little to your project.
The real value comes from information the model cannot reliably infer.
Consider a financial application that handles investment funds. A domain-specific skill might need to explain:
- The difference between TEFAS and BEFAS data.
- Which identifiers are used by internal services.
- How fund prices and historical values are represented.
- Which calculations depend on business-day boundaries.
- How cached data should be interpreted.
- Which fields can be missing or delayed.
- What the application must never assume about incomplete data.
The same principle applies to networking.
A skill for WebSocket or socket-based integrations might document:
- The expected connection lifecycle.
- Authentication and subscription requirements.
- Message formats and event types.
- Reconnection and resubscription behavior.
- Heartbeat and timeout rules.
- Duplicate messages and ordering assumptions.
- Cleanup requirements when a connection closes.
These details can be far more valuable than another page explaining what a WebSocket is.
Where should this knowledge come from?
I would start with existing engineering artifacts rather than asking an LLM to invent the rules:
- Architecture documentation and API specifications.
- Existing code and configuration.
- Incident reports and production runbooks.
- Code review feedback.
- Bug fixes and recurring agent mistakes.
- Decisions made during implementation.
- Test cases that capture important edge cases.
A particularly useful technique is to extract a skill after completing a real task with an agent.
When you have to correct the agent, ask yourself: is this correction specific to the current task, or is it a reusable rule that should apply next time?
If it is reusable, it probably belongs in a skill, a project rule, a test, or a combination of these.
3. Treat skills as seriously as code
This is one of my strongest recommendations.
Everything you put into a skill can influence the code an agent writes.
A misleading instruction can produce a misleading implementation. An outdated API example can cause incorrect integrations. A missing edge case can become a recurring bug.
That makes skill quality an engineering concern, not merely a documentation concern.
Imagine a skill that tells an agent to use a shared HTTP client but fails to mention the project's authentication middleware. The agent might generate code that compiles, passes a basic test, and still violates the application's architecture.
Or imagine a skill that describes a socket connection but omits the resubscription procedure after reconnecting. The implementation may work during normal operation and silently stop receiving updates after a network interruption.
These are not just writing problems. They are failures in the instructions that guide implementation.
I would manage important skills with practices similar to those used for production code:
- Version them in Git.
- Review changes through pull requests.
- Keep examples aligned with the actual implementation.
- Validate scripts and commands before recommending them.
- Test skills against representative tasks.
- Record recurring failures and update the relevant instructions.
- Remove outdated rules instead of accumulating contradictory ones.
A skill should evolve with the codebase.
If a service changes its API contract, the related skill may need to change too. If a production incident reveals a missing constraint, the skill should capture the lesson where future agents will see it.
The goal is not to make skills long. It is to make their instructions dependable.
4. Organize skills around domains and coherent tasks
A skill library should reflect how work is actually performed.
I prefer starting with domain-oriented capabilities and then defining the workflows within each domain.
For example:
.agents/
├── skills/
│ ├── fund-analysis/
│ │ ├── SKILL.md
│ │ └── scripts/
│ ├── realtime-data/
│ │ ├── SKILL.md
│ │ └── scripts/
│ ├── api-integration/
│ │ └── SKILL.md
│ └── incident-investigation/
│ └── SKILL.md
└── agents/
├── analyst.md
├── developer.md
└── reviewer.md
This is an illustrative layout, not a requirement of the Agent Skills specification. Exact discovery paths and agent configuration vary between tools.
The key is to establish clear boundaries.
A fund-analysis skill might own fund data interpretation and validation. A realtime-data skill might own connection lifecycle rules, subscriptions, and message handling. An api-integration skill might own the project's conventions for consuming external APIs.
Avoid creating one enormous skill that covers the entire application. Also avoid splitting every tiny instruction into a separate skill.
A useful rule of thumb:
One skill should represent a coherent unit of work that can be reused independently.
If two sets of instructions almost always need to be loaded together, consider whether they belong in the same skill. If one skill frequently activates for unrelated tasks, narrow its scope.
5. Make skill activation an explicit engineering decision
A skill is useful only if the agent knows when to use it.
The description field in SKILL.md plays a critical role in skill discovery. It should clearly describe the task and the situations in which the skill applies.
For example, this is too vague:
---
name: realtime-data
description: "Helps with realtime data."
---
A more useful version would be:
---
name: realtime-data
description: >
Use when implementing, debugging, or reviewing WebSocket and socket-based
integrations in this application. Follow the project's connection lifecycle,
authentication, subscription, reconnection, message validation, and cleanup
rules.
---
The second description gives the agent a much clearer signal about when the skill is relevant.
However, discovery alone is not enough for every workflow.
Put required skill rules in the relevant agent definition
In my experience, if a particular agent is expected to follow a skill for its core responsibilities, I would make that requirement explicit in the agent's own instruction file.
For example, a developer agent might contain:
## Required Skills
- Before modifying fund-related business logic, load and follow
the `fund-analysis` skill.
- Before modifying socket connections, subscriptions, or reconnection
behavior, load and follow the `realtime-data` skill.
- If a required skill is unavailable, report the problem rather than
silently proceeding as if the skill had been applied.
This is an illustrative convention, not universal syntax. The actual invocation mechanism depends on the agent runtime.
The principle is what matters: do not rely entirely on the model guessing when a critical skill should be used.
If your agent framework supports explicit skill invocation, use it. If it supports mandatory preconditions or workflow gates, use those for important tasks. If it only supports automatic discovery, test that discovery carefully.
There is also an important distinction between instructions and enforcement.
A rule inside a skill tells an agent what it should do. A test, permission boundary, validator, or runtime policy can verify or constrain what it actually does.
For critical requirements, combine both.
6. Give skills a clear workflow
A skill should guide the agent through a sequence of decisions and actions, not just present a list of facts.
For example, a fund analysis workflow could look like this:
Understand the request
↓
Identify the fund and required data
↓
Fetch data from the approved source
↓
Validate completeness and freshness
↓
Perform the requested calculation
↓
Verify the result
↓
Return the result with relevant limitations
This sequence makes the intended process visible.
A corresponding section in SKILL.md might look like:
## Workflow
1. Identify the requested fund and resolve its canonical identifier.
2. Fetch the required data using the approved script.
3. Validate that the response contains all required fields.
4. Check data freshness before calculating performance.
5. Never silently replace missing values with zero.
6. Run the calculation and validate the result.
7. Return the result with its date range and data limitations.
## Failure handling
- If the fund cannot be identified, ask for clarification.
- If the source is unavailable, report the failure.
- If the data is incomplete, do not present the result as complete.
- Do not invent prices or historical values.
Notice the difference between a general instruction such as “analyze the fund carefully” and a sequence with clear inputs, validation steps, and failure conditions.
The workflow should make it clear what the agent needs to accomplish and when it can consider the task complete.
Prefer clear defaults over a menu of alternatives
If a skill gives an agent five equivalent ways to accomplish a routine task, the agent must spend additional effort choosing among them.
When one approach is the established project default, document it directly and mention alternatives only when they are genuinely needed.
Be strict where the process is fragile, such as a migration or an irreversible operation. Allow flexibility where several approaches are safe and equally valid.
7. Use scripts and CLIs to make skills executable
This is where skills become particularly interesting.
A skill does not have to consist entirely of natural-language instructions. It can include executable scripts that perform predictable operations.
For example:
fund-analysis/
├── SKILL.md
└── scripts/
├── fetch-fund.ts
├── validate-response.ts
└── calculate-performance.ts
The skill explains the workflow. The scripts perform deterministic work.
This separation has several advantages:
- The same operation can be executed consistently.
- Input validation can happen before data reaches the model.
- Large responses can be filtered or summarized.
- Error handling can be implemented and tested in code.
- The agent can receive structured output instead of raw payloads.
- Repeated instructions can be replaced with reusable functionality.
Example: optimize the data before returning it to the agent
Imagine you have an MCP server that returns a large response containing thousands of records, extensive metadata, and fields that are irrelevant to the current task.
Sending the entire response into the agent's context may increase token consumption and make the useful information harder to identify.
Instead, you could expose a CLI command that accepts filters and returns only the relevant fields:
fund-cli history \
--fund ABC \
--from 2026-01-01 \
--to 2026-06-30 \
--fields date,price \
--format json
The exact command is illustrative. A real implementation would need to define supported flags, validate dates and identifiers, and handle errors explicitly.
The output might be:
{
"fund": "ABC",
"currency": "TRY",
"data": [
{ "date": "2026-01-02", "price": 12.41 },
{ "date": "2026-01-05", "price": 12.58 }
]
}
The agent receives a smaller, task-specific response rather than an unnecessarily large payload.
This is not only a token optimization. It is also a way to control which data enters the model's reasoning process.
Can a CLI replace an MCP server?
Sometimes, yes.
If an agent can execute local commands, a well-designed CLI may provide everything it needs for a particular workflow. You may not need to create a separate MCP server for every internal capability.
A CLI can be a good fit when:
- The operation is naturally expressed as a command.
- You need predictable filtering and output formats.
- The same functionality is useful to humans, scripts, and agents.
- The runtime already provides a secure way to execute commands.
- You want to reuse existing authentication, validation, and application logic.
MCP can be a better fit when you need standardized tool discovery, structured tool schemas, remote integrations, or interoperability across clients that support MCP.
These approaches are not mutually exclusive. An MCP tool can call a shared internal service or CLI, and a CLI can expose the same business logic to other consumers.
The design goal should not be to eliminate MCP at all costs. It should be to avoid unnecessary integration layers and unnecessary context.
Choose the simplest reliable interface for the operation, and return only the information the agent needs.
One important caveat: scripts are executable code. Review them, validate their inputs, restrict their permissions, and control network and filesystem access. A skill should not become a backdoor for unrestricted command execution.
8. Be deliberate about reference files and context
The Agent Skills format supports progressive disclosure: an agent discovers a skill through its metadata, loads the main instructions when the skill is activated, and can access additional resources when needed.
This is useful, but it can be misapplied.
I would not split a skill into many reference files simply to make the directory look organized. Nor would I instruct an agent to read every reference file at the beginning of every task.
Both patterns can defeat the purpose of context management.
A skill should contain the information that the agent needs to follow the workflow reliably. Additional files should exist for a clear reason.
For example:
fund-analysis/
├── SKILL.md
└── references/
├── fund-identifiers.md
└── performance-calculations.md
The main skill could say:
## Data requirements
Resolve the canonical fund identifier before fetching data.
Read `references/fund-identifiers.md` if the incoming identifier is
ambiguous or needs to be mapped to an internal record.
Read `references/performance-calculations.md` before implementing
or changing a performance calculation.
The important part is not the presence of the references/ directory. It is the explicit connection between a situation and the resource needed to handle it.
What belongs in the main skill?
Keep critical instructions, the main workflow, important edge cases, and essential validation requirements in SKILL.md.
Move detailed material into a separate file when it is genuinely optional for most executions, such as a long schema catalog or an extensive API reference.
If the agent must know a rule before making a decision, do not hide it in a file the agent may never realize it needs to read.
This is especially important for non-obvious failure cases. If a socket connection requires resubscription after reconnecting, that rule belongs in the main workflow unless the workflow itself explicitly directs the agent to load the relevant resource before handling reconnection.
Progressive disclosure should reduce unnecessary context, not make important instructions harder to discover.
9. Design skills for predictable outputs
An agent may perform the right operations and still return an output that is difficult to consume.
If a skill is used in an automated pipeline, its output contract matters.
For example, an incident-investigation skill could require:
## Required output
1. Summary
2. Observed evidence
3. Likely root cause
4. Confidence and remaining uncertainty
5. Recommended next steps
6. Validation performed
For machine-consumed output, use a defined JSON schema or another structured format where appropriate.
Do not ask for a generic “detailed report” if downstream systems depend on specific fields.
The same applies to scripts. A script should return predictable output, meaningful exit codes, and useful error messages. If it returns JSON, define the schema and avoid mixing debugging output into standard output when that would break parsing.
The more automated the workflow becomes, the more important these contracts become.
10. Test skills as you test code
Writing a skill and committing it is not the finish line.
A skill can be syntactically valid and still be ineffective. Its description might not trigger when needed, its workflow might skip a critical step, or its instructions might conflict with the actual codebase.
I recommend testing at two levels.
Level 1: Does the skill activate when it should?
Create a set of realistic prompts and label whether each one should invoke the skill.
For a socket integration skill, positive cases might include:
- “Implement reconnect handling for the price stream.”
- “The connection is restored, but subscriptions are missing.”
- “Review the cleanup logic when the component unmounts.”
Negative cases might include:
- “Rename this CSS class.”
- “Update a static marketing page.”
- “Explain what a WebSocket is without changing the project.”
Test both obvious requests and less explicit requests. Otherwise, you may only prove that the skill works when the user uses exactly the same terminology as its description.
The official Agent Skills guidance recommends evaluating activation with both positive and negative examples and refining the description based on observed behavior. See the description optimization guide.
Level 2: Does the skill improve the result?
Activation alone does not prove quality.
Run representative tasks and check whether the agent:
- Follows the intended workflow.
- Uses the correct tools and scripts.
- Respects domain-specific rules.
- Handles missing or invalid data correctly.
- Produces the required output format.
- Avoids unnecessary calls and irrelevant work.
- Passes the relevant tests.
Keep successful examples as well as failures. Successful runs help establish what good behavior looks like; failed runs reveal where the instructions need improvement.
When an agent repeatedly makes the same mistake, investigate the cause rather than automatically adding another paragraph.
Sometimes the fix is a clearer skill instruction. Sometimes it is a better script, a test, a permission restriction, or a change to the agent's orchestration.
11. Keep skills, agent rules, tools, and tests in their proper roles
These components solve different problems.
| Component | Primary responsibility |
|---|---|
| Agent rules | Define the agent's role, responsibilities, and required behavior |
| Skills | Provide reusable domain knowledge and task-specific workflows |
| CLI/scripts | Execute predictable operations and validate data |
| MCP/tools | Expose capabilities through a callable interface |
| Tests and validators | Check whether implementation and outputs meet requirements |
| Runtime permissions | Restrict which actions the agent can actually perform |
Confusing these responsibilities creates brittle systems.
For example, putting every project rule into every skill causes duplication and inconsistencies. Putting every operation into an MCP server may add unnecessary complexity. Relying on a skill instruction to prevent a destructive action is weaker than enforcing the relevant permission at the runtime level.
The architecture should make responsibilities clear.
A developer agent can have a general role and a set of required rules. A skill can provide the detailed procedure for a particular domain. A script can validate inputs and fetch data. A test can verify the output. A permission boundary can prevent the agent from performing an unauthorized operation.
Each layer contributes something different.
12. A practical checklist for creating a new skill
Before considering a skill ready for use, I would ask the following questions.
Scope and domain
- Does it solve a clearly defined, repeatable task?
- Does it capture project-specific knowledge rather than generic advice?
- Are its responsibilities distinct from other skills?
Activation
- Does the description explain when to use it?
- Have positive and negative activation cases been tested?
- Is invocation explicitly required for agents that must always use it?
Workflow
- Are the steps ordered and actionable?
- Are the expected inputs and outputs clear?
- Are failure cases and validation steps documented?
- Are defaults provided instead of a long list of equally weighted options?
Context
- Is the main skill focused on the information needed for the workflow?
- Are critical rules available before the agent needs to act on them?
- Are additional resources loaded only when relevant?
- Have redundant instructions and unnecessary explanations been removed?
Execution and reliability
- Can deterministic work be handled by a tested script or CLI?
- Are outputs structured and appropriately filtered?
- Are permissions, input validation, and error handling explicit?
- Has the skill been tested against real tasks and revised based on the results?
Maintenance
- Is the skill version-controlled and reviewed?
- Does it reflect the current implementation?
- Are recurring mistakes converted into reusable guidance or automated checks?
Conclusion: Skills are part of the engineering system
As AI agents take on more complex software tasks, the quality of their instructions becomes increasingly important.
A good skill captures knowledge that would otherwise be lost between sessions or repeatedly explained in prompts. It defines a reliable workflow, helps the agent choose the right tools, and makes recurring tasks more consistent.
My approach is to start with real engineering experience, organize skills around coherent domains, make critical invocation rules explicit, use scripts when deterministic execution is preferable, and test the resulting behavior against real tasks.
I also try to be careful with context. More instructions do not automatically produce better results. The objective is to provide the right information at the right time, with a clear path from understanding the task to validating the outcome.
Ultimately, I think the most useful way to view skills is this:
A skill is a version-controlled engineering artifact that influences how an AI agent works. Design it, review it, test it, and maintain it accordingly.
The model may be responsible for reasoning, but the system around it determines much of how reliably that reasoning turns into useful work.
Further Reading
Official Agent Skills Documentation
- Agent Skills — Overview
- Agent Skills — Specification
- Best Practices for Skill Creators
- Optimizing Skill Descriptions
Building and Using Agent Skills
- Anthropic — Agent Skills Documentation
- Anthropic — Creating Custom Skills
- OpenAI — Codex Customization
- OpenAI — Building Skills
Agent Architecture and Tool Integration
- Anthropic — Building Effective Agents
- Model Context Protocol (MCP) — Documentation
- Anthropic — Public Skills Repository
These resources provide additional perspectives on skill design, progressive disclosure, agent workflows, tool integration, and the practical implementation of reusable agent capabilities.
Top comments (1)
The distinction between generic advice and project-specific operational knowledge is important. A useful skill regression is a task where a missing or stale source must cause the agent to stop rather than invent a plausible value.