Zira

Posted on Jul 16

Microsoft Agent Framework Skills: Keep Agent Context Lean Without Trusting Scripts

#ai #agents #security #python

Your agent keeps getting worse as you add "just one more" policy document, runbook, or automation recipe to its system prompt.

The tempting fix is to give it the whole company handbook. The expensive result is a larger prompt on every turn, blurrier instructions, and scripts that can become invisible execution paths.

Microsoft’s Agent Skills for Python reached stable release on July 15. The useful idea is not the brand name. It is the runtime pattern: advertise a small catalogue, load a skill only when relevant, then fetch its resources or run its script only when needed. Microsoft calls this four-stage progressive disclosure: advertise → load instructions → read resources → run scripts. The announcement and the Agent Skills documentation describe the model and its controls.

That can make a production agent cheaper to reason about, but it does not make bundled automation safe by default. Here is a practical way to adopt it.

The architecture: a catalogue, not a mega-prompt

A file-based skill is a directory with a required SKILL.md plus optional scripts and references:

skills/
└── incident-triage/
    ├── SKILL.md
    ├── references/
    │   └── severity-policy.md
    └── scripts/
        └── fetch_service_health.py

The skill description should be short enough to route from. The body should contain the operating procedure. Large policy documents and executable code should stay outside the core instructions until the agent actually needs them. That mirrors the open Agent Skills specification: portable instruction packages with optional resources and scripts.

For a coding or operations agent, this avoids carrying every deployment runbook, repository convention, and incident policy into every code-edit turn.

Start with a read-only skill

Do not make the first skill a deployment or database-mutation tool. Start with a bounded workflow such as incident triage, dependency-policy lookup, or PR-review guidance.

Microsoft’s Python provider exposes tools to load a skill, read a resource, and run a script. Those tools require approval by default. In the example below, only the two read-only steps are approved for unattended use. Script execution stays behind approval.

from pathlib import Path
from agent_framework import Agent, SkillsProvider

skills = SkillsProvider.from_paths(
    skill_paths=str(Path(__file__).parent / "skills"),
    # Only for a reviewed, read-only skill catalogue:
    disable_load_skill_approval=True,
    disable_read_skill_resource_approval=True,
    # Do not disable approval for run_skill_script.
)

agent = Agent(
    client=client,
    instructions="Use approved skills when they match the request.",
    context_providers=[skills],
)

That configuration is intentionally narrow. The release post’s sample uses the same split: trusted read-only loading can be unattended, while scripts still need approval.

A production policy: decide the boundary before writing the skill

Skill operation	Good first policy	Why
Advertise skill name and description	Automatic	It is routing metadata, not execution.
Load reviewed instructions	Automatic only for a curated catalogue	An unreviewed instruction can redirect behavior.
Read a static policy or template	Automatic if tenant-scoped and non-sensitive	It should not disclose cross-tenant or secret data.
Run a script	Require approval by default	Scripts cross from guidance into action.
Write, deploy, or call external systems	Separate tool with its own least-privilege policy	A skill is not an authorization system.

This is especially important because the stable release supports file-based, class-based, and code-defined skills. For file-based scripts, the framework delegates execution to a runner you provide, so sandboxing, resource limits, and audit logging remain your responsibility. Microsoft explicitly recommends reviewing skill content, sandboxing file-based scripts, and logging skill, resource, and script use before deployment. Source

Make selection observable

Progressive disclosure can reduce unnecessary context, but it introduces a new failure mode: the agent may select the wrong skill or never load the right one.

Log these fields for every run:

request and selected skill IDs
skill version or content hash
whether the agent loaded instructions, read a resource, or requested a script
approval decision and actor
tool arguments, result class, latency, and failure reason

Then turn real mistakes into regression cases. If an agent loads a deployment skill for a documentation request, add that request to an evaluation set and assert both the expected answer and the absence of the wrong skill call. This extends the trace-to-test loop from Stop Replaying Coding-Agent Bugs by Hand: Turn Traces Into Regression Tests.

What this does not solve

Skills are not a prompt-injection firewall. A malicious document, a compromised shared skills repository, or overly broad tool credentials can still steer behavior or cause damage. Filtering which skills are exposed helps, but it is not a substitute for tenant isolation, signed/reviewed content, sandboxing, and per-action authorization.

It is also not automatically cheaper. A poorly described catalogue can trigger extra model turns to discover and load skills. Measure task success, tool calls, and end-to-end cost before claiming a context-saving win.

For coding agents that can execute shell commands, keep the execution boundary explicit. The same defense-in-depth principle applies in Your Coding Agent’s Approval Prompt Is a Security Boundary, and so does turning risky behavior into auditable tests in Your Coding Agent Is Tripping EDR Alarms (And the Rules Are Right).

What to do now

Pick one read-only, repetitive workflow with stable source material.
Package its instructions, references, and script separately instead of pasting them into a global prompt.
Start with a curated catalogue and approval for every script.
Add sandbox limits, immutable skill versions, and an audit event for every load/read/run.
Run 20 representative requests and compare success rate, wrong-skill rate, latency, and cost against the mega-prompt baseline.
Only then consider unattended execution for a narrowly scoped, idempotent script.

The broader lesson is simple: treat a skill as a versioned dependency with an execution boundary, not as a friendly folder of prompts. That makes specialized agent knowledge reusable without quietly turning every agent into an all-access automation bot.

Sources

How would you decide which of your agent’s skills may load automatically, and which should always require a human approval?

DEV Community