SkillOpt, from Microsoft, trains a single markdown file the way you would train a neural network, with epochs, a batch size, a learning rate, and a validation gate. The weights never move. What changes is the text of a skill document that a frozen model reads before it acts.
That framing is the whole point of the repo. Most agent skills today get hand-written, generated in one shot by a strong model, or nudged along by loose self-revision. None of those behaves like an optimizer, and none reliably beats its own starting point once feedback arrives. SkillOpt treats the skill document as the trainable state of a frozen agent and puts it through a disciplined loop: rollout, reflect, aggregate, select, update, evaluate.
How the training loop works
A separate optimizer model reads scored rollouts and proposes bounded edits to one skill document: add, delete, or replace. A candidate edit only survives if it strictly improves a held-out validation score. That gate is the core rule. Anything that fails to move the validation number gets rejected and parked in a rejected-edit buffer, which keeps training from wandering. A textual learning-rate budget caps how much the document can change at once, and an epoch-wise slow or meta update keeps the whole process stable across passes.
The output is deliberately small. The deployed artifact is a best_skill.md file, usually 300 to 2,000 tokens, that runs against the unchanged target model. Because the skill is just text the model reads, deployment adds zero inference-time model calls. You pay the optimizer cost once during training, then ship a plain document.
What the numbers say, and what they do not
The README reports evaluation across six benchmarks, seven target models, and three execution harnesses: direct chat, the Codex CLI, and the Claude Code CLI. SkillOpt is best or tied-best on all 52 evaluated cells. On GPT-5.5 it lifts average no-skill accuracy by +23.5 points in direct chat, +24.8 inside the Codex agentic loop, and +19.1 inside Claude Code. The authors also claim the trained skills transfer: across model scales, between the Codex and Claude Code harnesses, and to nearby benchmarks without further optimization.
Those are the repo's own figures, drawn from the linked paper. The README does not print per-cell tables, so treat the headline lifts as the authors' reported results rather than independent measurements.
Sleep mode and cross-tool reach
Version 0.2.0 adds SkillOpt-Sleep, a nightly offline engine that runs while the agent is idle. Its loop is harvest, mine, replay, consolidate, all behind the same held-out validation gate. It brings a multi-objective reward, experience replay with dream rollouts, and long-term memory, and it ships as a skillopt-sleep CLI. The premise: a local coding agent reviews its past sessions overnight, replays recurring tasks, and folds only validated skills back into its document.
The same release widens the tool surface. SkillOpt now carries backends and plugin shells for Claude, Codex, Copilot, Devin, and OpenClaw, plus Windows robustness fixes and hardened JSON parsing. The framework already supported OpenAI, Azure, Claude, Qwen, and MiniMax as chat backends. Adding another is a defined contract: drop a backend module under skillopt/model/, register it, and wire it through the router. Adding a benchmark follows a similar pattern with a dataloader.py, a rollout.py, and an initial.md seed skill. An optional Gradio WebUI lets you watch training.
The bet SkillOpt makes is worth sitting with. If a short markdown file, trained with real optimizer discipline, can lift a frozen model by twenty-odd points and then carry over to other models and harnesses, the skill document starts to look less like documentation and more like a portable, cheap layer of learned behavior that never touches a single weight.
GitHub: https://github.com/microsoft/SkillOpt
Curated by Agent Palisade — practical AI for small and mid-sized businesses.
Top comments (0)