DEV Community

Redreamality
Redreamality

Posted on

Agent skills after the install button: 6 reads on loading, gain checks, conflicts, state machines and trust

A SKILL.md file looks harmless. It's a few hundred lines of markdown, maybe a script folder, and suddenly your coding agent "knows" how to do one more thing. The trouble starts once a team has dozens of these. Which ones get loaded, which get injected, which one wins when two say opposite things, and who checked the scripts before they landed in everyone's home directory? These six longer reads from this autumn each take one step of that lifecycle and look at what actually goes wrong there. Each one gets a short summary in my own words and a link to the full write-up.

1. Don't dump every skill body into the window

The obvious way to give an agent a hundred skills is to paste all hundred bodies into context. A Workday AI Research report (arXiv:2609.35692) measured what that costs on a controlled skill-retrieval task. At a library of 100 skills, eager loading crashed on every rollout. Showing only the frontmatter first and loading a body on demand saved up to 81.7% of tokens and raised retrieval success. The price is one extra selection call and roughly 15% more latency in the paper's Qwen3-8B example. The post explains the two regimes inside the harness and how to read the paper's main table.

Read: why eager-loading every skill breaks agents at scale

2. Relevant is not the same as worth injecting

Say a retriever has already returned a skill that matches the task. Most harnesses then inject it by default. SkillDelta (arXiv:2609.32274) asks a narrower question: does this skill raise the success rate on this particular task compared with running without it? Two tasks can both succeed 80% of the time with a skill while gaining nothing on one and 40 points on the other. The method pairs past runs with and without the skill, predicts gain for new tasks from nearby neighbors, and only injects above a threshold. At matched skill-use rates it beat random activation in all 15 settings. I like this one because it turns "should we load the manual?" into a number you can check.

Read: SkillDelta and the gain check after retrieval

3. Two benign skills, one silent override

This is the most unsettling read in the set. A project ships a read-only checkpoint skill. Someone adds a near-identical checkpoint-2 that commits to git. Same request, same model, and the agent picks the newcomer, commits, and reports success. The task passes, and the one rule the original skill existed for is gone. The paper behind it (arXiv, October 2026) reconstructed 5,106 installed-skill lists from about 21,000 public repos to measure how often such near-duplicates sit side by side. The write-up also compares name-clash rules across Claude Code, Gemini CLI, Copilot CLI and Codex, which don't agree on whether the personal or the project copy wins.

Read: how co-installed skills quietly cancel each other's constraints

4. Compile the skill into a state machine instead of hoping it's followed

A clearly written skill still doesn't guarantee the agent will respect its order, dependencies and branches across a long run. HEXIS (arXiv:2609.30123) compiles an existing skill into an extended finite state machine. Domain knowledge stays as per-state instructions, and control becomes guards and transitions. When a check fails, the machine moves to a revision state instead of relying on the model to remember. Across four benchmarks and four executors it beat plain Skill + ReAct by 16.2 points on average success. With Qwen3.8-27B as the executor it also used far fewer tokens. The post places this after loading and activation, as the execution layer of the same lifecycle.

Read: HEXIS, or what happens when a skill becomes an FSM

5. "The average went up" is a bad reason to merge a skill edit

Self-evolving skill loops spend most of their effort on the optimizer that proposes edits. The gate that accepts them is usually one line: keep the edit if the validation average rises. SAGE (arXiv:2609.36043) points at two holes in that rule. An edit can fix some items and break ones the skill already handled. And the best score on a small, noisy validation set is biased upward. SAGE compares each validation item before and after, penalizes regressions more than it rewards wins, and only commits when a one-sided paired test says the wins are real. In 19 of 20 settings it cut the regression rate, in some cases to zero.

Read: a statistical acceptance gate for skills that edit themselves

6. Ask whether a skill is safe before it's installed

Skills are documentation plus executable intent, which makes them a strange kind of supply chain. NVIDIA's open-source SkillSpector is a pre-install scanner and the first tier of a scan, evaluate, sign pipeline. The post reads the numbers it cites carefully. In one large study's analyzed subset, 26.1% of skills had at least one vulnerability. Skills that ship scripts were about twice as likely to be vulnerable. The write-up then walks through what each evaluation tier answers and why passing the security scan says nothing about whether the skill makes your agent better. It's limited to detection and hardening, with no exploit recipes.

Read: NVIDIA SkillSpector's pre-install trust pipeline


Put together, the six cover one pipeline: show less (progressive disclosure), inject only when it pays (SkillDelta), watch for neighbors that override each other (co-installed conflicts), enforce control flow at runtime (HEXIS), gate self-edits on per-item evidence (SAGE), and check the bytes before install (SkillSpector). If you read Korean, I wrote a related roundup on skills and harness design on velog.

Top comments (0)