Existing trajectory‑based methods require environment interaction, and prior approaches have achieved only modest performance gains. Mining over a million atomic operations from open‑source repositories shatters that ceiling, lifting average benchmark scores by 11.7% [1].
Before Code2Skill, two dominant routes existed: trajectory‑based synthesis, which requires costly environment rollouts, and document‑derived skill extraction, which often lacks executable evidence and rigorous verification [1].
Code2Skill processes 19,769 popular GitHub projects into a verified repository of 1,006,822 atomic‑operation records, each annotated with workflow, boundary, provenance, and source‑evidence metadata, collectively called the CodeSkillBank [1].
When agents retrieve skills from CodeSkillBank, their macro‑average score climbs from 42.90 to 47.90—a five‑point jump representing an 11.7% improvement—and the enhancement appears in 57 of the 72 protocol‑matched evaluations across nine model settings and eight benchmarks [1].
On all seven shared downstream benchmarks, Code2Skill ranks first with an average score of 49.5, outpacing even an oracle that selects the best result among trajectory‑based banks (Trace2Skill, ExpeL, SkillRL‑Bank) by 9.5 points (oracle avg 40.1) [1].
The study’s validation hinges on source‑body‑blind reconstruction and source‑aware comparison, limiting confidence to the reported 72 protocol‑matched evaluations; moreover, AI‑generated code achieves only a marginally higher pass rate (93.50% vs 93.00%) than human code, leaving open whether the bank will generalize beyond the tested domains.
If these findings hold, practitioners should replace custom trajectory‑derived skill collections with the CodeSkillBank for new tasks and re‑run existing agentic benchmarks using repository‑derived skills as the default procedural knowledge source.
Top comments (1)
Official Platform Update
Security protocols have been updated for all developer accounts.