Reef: The Infra That Lets Agents Get Smarter Over Time

Most AI agents hit the same ceiling: no matter how capable, they're only one-shot smart. They don't learn from past runs, they don't get better with use — the model weights are frozen, the prompt is hardcoded, the capability is frozen.

Reef (2633 stars, Apache-2.0, Python, by Human-Agent Society) aims to be the base layer for that. Its one-liner: continual learning infra for self-improving agents.

The problem it attacks

Reef is the first open-source infrastructure for continually self-improving agents. It connects the four stages into a loop:

inference → feedback → learning → versioned delivery.

Concretely, it lets you:

• Train model weights with Slime + SGLang — not just tune prompts, but actually improve the model;
• Improve the agent's harness — prompts, rules, and skills, the "behavior layer";
• Deliver versioned — every improvement is versioned, rollback-able, traceable.

In short: turn an agent from one-shot smart into accumulating smart.

Three signals it's worth watching

• It fills the emptiest slot in the agent space. Plenty of people build agent inference, tool-calling, and even memory — but the self-improvement loop, what to do with feedback after inference, is barely covered. Reef is the first open-source project to do it systematically.
• The authors have an academic pedigree. Human-Agent Society is an active agent-research org; there's theory and papers behind it, not just a demo. "Sourced" projects like this usually outlast "peaked-at-launch" ones.
• Learning isn't just prompt-tuning. It explicitly supports training model weights (Slime + SGLang) and improving the harness — covering both the model layer and the behavior layer. That's what separates it from the many prompt-engineering-only projects.

The honest caveat

It's early, and it's infrastructure, not a turnkey product — 2633 stars, 199 forks. You get a framework and tools, not a black box that makes your agent smarter on install. Running it well needs agent-engineering experience: how feedback is collected, when learning triggers, how versions are managed. And "agent self-improvement" itself is still very early; don't expect it to transform an ordinary agent overnight.

I've localized the README and core docs to Chinese: https://github.com/yangshun2005/reef-cn

If you find this project useful, a star on the original repo supports the author's ongoing maintenance.