Reading the reef README, the detail I would study before any training recipe is a response header. Reef inference responses include x-reef-agent-record-id, and the README calls its value a receipt. A later report names the receipts it evaluates, and Reef matches that feedback to the recorded interactions. That link between what the agent did and how it gets judged is where anyone running this in front of users should start asking about review and rollback.
The README describes Reef as open-source infrastructure for continual self-improving agents, then gets specific about the mechanics: Reef connects agent inference, feedback, learning, and versioned delivery, and it can target either model weights or an agent's harness, including its prompts, rules, and skills.
A receipt on each inference response
Reef exposes an inference endpoint that the README describes as OpenAI- and Anthropic-compatible, taking the provider's own request body at /v1/chat/completions and /v1/messages. Each request carries an x-reef-scenario header. A new scenario name creates a scenario using the deployment's configured recipe, and the README states that requests do not select recipes.
The response adds the receipt header. Your code then posts to /reef/report, and a report can carry a numeric score, a feedback field holding plain text or a structured object, and references listing the receipts being judged. The endpoint validates reports against a report schema.
Because the receipt travels with the response, the judgment does not have to happen inline. The harness example tells you to run your tests first and then report the actual result. The recipe table also lists a "Learning from usage" category for real interaction where "no one reports a score or feedback arrives late."
The loop in four steps
The README splits each learning cycle into four steps and names the modules behind each. Serve handles agent requests and records interactions (service/, runtime/). Observe matches feedback to recorded interactions and handles eligibility (storage/records.py, train/processors/). Grow produces an update from eligible records through a recipe (recipe/, train/). Commit applies the configured selection policy and publishes accepted updates, with candidate evaluation in train/evaluation/, version history in artifact/, and delivery in surface/.
The two learning surfaces use that loop differently. On the weight path, the SAO example deployment uses each eligible scored rollout to run a training step. Per the README, once the recipe has enough feedback it trains, synchronizes the updated weights to the serving runtime, and later requests use the current version without restarting Reef. That path needs a trainable model and a supported GPU stack.
The harness path needs a model endpoint instead of training GPUs. The harness-evolve recipe points at an upstream provider, which is a local Ollama server in the example. A reef-pi harness installed from the running server then executes tasks and sends reports such as reef-pi report --score 0 --feedback "missed the empty-token case".
Where the gate sits
For an operator, the question is what stands between a bad report and a changed agent. The README answers it for the harness tutorial: failed reports trigger a candidate skill update, Reef evaluates that candidate against the current harness on the tutorial's three coding tasks, and it publishes the candidate only if it wins. The README points to the tutorial for customizing the tasks and the evaluation. If you adopt this path, those tasks are part of the publication decision, so they deserve the care you would give a test suite.
For weights, the README's SAO walkthrough describes training and synchronization but does not spell out an evaluation step before new weights reach serving. The general Commit step does mention a configured selection policy, so the architecture has a place for a gate. What the SAO example puts there is not described in the README text I worked from.
What it says about review and rollback
Reef keeps versions. Its comparison table lists version management and staying live through updates as abilities that an inference engine or an RL training framework lacks on its own. The Commit step writes to a version history, and the installation note says artifact and checkpoint functionality requires the git-lfs system package, with Reef initializing Git LFS locally for its artifact repositories.
The part of the README I read does not describe operations built over that history. I found no command there for rolling back to an earlier version, no way to pin a scenario to a known-good artifact, and no human approval step before an accepted update is published. Before relying on rollback in production, read the docs or the artifact/ module.
Getting a feel for it
The package installs with uv pip install reef-infra, and the README says to use a source checkout for the training examples. The minimal starting point runs Reef as a pure inference server with uv run reef serve --inference.model-path Qwen/Qwen2.5-1.5B-Instruct. If you want to watch the gate work without training GPUs, the documented harness example uses a local Ollama model, and a failing report is what sets a candidate update in motion.
GitHub: https://github.com/Human-Agent-Society/reef
Curated by Agent Palisade — practical AI for small and mid-sized businesses.
Top comments (0)