I built an open-source "CI for AI agents", and here is why I went open-source

#ai #webdev #opensource #showdev

Hi, my name is Max. I built Reelier, an open-source tool that gives AI agent workflows the thing normal code has had for decades: snapshot tests and CI.

Here is the itch.

My coding agent kept re-deriving the same tool-call workflow on every run.
Fetch this, transform that, post the result.
Same tokens, same seconds, for a task with exactly one correct answer.

And every so often it did it a little differently, and nothing caught it. That second part is the scary one.

Why open-source it

Agent tooling is flooded right now. New closed tools ship every day, and most of them ask you to trust a black box with your workflows and your keys.

Open-source is how a small tool earns trust in that noise: you can read exactly what it does, run it on your own machine, and keep your data as plain files.

What it actually does

You record one agent run that worked.
Reelier compiles it into a readable SKILL.md file, a recipe with typed inputs, typed outputs, and an assertion on every step.
Then it replays that file deterministically: no model call, 0 tokens, byte-identical, in milliseconds.
Every replay is a receipt.

And reelier diff compares two runs and exits 1 when a step drifts, so a silent regression fails your build instead of reaching production.

The honest part

It only replays deterministic tool calls, HTTP and MCP.

It does not replay file edits or free-form reasoning, and it never fabricates a step it cannot reproduce. It is for the deterministic half of what agents do.

I say that up front because the version that would go viral (record my whole agent) is the version I will not claim, since it would not be true.

Numbers, from the benchmark in the repo: 1000 of 1000 replays byte-identical, 0 tokens per replay, about 50x cheaper and 59x faster than the agent re-running the same task.