Disclosure: I work on this project. Every number below was read out of the source, and I have included the file and line so you can check it rather than take my word for it.
Most AI coding tools fail before they write a single line of code. The prompt was vague, and the model quietly filled the gaps with assumptions you never agreed to.
You ask for "a task management CLI." The model picks a data model, a priority scheme, a persistence layer. All reasonable, none of them yours. You find out three files in, during review, and you rework it. That's the loop most of us are stuck in: prompt, guess, rework, repeat.
Ouroboros is an open-source Agent OS that fixes the input instead of the output. It's a local-first runtime layer that sits in front of Claude Code, Codex CLI, OpenCode, Gemini CLI, GitHub Copilot CLI, Kiro, Hermes, Pi, and Zcode, and replaces ad-hoc prompting with a five-stage, replayable workflow: interview, seed, execute, evaluate, evolve.
The real problem is unclear intent
Ouroboros' own framing of this is a simple table:
| Problem | What happens | Ouroboros fix |
|---|---|---|
| Vague prompts | AI guesses, you rework | Socratic interview exposes hidden assumptions |
| No spec | Architecture drifts mid-build | Immutable seed spec locks intent before code |
| Manual QA | "Looks good" isn't verification | 3-stage automated evaluation gate |
The fix targets clarity, not capability.
The loop
Interview -> Seed -> Execute -> Evaluate
^ |
+---- Evolutionary Loop ----+
- Interview: Socratic questioning surfaces the assumptions you didn't know you were making.
- Seed: your answers crystallize into an immutable specification: acceptance criteria, ontology, constraints.
- Execute: the seed runs through a Double Diamond decomposition (Discover → Define → Design → Deliver).
- Evaluate: a 3-stage gate: Mechanical (free, deterministic checks) → Semantic → Multi-Model Consensus.
- Evolve: the evaluation output feeds back into the next generation's seed, and the cycle repeats until the system stops learning anything new.
Each cycle is meant to converge, not just repeat. The stopping condition isn't a timer or a step count. It's math.
The interview ends when the math says so
This is the part I found most concrete. Ouroboros scores ambiguity as the inverse of weighted clarity across four dimensions (goal, constraints, success criteria, and context for existing codebases):
Ambiguity = 1 - Sum(clarity_i * weight_i)
A greenfield example from the README:
Goal: 0.9 * 0.4 = 0.36
Constraint: 0.8 * 0.3 = 0.24
Success: 0.7 * 0.3 = 0.21
------
Clarity = 0.81
Ambiguity = 1 - 0.81 = 0.19 <= 0.2 -> Ready for Seed
Above 0.2 the system keeps asking instead of letting you start on a foundation it thinks is shaky. The threshold is 0.20 at auto/interview_driver.py:125, and the weights are 0.40 / 0.30 / 0.30 at bigbang/ambiguity.py:48-50.
You can overrule it. Passing force=true bypasses the gate deliberately, and the code says so in as many words: "force=True intentionally bypasses BOTH the ambiguity threshold" (mcp/tools/authoring_handlers.py:1378). So this is a default that argues with you, not a lock. I think that is the right call, because a gate you cannot override eventually gets worked around in worse ways. But it does mean the guarantee is softer than "it will not let you."
The evolutionary loop has a matching gate on the way out. It converges when ontology similarity between the last two generations reaches 0.95, and only if that generation's evaluation was approved: high similarity with a rejected evaluation does not count (evolution/convergence.py:144, :148). A separate detector watches for stagnation: if that similarity sits unchanged for three straight generations, the loop stops as "not progressing" rather than calling it converged (convergence.py:54), and oscillation and repetitive-feedback detection run alongside so it doesn't spin on a question it already answered.
The weights are hardcoded. 40% goal, 30% constraints, 30% success criteria. That is somebody's judgment call rather than a derived constant, and it is worth knowing that before you trust the number. What I like is that it is an arbitrary you can go read and recompute, instead of one buried in a prompt.
How it actually runs
The installer auto-detects which supported runtime you're using (Claude Code, Codex CLI, GitHub Copilot CLI, OpenCode, Hermes, Gemini, Kiro CLI, Pi CLI, Zcode) and registers the MCP server where the host supports it. Everything after ooo interview (seed generation, execution, evaluation, the evolve loop) is driven from inside that same session. There's also a plain ouroboros CLI for the terminal directly (ouroboros run seed.yaml, ouroboros status executions, and so on).
ooo ralph runs the evolutionary loop across session boundaries. If your machine restarts mid-loop, it reconstructs the lineage from an event store and picks up where it left off rather than starting over.
What it does not fix
It will not make the first attempt better code. The model is the same model. What changes is that the input to that attempt is something you agreed to out loud, and there is a paper trail (the seed, the ledger, the evaluation stages) to read afterward instead of reconstructing what you meant from a diff.
It also costs you the thing some people came for. If your actual want is to type one line and walk away, an interview that keeps asking is friction, and calling that friction a feature does not make it stop being friction. The bet is that the questions are cheaper than the rework. That bet is wrong for a throwaway script and I would not use it there.
Try the part that takes 30 seconds
You do not have to install anything to judge the idea. Open the scoring code and see whether you agree with the weights:
bigbang/ambiguity.py:48-50: 40% goal, 30% constraints, 30% success criteria.
If you want to run it:
curl -fsSL https://raw.githubusercontent.com/Q00/ouroboros/main/scripts/install.sh | OUROBOROS_INSTALL_REF=devto bash
Then, inside your agent session, run these in order (ooo setup is a one-time step):
> ooo setup
> ooo interview "I want to build a task management CLI"
The interview is the whole pitch. If it asks you something you had not decided yet, that is the product working. If it asks you three things you had already written in the prompt, that is a bug and I would like to see the transcript.
MIT, Python 3.12+, runtime guides per CLI: github.com/Q00/ouroboros.
The question I actually want answered
Everybody agrees vague prompts are the problem. Almost nobody agrees on who should fix it. Three positions I keep running into:
- The model should ask. Clarification is the model's job and a harness that does it is a workaround for a weak model.
- The harness should ask, because you want the same questions every time regardless of which model is behind it.
- Neither should ask. Write a better prompt.
I built around position 2 and the ambiguity score is what that position looks like in code. If you hold 1 or 3, I would rather hear the argument than the star.
Where do you catch it today? During the interview, at review, or three files in?
Top comments (7)
I like the choice to attack ambiguity before execution. The part I would want to see measured is whether the clarity score predicts fewer rollback cycles after the first seed, not just a cleaner prompt. If it does, that becomes a useful control surface rather than another prompting ritual.
That is the measurement I do not have, and you have put your finger on the exact
gap.
What exists today is the score and the gate: ambiguity is scored as the inverse of
weighted clarity across goal, constraints and success criteria, and a seed is
refused below 0.2 unless you force past it. What does not exist is any link from
that score to what happens afterwards. Nothing in the system currently records
"this seed started at 0.31, and here is how many generations it took to converge"
in a form you could regress on.
The data is not obviously missing, which is the frustrating part. Every run leaves
a lineage with per-generation ontology similarity, and the evolution loop records
why it stopped: converged, stagnant, oscillating, budget exhausted. So the two
halves exist in the same database and nobody has joined them.
My honest expectation is that the correlation would be weaker than the design
implies. The score measures how well you answered its questions, not whether the
answers were true. Someone confidently wrong scores well. That failure mode would
show up exactly as you describe: a clean prompt, and rollbacks anyway.
If I get to run that analysis I will publish the result whichever way it comes out.
It is more interesting if the gate turns out not to predict anything, because that
is a claim currently doing work in the README.
Correcting myself before anyone else has to: I wrote above that a seed is refused below 0.2, and that is backwards. The score measures ambiguity, so generation is blocked while the score is above 0.2, and passing force explicitly is the way through the gate. The article body had the same error and was fixed days ago; this comment kept it, and a comment is just as live as a paragraph. Source: AMBIGUITY_THRESHOLD in src/ouroboros/bigbang/ambiguity.py.
That correction matters. A threshold sign error changes the whole governance story, because the gate is no longer a confidence filter, it is a refusal policy around unresolved ambiguity.
Refusal policy is the right name for it, and the sign error shows why naming matters: I had described a confidence filter, which sounds like the tool vouching for quality, when what ships is a refusal to generate a spec while ambiguity stays above the threshold. The remaining honesty note is the escape hatch: the refusal is a default, not a wall. A user can force past the gate explicitly, so the guarantee is narrower than "no vague seed exists". It is "no vague seed exists without someone having chosen it".
실행 전에 모호성을 수치화해 질문을 계속하고, Seed를 불변 명세로 잠그는 흐름이 인상적입니다. 특히 첫 결과의 품질보다 ‘무엇을 요청했는지 추적 가능하다’는 보장을 목표로 한 점이 실제 AI 개발의 재작업 원인을 잘 짚었습니다.
읽어주셔서 감사합니다. 하네스는 에이전트를 측정해야하며, 이를 통해 개선할 수 있습니다.
이 과정에서 제일 중요한건 replayable 하냐 입니다. 트레이스를 저장하고 에이전트의 trajectory를 읽으면 우리는 검증가능한 환경을 만들 수 있습니다.
이러한 에이전트한테 줄 수 있는 건 인간의 방향성을 모호하지않게 담아 seed로 주는것입니다.