When Scarab Diagnostic Suite started taking shape, I thought I was building diagnostics.
That was the obvious word for it.
AI coding agents were changing code quickly. Repos were drifting. Bugs were appearing in strange places. Tests were passing without proving the right thing. Files were taking ownership of behavior they should not own.
So the first question was simple:
Can a system drop into a messy repo, locate the real failure surface, and identify the narrow repair lane without making everything worse?
That is where Scarab started.
And that mode matters.
Because anyone working seriously with AI-assisted software development has seen the pattern by now.
The agent patches one issue.
The patch disturbs another part of the system.
The agent patches that.
Then another hidden assumption breaks.
Then a test gets adjusted.
Then a workaround becomes structure.
Then, eventually, the repo is "working" in a way that no longer fully resembles itself.
The system runs.
But the truth drifted.
The failure class
That is the failure class Scarab Systems is built around.
Not just broken code.
Software drift.
Boundary failures.
Repo-truth misalignment.
Verification gaps.
Entropy.
The quiet moment when the code appears functional, but the repo has moved away from its own architecture, obligations, and intended shape.
Over the last few weeks, Scarab has been field-tested across more than 30 public software failure surfaces, including patches and diagnostic reports against major open-source platforms.
That public field record matters because it proves the first mode:
Scarab can enter a live failure surface, identify the boundary that stopped preserving truth, and guide a narrow repair lane that does not casually break the rest of the repo.
But the more interesting discovery was not only that Scarab could help find bugs.
The more interesting discovery was that truthful repair behaves differently.
Truthful repair behaves differently
Most patching work tends to chase symptoms.
A bug appears here.
A workaround appears there.
A test gets adjusted somewhere else.
The repo slowly becomes a negotiation between visible errors and local patches.
But when a repair is aligned with the repo's actual truth, something else happens.
The repair does not just silence the visible failure.
It moves the repo closer to itself.
That distinction matters.
A repo does not need to become some abstract "working version" of itself.
It needs to preserve the system it is actually obligated to be.
That is why I stopped thinking about Scarab as only a diagnostic tool for individual failures.
The second mode is stepwise repair.
Stepwise repair, not entropy management
In messy AI-assisted repos, failures are not always isolated.
They collect in hot spots.
One bug may be the visible edge of a deeper boundary problem.
One test failure may reflect a responsibility that moved.
One runtime issue may expose a false assumption that has spread.
One generated artifact may have quietly been treated as source truth.
If the coding agent repairs one bug at a time without understanding the repo's truth boundaries, it can go in circles forever:
- Patch the bug.
- Patch what the patch broke.
- Patch the new workaround.
- Patch the test.
- Patch the side effect.
- Patch the patch.
That is not repair.
That is entropy management.
Scarab is being developed around a different theory:
If repairs happen along the repo's truthful boundaries, the system can be brought step by step toward quiet.
Not quiet because the errors are hidden.
Quiet because the repo is becoming more coherent.
That is a very different thing.
A truthful boundary repair can ripple outward in a good way.
It can reduce pressure on nearby failure surfaces.
It can clarify ownership.
It can remove the need for a workaround.
It can make a test meaningful again.
It can restore the difference between source truth and downstream output.
It can bring the system closer to its baseline instead of dragging it into a patched abstraction of itself.
That is the part I think matters for the future of AI coding agents.
Scarab does not replace the coding agent
The next serious leap is not simply making agents faster.
They are already fast.
The next leap is making sure they do not destroy coherence while they move.
But this part is important:
Scarab is not trying to replace the AI coding agent.
It is not another model.
It is not a second developer.
It is not a magical correctness oracle.
Scarab Diagnostic Suite is a technical diagnostic layer that produces the right kind of repo-grounded findings for the coding agent to act on.
The agent is still the implementer.
The human still gives intent.
Scarab reads the repo, identifies the relevant truth surfaces, exposes the boundary conditions, and returns evidence-backed findings that help the agent stay inside the right lane.
That is why the system can be software-agnostic and agent-agnostic.
It does not need to own the repo.
It does not need to replace the workflow.
It does not need to care whether the developer is using Codex, Claude Code, Cursor, Copilot, Devin, a local model, or a human engineer.
The role is simpler and more technical:
- Give the implementer better findings.
- Make the repo's truth easier to see.
- Keep the repair or build path aligned with the system that already exists.
The agent should not grade its own correctness
True autonomous coding agents cannot be built on guesswork alone.
They need a deterministic layer outside the model.
A layer that can tell the agent:
- This surface owns this behavior.
- This boundary cannot move casually.
- This artifact is not source truth.
- This test proves this claim.
- This config carries this runtime obligation.
- This repair lane is narrow.
- This change preserved the repo.
- This change drifted.
The agent should not have to invent the repo's architecture every time it opens a task.
It should not have to decide what is canonical from whatever happens to fit in context.
It should not grade its own correctness.
It should not silently move system boundaries and then announce that the build passed.
That is too much authority for a probabilistic worker inside a complex system.
The repo needs its own governed relationship to truth.
Three modes of Scarab
That is where Scarab is moving next.
Right now, I see three operating modes taking shape.
1. Public field diagnostics
Enter a failure surface.
Identify the boundary.
Produce a narrow repair lane.
This is the easiest mode to see publicly because it shows up in open-source issues, patches, and field reports.
A repo has a visible failure.
Something drifted.
Scarab asks:
What truth was this system supposed to preserve?
Which boundary stopped preserving it?
What evidence proves where that happened?
2. Stepwise repo quieting
Move through hot spots in a sequence that brings the system closer to coherence instead of chasing symptoms forever.
This matters because messy repos do not always fail one bug at a time.
They often fail around pressure points.
If repair follows the repo's truthful boundaries, one good repair can lower pressure elsewhere.
Not because the system was magically fixed.
Because the repair moved the repo closer to its actual obligations.
3. Active agentic governance
This is the mode I am working toward now.
Continuous monitoring while AI-assisted development is happening.
Not just:
What broke?
But:
- What is the agent about to build on?
- Is it building on repo truth or residue?
- Is this feature strengthening the system or introducing a hidden drift surface?
- Did this layer preserve the boundaries beneath it?
- Did the repo become more coherent after the change?
That is the future I am interested in.
The missing layer
Humans describe intent in real language.
Repos preserve truth mechanically.
AI coding agents operate inside governed boundaries.
Diagnostics verify whether the system stayed coherent.
Each new feature should not make the repo heavier, stranger, and harder to trust.
Each layer of work should strengthen the repo as itself.
That is the real promise of AI-assisted development.
Not just more code.
Not just faster code.
Not a world where humans spend all day babysitting the thing that was supposed to give them leverage.
A world where the agent can move quickly because the operating boundary is clear.
A world where the human does not have to manually hold every architectural truth in their head.
A world where the repo can tell the agent what must remain true.
That is the layer I think serious autonomous software development is missing.
And that is what Scarab Systems is being built to explore.
The public field reports are not the end of the story.
They are the proof trail.
The bigger implication is that repo-side deterministic governance may be one of the missing foundations for true autonomous coding agents.
Because autonomy without truth is just acceleration.
And acceleration without truth is drift.
Top comments (16)
Strong overlap with something I just published on the same axis: dev.to/jugeni/vibe-coding-is-not-a-level-its-an-axis-12gb
You're solving the tooling side — deterministic governance layer that catches drift. I framed it as a missing horizontal axis next to the autonomy ladder ("L1 + High operator discipline > L5 + Low operator discipline"). Different vocabulary, same underlying claim: the model alone can't be the source of truth about the repo.
The line "agent should not have to invent the repo's architecture every time it opens a task" is exactly the failure mode that locked decisions and source-anchored notes are designed to kill. Different angle of attack on the same problem.
Curious how Scarab handles the capture habit — in my setup that's the hardest part, not the schema.
I think this is a useful overlap, but I’d separate the layers a little more sharply.
What you’re describing feels to me like operator-side continuity: decisions, notes, locked choices, provenance, and context that survives across sessions. That can absolutely reduce relitigation and make the model less wobbly over time.
But I don’t think that is the same problem Scarab is aimed at.
Scarab is not really trying to improve the operator’s memory discipline or create a better note store for the model. The target is repo-side technical governance: what the codebase itself can expose as evidence, boundaries, obligations, drift surfaces, source-truth relationships, and verification findings.
So when you ask about the capture habit, my answer is that Scarab tries to move capture away from “the operator remembered to record the decision” and toward diagnostic events inside the repo.
For example:
What boundary did the repo show was responsible?
What test actually proves the claim?
What artifact is source truth vs downstream output?
What config or runtime obligation must remain true?
What failure surface shows drift?
What verification result confirms the repo moved closer to coherence?
That is a different kind of state from session memory.
It is less “the model should remember what we decided” and more “the repo should be able to produce governed findings the implementer can act on.”
That distinction matters because operator-side capture can become its own drift surface. If the human captures the wrong abstraction, scopes it too broadly, locks the wrong decision, or fails to update it, the model may become more consistent around the wrong thing.
So I agree with the underlying claim that the model alone cannot be the source of truth.
But I’d frame Scarab as a different answer to that problem:
The coding agent remains the implementer.
The human still gives intent.
Scarab reads the repo and produces repo-grounded findings.
Those findings help govern the lane the agent works inside.
So the question I keep coming back to is:
What truth should be captured by the operator, and what truth should be mechanically surfaced by the repo itself?
Because for autonomous coding agents, I don’t think session continuity is enough. The repo needs its own governed relationship to truth.
Thanks — the layer separation is the move. I hadn't articulated the "capture becomes its own drift surface" risk as sharply until you named it.
Split criterion I keep coming back to: operator captures intent, repo surfaces evidence. If the code can prove it, the operator shouldn't have to remember it. If the code can't recover it, no one else will. Concretely:
Operator-side — the slice code can't surface:
Intent — why this trade-off, not the other. Code shows what, never why.
Rejected paths — considered and dropped. The codebase only contains chosen branches; rejected ones disappear and get re-litigated by the next operator.
Locked invariants — binding decisions that outlive any implementation. "We never X" isn't in the code, but it constrains every PR.
Process state — what's in flight, blocked, waiting on a vendor. The repo has no concept of "thread parked since Tuesday."
Implicit-by-convention — practices that show up as effects in code, never as rules.
Repo-side — your domain, named: boundaries, source-truth chains, test coverage as proof, runtime obligations, drift surfaces between docs/config/code, verification findings.
Where the layers should meet is the handoff. Three interfaces I think both sides need:
Decision → diagnostic peg. A locked decision carries a verifiable_by pointer at the test or scan that confirms it. If the diagnostic breaks, the decision auto-flags as stale instead of silently codifying a lie.
Repo finding → proposed decision. A drift surface in code generates a proposed decision the operator accepts, rejects, or locks. Capture starts from evidence, not memory.
Backward pointer. Every accepted decision points at the artifact that grounded it. Audit trail in both directions.
Honest gap on my side: what I'm shipping today handles operator-side only. The handoff is a feature absence, and the question you closed with is the one I keep parking. For autonomous agents the answer probably isn't either layer alone — it's whether the bridge between them exists. That's the most interesting problem in this space right now.
Yes — I think “operator captures intent, repo surfaces evidence” is a useful split.
Where I’d push the frame one layer further is that Scarab is not mainly about the handoff between notes and diagnostics. That bridge matters, but the larger problem I’m working on is the governance interface between the repo and the AI coding agent.
That is the layer I keep coming back to.
The coding agent should still do what it does best: implement.
The human still gives intent.
But the repo needs a deterministic interface layer that can surface what must remain true before, during, and after the agent acts.
So the question is not only:
What should the operator remember?
Or even:
What should the repo prove?
The deeper question is:
What governed findings does the agent need in order to act without drifting the system?
That is where Scarab is aimed.
A decision record can preserve why something was chosen.
A note store can preserve context across sessions.
A trace can point from intent to artifact.
But Scarab’s lane is repo-side governance: boundaries, source-truth chains, verification findings, drift surfaces, runtime obligations, and evidence-backed repair or build lanes that the coding agent can use without turning the conversation itself into the truth layer.
That distinction matters because the bridge cannot become the governor.
The operator layer can carry intent.
The repo layer can surface evidence.
But the governance layer has to determine what findings are valid enough to constrain agent action.
That is the part I explored more directly in my LinkedIn version of this article, where I framed Scarab less as a diagnostic tool and more as an operating layer for AI agents.
linkedin.com/pulse/i-thought-build...
The larger claim is still the same:
The agent should not have to invent the repo’s architecture every time it opens a task.
And the model alone cannot be the source of truth.
For autonomous coding agents, the missing foundation is not just better memory, better notes, or better continuity.
It is a deterministic governance interface between AI action and repo truth.
Three layers I'll take, with one push back: governance isn't a separate stratum sitting above operator and repo — it's a function that runs across both, and its inputs are deterministic but its threshold isn't.
The deterministic side is what you already named: boundaries, tests, drift surfaces, verification findings. The codebase can produce those without human authorship. But the question „which findings are valid enough to constrain agent action" is policy, not engineering. Coverage at 70 vs 80, a lint warning treated as blocker vs nice-to-have, a failing contract test counted against the build vs flagged — those are calls a human authored once and the governance layer enforces forever. Without that authoring, the same diagnostic surface produces two opposite agent behaviors in two different repos.
So I'd read the stack as: repo surfaces evidence (deterministic), operator authors policy (about which evidence binds), governance mechanically enforces the policy against agent action (deterministic again). The middle step is what keeps governance from being a black box — and it's where the operator-side notes I was describing earn their keep. „We never accept a PR with a regression on contract test X" is intent-as-policy. The codebase can't generate that. The governance layer can't act on it without it.
Which opens the deeper question: how do you stop policy itself from getting rewritten silently? Once policy drifts, the deterministic enforcement on top of it just executes the drift faster and more confidently. Same staleness problem Rapls flagged in the other thread, one layer up. Going to read the LinkedIn longform once the link resolves — the governance framing landed.
I agree that policy has to exist somewhere, and that humans author some of the binding thresholds. But I’d separate policy enforcement from diagnostic governance more sharply.
A coverage threshold, lint blocker, contract test, or “never accept X” rule can constrain action. That is policy.
But Scarab is not only asking whether a pre-authored policy fired.
The diagnostic question is deeper:
Is this evidence actually proving the repo truth it claims to prove?
A test can pass and still be theatrical.
A threshold can be met while the system drifts.
A locked invariant can be stale.
A policy can enforce yesterday’s wrong abstraction.
A scan can surface a signal without explaining whether the system is still coherent.
That is why I don’t treat governance as simply “operator authors policy, repo surfaces evidence, governance enforces.”
For Scarab, the governance interface has to evaluate the relationship between the repo’s truth surfaces, the evidence produced, and the agent action being constrained.
So yes: the operator can author intent or policy.
And yes: the repo can surface deterministic evidence.
But the diagnostic governance layer has to determine whether the evidence is valid, current, truth-bearing, and strong enough to bind the coding agent’s next move.
Otherwise the system is just enforcing drift more confidently.
That is the distinction I’m trying to hold:
Policy can tell the agent what rule to respect.
Diagnostics has to determine whether the rule, evidence, test, boundary, or artifact is still preserving the repo’s actual truth.
That is why I keep framing Scarab as a deterministic governance interface between AI action and repo truth, not simply a policy layer or a handoff between operator memory and repo signals.
The diagnostic-vs-policy split holds and you've named it sharper than I did. „Is this evidence actually proving what it claims to prove" is a different function than „does the agent respect the rule." Conceded.
Where I'd press is on the determinism claim. A theatrical test, a stale invariant, an enforced-but-wrong abstraction — those don't get caught by a Boolean check on the evidence. Catching them requires a criterion for what counts as truth-bearing in the first place. „This test must execute the path under production load, not just return assertion green" is itself an authored standard. „This invariant is preserved if and only if the migration script also runs on the staging dataset" is authored. The diagnostic layer evaluates evidence against that criterion deterministically — but the criterion is upstream, and someone wrote it.
Which collapses your closing line back into the same question, one fractal level deeper: how do you stop diagnostic criteria themselves from getting rewritten silently? Once „what counts as valid evidence" drifts, the deterministic truth-evaluation on top of it just executes confidently against a forged ground truth.
I don't think that's a bug in your framing — it's where operator-side authoring re-enters the picture, one layer further in than I had it. Three loci of authored intent, not one: policy (what the agent must respect), diagnostic criteria (what evidence is allowed to bind), lifecycle (when authored entries at either level go stale). The deterministic governance interface enforces all three. Without authoring at every level, determinism just means enforcing drift more confidently.
Worth flagging the meta: this thread itself is the demonstration. Two operators authoring policy for their respective systems across multiple turns, anchoring locked positions, surfacing where each other's framing was thin. The discussion is enacting what we're both claiming the discussion should be — probably the cleanest signal I've seen all week that the axis is real, not invented.
I think this is the place where I’d separate the frame again.
Yes, some policy is authored.
Yes, some diagnostic criteria are authored.
Yes, lifecycle rules matter.
But I would not make authored criteria the root of truth.
For Scarab, authored policy is one surface among several. It can guide action, but it can also drift, become stale, overfit yesterday’s failure, or preserve the wrong abstraction. So the diagnostic layer cannot simply inherit authored criteria as ground truth and enforce them harder.
That is actually the same problem one layer up.
A test is not automatically proof.
A policy is not automatically truth.
A diagnostic criterion is not automatically valid just because someone authored it.
A locked invariant can be correct, stale, incomplete, or actively wrong depending on what the repo is now proving under change.
So the diagnostic question is not only:
“What authored standard should this evidence be evaluated against?”
It is also:
“Is that standard still truth-bearing in relation to the repo’s current behavior, boundaries, artifacts, and obligations?”
That is where I think Scarab differs from a policy-as-code or operator-continuity model.
Policy can constrain agent action.
Tests can provide evidence.
Operator intent can explain why a direction exists.
But diagnostics has to evaluate whether those surfaces still preserve the repo’s truth, or whether they have become drift surfaces themselves.
So I’d frame the stack less as:
operator authors policy → repo surfaces evidence → governance enforces
and more as:
repo surfaces evidence
authored rules become inspectable governance surfaces
diagnostics evaluates whether the evidence and rules still preserve repo truth
governance constrains agent action based on validated findings
That distinction matters because otherwise “determinism” can become exactly what you named: enforcing drift more confidently.
The point I’m holding is that Scarab is not treating authored policy, tests, notes, or criteria as sacred roots.
They are all surfaces.
Some are human-authored.
Some are repo-emergent.
Some are runtime-proven.
Some are stale.
Some are theatrical.
Some are binding.
The diagnostic layer’s job is to tell the difference before the agent builds on them.
That is why I keep framing this as a deterministic governance interface between AI action and repo truth.
Not because humans author nothing.
Because whatever humans author still has to be diagnosed against what the repo is actually preserving.
The reframe lands and I'd take it: authored criteria aren't sacred either; they're inspectable surfaces with the same failure modes as evidence — stale, theatrical, wrong-abstraction. Treating policy as ground truth was exactly the slippage you were naming.
But moving everything onto the surface plane opens a question that has to live somewhere outside the gauntlet: what does the diagnostic layer use as its reference frame? Every check evaluates against something. If „does this evidence still preserve repo truth" is the test, then „repo truth" is the anchor — and „repo truth" is itself either (a) what the code currently does, in which case drift becomes definitional rather than detectable, (b) an emergent property of the codebase's invariants, which is itself a learned model with its own failure modes, or (c) authored at a meta-level that diagnostics inherits — which is where authoring quietly re-enters one floor up.
(a) collapses because the codebase under change is the thing we're trying to govern, not the standard we measure against. (b) is honest but pushes the boot-strap onto a separate problem — how do you trust the emergent invariants the system is producing about itself. (c) is what I'd been pointing at, except — agreed — without treating the meta-criterion as immune to its own staleness.
The frame I'd end on: diagnostics needs at least one fixed reference point per repo, and that point is itself an authored artifact the governance interface treats as the only surface not subject to its own evaluation — held under explicit version, audited on change, deliberately replaceable when it fails. Less „sacred root" than „load-bearing wall you don't move without a structural review." Whether that wall is an operator-side manifest, a repo's declared invariant set, or the LinkedIn-longform style explicit framing — it has to exist somewhere, or the diagnostic recursion has nothing to terminate against.
Yes — that is the right question to end on.
What is the diagnostic layer using as its reference frame?
That is exactly where Scarab lives.
A repo cannot be measured only against “what the code currently does,” because the current code may already be drifted.
It also cannot be measured only against operator memory, policy notes, or authored criteria, because those can drift too.
The reference frame has to be repo-local, inspectable, versioned, and governed.
That is one of the core reasons Scarab Diagnostic Suite exists.
Scarab is designed to establish and work against baseline repo truth: the project’s declared structure, boundaries, obligations, source-truth relationships, verification expectations, runtime surfaces, and evidence trails.
Then the diagnostic question becomes:
Where is that baseline truth still being preserved?
Where has it been contradicted?
Where has it become stale?
Where has an authored rule become residue?
Where has a test stopped proving what it claims?
Where has the agent built on a false center?
So I agree that there has to be a reference frame.
I would just avoid treating that reference frame as a sacred root.
In Scarab’s framing, even the baseline is governed.
It is versioned.
It is inspectable.
It is change-controlled.
It can be refreshed.
It can be contradicted by evidence.
It can be promoted, retired, or revised deliberately.
That is the difference between a static policy root and a governed diagnostic baseline.
The beauty of the system is that the repo is not left to define truth by whatever it currently happens to contain, and the operator is not forced to manually carry the whole truth layer in memory.
Scarab sits at that interface.
It records baseline truth, detects where that truth is broken or drifting, and produces repo-grounded findings the coding agent can act on.
That is why I keep calling it a deterministic governance interface between AI action and repo truth.
The agent still implements.
The human still gives intent.
But Scarab gives the system a governed reference frame so the agent is not building from conversational memory, stale policy, or whatever the codebase accidentally became yesterday.
So yes: the load-bearing wall has to exist.
Scarab’s answer is that the wall itself must be repo-local, explicit, inspectable, versioned, and governed — not hidden in the model, not floating in the chat, and not silently rewritten by the agent.
Yes — and the „governed baseline" framing is sharper than the static-but-mortal wall I was holding. Versioned, change-controlled, deliberately revisable: that's continuous lifecycle, not periodic structural review. Conceded on the shape.
One caveat on cadence: in rapid dev / research mode, where we're exploring different paths in parallel, continuous governance on the baseline would fire too often to be useful — every probe looks like contradiction. What I run instead is weekly agile retro: a session where the patterns from a week's worth of exploration get harvested, the working ones promoted into the wall, the broken ones archived. Periodic at the retro cadence, continuous within the boundary the retro draws. Different work mode, probably same destination at lower frequency.
What we land on the same is the part harder to dispute: the reference frame has to be explicit, inspectable, not hidden in the model, not floating in conversation, not silently rewritten. Whether you keep it entirely repo-local (Scarab) or split between repo-local diagnostic baseline and operator-side intent — where I'd still draw the line for the part the codebase can't express, the vendor constraints, deferred trade-offs, unrecorded why-nots — the failure mode of NOT having the frame is identical: agent builds from drift.
The bootstrap I won't fully resolve here is who governs the governance — even a continuously-refreshed baseline has a hand on the refresh, and that hand authors something. That's probably the seam where the frameworks still differ, and probably where the next year of work lives. Strong thread. Going to keep pulling at this wall.
Yes — that seam is real, and it’s one Scarab handles directly. Governance changes move through a reviewed governance-adoption path: they have to become explicit, inspectable, evidence-backed repo state before Scarab treats them as part of the accepted baseline.
That's the right shape for production governance — explicit, inspectable, evidence-backed adoption path before promotion. It's also where the work-mode divide gets sharpest. In rapid dev / research, routing every concept through a reviewed governance-adoption gate would tax the discovery itself — half the ideas we explore are supposed to die before they're worth governing. Different answer for different problem; both methodologies are well-tested in software ops history. Yours for shipped systems, retro-driven harvest for what comes before them. Appreciate the thread.
Agreed on the distinction between exploration and adoption. I wouldn’t route every research thought through a governance gate either.
The Scarab line is simpler: exploration can stay provisional, but adoption into repo truth cannot be silent. Once an idea, boundary, policy, invariant, or repair lane becomes part of the accepted baseline the agent will build on, it has to become explicit, inspectable, and evidence-backed.
So I’d frame it less as “Scarab for shipped systems” and more as: Scarab governs what gets promoted into repo truth, whether that happens during repair, active build, or production hardening.
Fair refinement. „Governs what gets promoted into repo truth, whenever that happens" is the more accurate boundary — adoption gate, not shipped-vs-not. Updates how I'd recommend the tool. Strong thread, closing my side here.
Great explanation. I like how you broke down the agent workflow into clear steps. Understanding the thought-action-observation cycle makes AI agents much easier to grasp for beginners.