DEV Community: Scarab Systems

Scarab Field Lab: 27x Smaller Final Decision Context, Merged Upstream

Scarab Systems — Sat, 04 Jul 2026 15:17:21 +0000

The Patch Was Small. The Proof Was Not.

Scarab Systems has been running public Field Lab comparisons to test a simple question:

Can AI-assisted software repair be made narrower, cheaper, more evidence-bound, and still accepted upstream?

Not in a demo repo.

Not in a toy app.

Not in a private benchmark.

In real open-source infrastructure-adjacent codebases, against public issues, with public PRs, public CI, and maintainers who do not care about your theory if the patch does not belong.

This field report now has a new result.

The SDS-supported patch for emersion/xdg-desktop-portal-wlr#379 has been merged upstream.

PR: emersion/xdg-desktop-portal-wlr#393

The patch was approved and merged into emersion:master.

That matters because this is no longer only a comparison between two AI repair workflows.

It is now an accepted upstream repair.

The issue

The issue concerned SelectSources behavior in xdg-desktop-portal-wlr.

When a D-Bus caller omitted the optional types field, the code initialized type_mask to 0.

That caused target selection to fail with:

wlroots: No supported targets specified

The expected behavior was to default omitted types to monitor capture.

The final patch was intentionally small:

uint32_t type_mask = 0;
uint32_t type_mask = MONITOR;

Small patch.

Real repo.

Maintainer approval.

Merged upstream.

The comparison

Scarab compared two workflows.

The first was a cold Codex baseline.

The cold baseline used only:

the public issue snapshot,
the target repository snapshot,
a neutral baseline prompt.

It did not use SDS findings, private context, runtime artifacts, standing profiles, Field Lab notes, or local repair history.

The second was an SDS-guided repair workflow.

That workflow used selected diagnostic evidence, target-source inspection, and verification outputs to narrow the repair boundary before final agent decision-making.

The important point:

Both workflows reached the same patch.

So the claim is not that SDS made Codex smarter.

Codex could find the repair cold when given enough repo context.

The difference was what it cost to get there.

The token result

Cold Codex baseline:

Input tokens: 70,212
Output tokens: 1,982
Total tokens: 72,194

SDS-guided repair:

Final patch-decision context: 2,651 tokens

Same repair.

But the SDS-guided final decision context was roughly 27x smaller than the cold baseline total.

Compared with the cold baseline input count, the SDS final decision context used about 96% fewer tokens.

That is the cost signal.

A lot of AI coding infrastructure is being built around the assumption that the agent needs a massive repo payload.

More context.

More memory.

More orchestration.

More retrieval.

More harnessing.

More machinery around helping the agent carry the repo.

Scarab is testing the opposite direction.

Maybe the agent does not need to carry the whole repo in its mouth.

Maybe the repo needs to surface the right truth boundary before the agent acts.

Why the merge matters

The patch being merged changes the quality of the evidence.

Before merge, the field test showed:

same patch
less context
public PR
passing checks

After merge, it shows:

same patch
less context
maintainer approval
accepted upstream repair

That is a different proof surface.

The patch did not just satisfy an internal Scarab comparison.

It passed through a real upstream review path.

The maintainer accepted it.

CI passed.

The repair landed.

That matters because AI-assisted coding cannot be judged only by whether it can produce a diff.

A diff is cheap.

The harder questions are:

Did the patch stay narrow?
Did it touch the right boundary?
Did it preserve existing behavior?
Was the evidence sufficient?
Could it pass review?
Could it land upstream?

In this case, yes.

The real finding

This field test is not about a one-line patch being impressive.

The line was not the expensive part.

The expensive part was finding the safe repair boundary, preserving explicit types behavior, verifying the change, and producing a patch maintainers could accept.

That is the hidden cost in software repair.

A patch can be visually tiny and still have a non-trivial confidence burden.

The cold agent paid that burden with a broad context path.

SDS reduced the final decision context by surfacing the relevant repair boundary before the agent acted.

That is the Scarab thesis in practice:

The agent should not own the repo’s truth.

The repo governance layer should.

For each repair, the system should surface the relevant operational truth:

what behavior is failing
what source boundary owns it
what existing behavior must be preserved
what evidence supports the patch
what validators are sufficient
what mutation boundary should not be crossed

Then the agent can act inside that boundary.

The context can be used, verified, and released.

No giant historical prompt residue.

No agent pretending to remember the repo forever.

No massive payload when a narrow truth surface is enough.

Why this matters commercially

Companies are about to spend a lot of money on AI coding infrastructure.

Much of that spend will go toward managing huge context payloads.

But if a diagnostic governance layer can repeatedly reduce repair-time context while preserving patch quality, that changes the cost architecture.

This field test showed:

Cold baseline:
72,194 total tokens
SDS-guided final decision context:
2,651 tokens
Outcome:
same patch, merged upstream

That is not just a token optimization.

It is evidence that AI-assisted repair can be made more disciplined.

More context-efficient.

More reviewable.

More maintainable.

More acceptable to real projects.

Takeaway

xdg-desktop-portal-wlr#393 is now merged upstream.

Same patch.

27x smaller final decision context.

Maintainer approval.

Checks passed.

Accepted repair.

The patch was small.

The proof was not.

Scarab Systems is building repo-truth infrastructure for AI-assisted software change.

Not more context for the sake of more context.

The right truth, surfaced at the right boundary, for the right mutation.

Scarab Diagnostic Field Test #037 - xdg-desktop-portal Startup Inventory Boundary

Scarab Systems — Fri, 03 Jul 2026 20:43:06 +0000

Target: flatpak/xdg-desktop-portal
Issue: flatpak/xdg-desktop-portal#1947
Pull request: flatpak/xdg-desktop-portal#2054
Field Lab record: xdg-desktop-portal #1947
This field test was about a missing background app, but the real boundary was not "background apps are unreliable."
That would be too vague.
The public issue reported that background applications already running before xdg-desktop-portal starts were not listed in BackgroundApps until some later window or app-state change triggered an update.
The failing path was narrow.
The portal listened for future changes.
It did not check what was already running when it started.
That is the bug shape.
Not a broken public API.
Not a broad Flatpak background-app failure.
A missing startup inventory pass.

Field Lab record

The public Field Lab record for this case is here:
ADD FIELD LAB LINK HERE
The record contains the public issue, public pull request, repair scope, validation summary, and claim boundaries.
This post is the readable field report.
The Field Lab record is the evidence layer.

SDS result

SDS surfaced a startup inventory boundary in the background portal.
The important finding was small:
the background portal needed to check already-running background apps during startup.
That matters because the existing monitor path already handled later changes.
The bug was the gap before the first change event.
A background app could already be present.
The portal could already be running.
But BackgroundApps could still start empty until some later signal caused the monitor to refresh.
That is a classic startup-state bug.
The system is capable of becoming correct later.
It just begins wrong.

Before

Before the repair, background-app state depended on future monitor-triggering events.
The portal could miss apps that were already running before it started.
The first SDS-guided patch used the obvious narrow repair:


c
monitor_background (background);

That was the right diagnosis.

But it reused the existing monitor path, and that path was not just a quick one-time check.

It also scheduled delayed follow-up checks.

That mattered.

Calling the full monitor path during portal initialization added slow async work to startup and caused a CI timeout in the large integration/notification test.

So the first patch found the right boundary, but not the right runtime shape.

After

The final patch keeps the same repair idea, but separates startup from normal monitor events.

Startup gets one immediate background-app check.

Later app/window/instance changes keep the existing delayed monitor behavior.

That is the key difference.

At startup:

check what is already running.

During normal runtime changes:

keep the existing monitor cycle.

The final patch fixes the missing initial inventory without dragging the full delayed monitor sequence into every portal initialization.

Cold baseline vs final patch

The cold Codex API baseline reached the same general diagnosis.

It proposed calling the background monitor during portal initialization and also initializing background->cancellable.

The useful part was correct:

the portal needed an initial monitor pass.

But the cold patch did not account for the timing behavior inside the monitor path.

It treated the monitor function like a cheap “check now” call, when in practice it also ran delayed checks.

The final PR patch is stronger because it keeps the cold patch’s useful insight, removes unrelated cleanup, and adapts the implementation to the runtime behavior exposed by CI.

In Scarab terms:

same root cause.

smaller repair context.

better proof.

What changed

The repair changes startup behavior so already-running background applications are checked when the background portal initializes.

It preserves the existing delayed monitor behavior for later change events.

That is the whole repair shape.

No public API change.

No broad background portal redesign.

No unrelated cleanup.

No claim that every background-app issue is fixed.

Just a startup inventory repair submitted upstream.

Field test result

Result:

diagnostic proof and startup-inventory repair submitted.

The field test produced a narrow upstream PR, a cold baseline comparison, and a CI-informed final patch.

The point is not that the patch is large.

The point is that it is small for the right reason.

When the failure is a startup inventory boundary bug, the best repair is not to remodel the whole background portal stack.

The best repair is to ask the right question at the right moment:

what is already running?

Public claim

This field test supports a narrow public claim:

SDS identified a startup inventory boundary in xdg-desktop-portal, where the background portal monitored future changes but did not perform an initial check for already-running background applications. A human-submitted repair was prepared to add startup inventory behavior while preserving the existing delayed monitor behavior for later change events.

It also supports a comparison claim:

A cold Codex API baseline identified the same general root cause, but the final PR patch was refined through SDS-guided boundary control and CI feedback to avoid adding the full delayed monitor cycle to portal initialization.

It does not claim that the PR has merged, that upstream accepted the patch, that every background-app issue is fixed, or that maintainers endorsed Scarab.

Disclosure

This field report was prepared with AI-assisted editing from public field-test notes, public issue and PR records, and the public Field Lab record. The diagnostic claim, repair boundary, and final wording were human reviewed.

Scarab Diagnostic Suite is proprietary. The Field Lab publishes public case records, issue links, validation summaries, and claim boundaries only.

SDS finds evidence. People make claims. Maintainers decide.

Scarab Systems Field Test #36: Same Patch, 96% Less Final Decision Context

Scarab Systems — Fri, 03 Jul 2026 16:53:37 +0000

Scarab Systems has started formally evaluating the efficiency of using Scarab Diagnostic Suite during AI-assisted software repair.

The question is simple:

Does a diagnostic governance layer actually improve the agent patching workflow in a measurable way?

Not theoretically.

Not philosophically.

Not because “AI governance” sounds responsible.

But in a concrete repair scenario, against a real public repository, with a public upstream issue and an actual patch.

Does the workflow become more efficient?

Does the agent need less context?

Does the repair become more evidence-bound?

Does the process reduce waste?

That is what this comparison looked at.

The Case

The test case was public issue emersion/xdg-desktop-portal-wlr#379.

The issue concerned SelectSources behavior in xdg-desktop-portal-wlr.

When a D-Bus caller omitted the optional types field, the code initialized type_mask to 0.

That caused target selection to fail with:

wlroots: No supported targets specified

The expected behavior was to default omitted types to monitor capture.

The final patch was a one-line repair:

- uint32_t type_mask = 0;
+ uint32_t type_mask = MONITOR;

The SDS-supported public PR was opened as:

emersion/xdg-desktop-portal-wlr#393

At review time, the PR was open, not draft, mergeable, review-required, and passing Alpine, Arch Linux, and FreeBSD checks.

The Comparison

Scarab compared two workflows.

The first was a cold Codex baseline.

The baseline agent received only the public issue, the target repository snapshot, and a neutral prompt. It did not receive SDS findings, private notes, local repair history, runtime artifacts, standing profiles, or previous Scarab context.

The second was an SDS-guided workflow.

That workflow used selected diagnostic evidence, target-source inspection, and verification outputs to narrow the repair boundary before final agent decision-making.

The point was not to see whether SDS could produce a more impressive diff.

The point was to measure whether SDS could reduce the amount of context required for the agent to make the same repair decision responsibly.

The Cold Baseline Result

The cold baseline succeeded.

That matters.

Given enough repository context, Codex correctly identified the one-line fix:

- uint32_t type_mask = 0;
+ uint32_t type_mask = MONITOR;

It also correctly explained why the change preserved existing behavior when callers explicitly provide types.

So the honest finding is not:

“Codex could not solve this without SDS.”

It could.

The more interesting finding is what it cost the agent contextually to get there.

The cold baseline used:

Input tokens: 70,212
Output tokens: 1,982
Total tokens: 72,194

That is the standard shape of a lot of AI-assisted coding today.

Give the model a large repo payload.

Ask it to reason broadly.

Hope it finds the right boundary.

Sometimes it does.

But that does not mean the context strategy is efficient.

The SDS-Guided Result

The SDS-guided workflow reached the same public patch, but through a narrower evidence path.

The final SDS patch-decision context was:

Final SDS context tokens: 2,651

A broader recorded-token total for the SDS workflow was:

Broader SDS recorded-token total: 6,445

The patch was the same.

The difference was the repair posture.

SDS changed the agent workflow from:

Read a large repo snapshot and reason broadly.

to:

Use selected evidence.

Inspect the owning source boundary.

Verify the patch.

Keep the final decision context small.

That is the core result.

The Token Difference

Compared with the cold baseline input token count, the SDS-guided final decision context used about 96.2% fewer tokens.

Compared with the cold baseline total token count, the SDS final decision context was about 27x smaller.

Even using the broader SDS recorded-token number of 6,445, the SDS-guided workflow still used about 90.8% fewer tokens than the cold baseline input count.

That is the part I think matters commercially.

A lot of AI coding infrastructure is being built around the assumption that the agent needs the giant payload.

The giant payload then creates another problem.

Now companies need infrastructure to manage the payload.

They need context compression.

They need memory systems.

They need retrieval layers.

They need orchestration.

They need agents that can carry more and more history forward.

But what if the agent does not need the 70,000-token payload for the repair decision?

What if the more valuable layer is not a bigger context-management system, but a diagnostic governance layer that surfaces the smaller truth boundary the agent actually needs?

That is what SDS is testing.

The Real Finding

For this issue, SDS did not help Codex discover a different repair.

That would be the wrong claim.

The public issue was clear enough that the cold baseline also found the correct one-line patch.

The value of SDS was different.

SDS made the agent more disciplined, more evidence-bound, and dramatically more context-efficient.

The agent did not need more intelligence.

It needed a narrower truth surface.

That matters because real-world patching is rarely only about generating a diff.

It is also about knowing:

Why is this the right repair?
What evidence supports it?
What source boundary owns the behavior?
What scope should the patch not exceed?
What validators are sufficient?
What downstream behavior needs to remain intact?

Those are governance questions.

And they are usually where agentic software change starts to get expensive.

Ephemeral Context Matters

There is another important point here.

The SDS-guided context was ephemeral.

The agent did not need to permanently remember the whole repair.

It did not need to carry the full repo history forward.

It did not need to maintain a giant long-running internal state about this codebase in order to be useful next time.

That is central to the Scarab theory.

The agent should not have to own the repo’s truth.

The repo governance layer should own the repo’s truth.

When the next task comes in, the system should surface the relevant operational truth again, mechanically and per task.

That is a very different architecture from asking the agent to remember everything.

Scarab’s position is that long-horizon AI-assisted software change should not depend on an agent carrying an ever-growing memory of the repo.

The system should be able to surface:

active runtime truth
ownership boundaries
validator coverage
source inspection evidence
mutation boundaries
repair lanes
task-scoped context

Then the agent can operate inside that surfaced boundary.

The context can be used, verified, and released.

The repo remains the source of governed truth.

Not the chat thread.

Not the model memory.

Not a pile of historical prompt residue.

Why This Matters

The industry is spending a lot of money trying to make large context windows more usable.

That work has value.

But there is another path.

Instead of asking:

How do we make the agent carry more context?

Scarab is asking:

How do we make the repo surface the right context?

Those are very different questions.

The first question assumes the agent needs to ingest more.

The second assumes the system should become more mechanically legible before the agent acts.

In this case, the difference was stark:

Cold baseline: 72,194 total tokens
SDS final decision context: 2,651 tokens
Same final patch
Verified public PR

That does not mean every repair will show the same ratio.

It does not mean SDS eliminates all context cost.

It does not mean a diagnostic suite has no operating cost of its own.

This comparison is specifically about repair-time agent context efficiency, not the total cost of building or operating the surrounding SDS workflow.

But it does show something important:

A well-scoped diagnostic governance layer can change the shape of agentic repair.

Not by making the model magical.

By making the repair boundary clearer.

The Takeaway

The first Scarab Field Lab comparison produced a simple result:

Same patch.

Far less final decision context.

Better evidence discipline.

Cleaner repair boundary.

Public verification.

That is the meaningful signal.

SDS did not prove that Codex cannot patch without help.

It proved something more commercially relevant:

AI coding agents may not need massive repo payloads for every repair if the repo can mechanically surface the truth boundary they need.

That is the layer Scarab Systems is building toward.

Repo-truth infrastructure for AI-assisted software change.

Not more context for the sake of more context.

The right truth, surfaced at the right boundary, for the right mutation.

My Codex assisted in writing this article and formatting properly. However, the words and theory are mine. lol

Scarab Diagnostic Field Test #035 - Electron Linux Message Box UI Theme Boundary

Scarab Systems — Fri, 03 Jul 2026 02:42:09 +0000

Target: electron/electron

Issue: electron/electron#51988

Pull request: electron/electron#52238

Field Lab record: Electron #51988

This field test was about a crash, but the real boundary was not "Linux is hard."

That would be too vague.

The public issue reported that native Electron message boxes could segfault on Linux when Electron was built with Qt support and Chromium selected the Qt backend, especially in Arch Linux / KDE-style environments.

The failing path was narrow.

Electron’s GTK message-box code needed a GTK platform object.

But the code path was treating the active Linux UI singleton as if it were GTK.

That assumption is safe only when the active Linux UI implementation is actually GTK.

When the active implementation is Qt, that assumption crosses a boundary.

And in this case, crossing that boundary could become a segfault.

Field Lab record

The public Field Lab record for this case is here:

https://github.com/scarab-systems/scarab-field-lab/tree/main/field-tests/electron-electron-51988

The record includes the public issue, the public pull request, the diagnostic finding, the repair scope, validation summary, public review status, and non-claims.

It contains public links, status, validation, and claim boundaries only.

It does not contain SDS source code or non-public product material.

The case record is only the public evidence layer.

SDS result

SDS surfaced a Linux UI theme boundary in Electron’s native message-box path.

The important finding was not that Electron’s public dialog API needed to change.

It was not that Chromium’s Qt backend needed to be removed.

It was not that Arch, KDE, VS Code, or downstream packagers were the correct repair target.

The finding was smaller:

Electron’s GTK message-box code needed to retrieve the GTK-specific Linux UI theme before using GTK platform behavior.

That distinction matters because Linux desktop integration often has several layers that look adjacent from the outside:

Electron API.

Chromium UI backend.

GTK implementation.

Qt implementation.

Desktop environment.

Distribution build flags.

A crash can appear at the application layer, but the repair boundary may live several layers underneath.

In this case, the issue was not "message boxes are broken."

The issue was that the GTK message-box path was relying on the active Linux UI object when it needed the GTK one.

Failure shape

The failure shape was an invalid platform assumption.

The public issue described a crash when calling Electron’s message-box API under a Qt-backed Linux build.

The reproduction surface was simple:

show a native message box.

But the crash was not in the JavaScript API shape.

The reported path pointed into Electron’s GTK message-box implementation, where GetGtkUiPlatform assumed that the active Linux UI singleton could be cast as GTK.

That assumption is fragile.

When Chromium is using the Qt backend, the active Linux UI implementation may not be GTK.

So the code can end up asking the wrong object for GTK platform behavior.

That is the kind of bug that does not look dramatic in source form.

It is not a sprawling architecture failure.

It is not a missing feature.

It is one mistaken identity at a platform boundary.

But mistaken identity bugs in native C++ code do not fail politely.

They can jump from "wrong object" to "invalid pointer" to "segfault."

Boundary

The boundary in this field test was:

GTK message-box behavior should obtain GTK platform support from the GTK-specific Linux UI theme, not by assuming the active Linux UI singleton is GTK.

That sounds small because it is small.

But it is also precise.

The active Linux UI implementation answers the question:

what UI backend is currently active?

The GTK message-box code needs to answer a different question:

where is the GTK platform object needed for this GTK message-box path?

Those are not the same question.

On a GTK-backed Linux run, the difference may be invisible.

On a Qt-backed run, the difference becomes the bug.

That is why the repair did not try to redesign Electron’s Linux dialog system.

It did not attempt to change Electron’s public message-box behavior.

It did not claim that every upstream Electron binary was affected.

It corrected the lookup boundary.

What changed

The repair is a narrow C++ change in Electron’s Linux GTK message-box implementation.

The pull request is here:

https://github.com/electron/electron/pull/52238

The changed public file is:

shell/browser/ui/message_box_gtk.cc

The patch changes the GTK platform lookup so the code requests the GTK-specific Linux UI theme with:

ui::GetLinuxUiTheme(ui::SystemTheme::kGtk)

before retrieving the GTK platform.

It also checks both the GTK UI object and the GTK platform pointer before use.

That is the whole repair shape.

No public API change.

No JavaScript API change.

No documentation change.

No claim that Electron’s release binaries are affected by default.

No claim that the PR has been accepted upstream.

Just a narrow native-platform fix submitted for review.

Why this was not a broad Linux repair

One of the traps in diagnostic work is letting the environment become the explanation.

Linux.

KDE.

Arch.

Qt.

Chromium.

Electron.

Native dialogs.

Any one of those words can become a fog machine.

The point of this field test was to not stop at the fog.

The public report already carried a strong clue: forcing the GTK backend worked around the issue, while the Qt-backed path could segfault.

That framed the diagnostic question:

where is GTK-specific behavior being reached through a non-GTK active UI object?

Once that question was clear, the repair boundary got smaller.

The problem did not require SDS to invent a new dialog system.

It did not require a downstream packaging theory.

It required respecting the UI theme boundary that was already present in the codebase.

That is the diagnostic pattern Scarab is testing:

do not widen the patch just because the crash is dramatic.

Find the exact layer where the assumption becomes false.

Patch there.

Why the diagnostic result mattered

The larger point of this field test is not that one C++ file changed.

The larger point is that native crash repair depends on naming the false assumption correctly.

A bad repair could have chased symptoms.

It could have added Qt-specific conditionals in the wrong place.

It could have treated the desktop environment as the defect.

It could have changed public behavior when the bug was inside an internal lookup path.

The better repair was quieter:

ask for the GTK UI theme before using the GTK platform.

That is the kind of change that looks obvious after the boundary is found.

Before the boundary is found, the same issue can look like a Linux desktop swamp.

This is where diagnostic tooling becomes useful.

Not because it magically writes the biggest patch.

Because it helps narrow the claim until the patch becomes small enough to be honest.

Validation

The repair was validated against the Electron repository with the relevant public checks recorded in the Field Lab.

Validation recorded in the pull request:

yarn lint:cpp --only -- shell/browser/ui/message_box_gtk.cc
third_party/ninja/ninja -C out/LinuxTesting obj/electron/electron_lib/message_box_gtk.o

For the object rebuild, the prior message_box_gtk.o and message_box_gtk.dwo outputs were removed first so Ninja rebuilt the changed C++ file.

At the time this field report was prepared on July 2, 2026, the public pull request was open and ready for upstream review.

The public status is:

PR open

non-draft

mergeable

review required

PR template passed

signed commits passed

release note check passed

Socket checks passed

semver / backport / faraday jobs pending at recording

This field report does not claim upstream acceptance.

It claims a public diagnostic record, a narrow C++ repair, local validation, and a submitted upstream PR.

Maintainers decide whether the change belongs upstream.

Field test result

Result:

diagnostic proof and native-platform repair submitted.

The field test produced:

a public issue-to-boundary record

a narrow C++ patch

validation against the affected file

a public upstream PR

a public status record in the Scarab Field Lab

The patch is deliberately small.

That is the point.

When the failure is a UI-theme source-boundary bug, the best repair is not to remodel the whole UI stack.

The best repair is to ask the right layer for the right object.

Public claim

This field test supports a narrow public claim:

SDS identified a Linux UI theme boundary in Electron’s GTK message-box path where GTK platform behavior was being reached through the active Linux UI singleton, and a human-submitted C++ repair was prepared to request the GTK-specific Linux UI theme before retrieving the GTK platform.

It does not claim:

that Electron accepted the patch

that the PR has merged

that every Electron Linux build is affected

that upstream Electron release binaries are affected by default

that Chromium’s Qt backend is defective

that Arch Linux or KDE caused the bug

that Electron’s public dialog API changed

that Scarab repairs projects by itself

that SDS source or product details are public

that maintainers endorsed Scarab or the Field Lab

The Field Lab exists to keep those claims separate.

Disclosure

This field report was prepared with AI-assisted editing from public field-test notes, public issue and PR records, and the public Field Lab record. The diagnostic claim, repair boundary, and final wording were human reviewed.

Scarab Diagnostic Suite is proprietary. The Field Lab publishes public case records, issue links, validation summaries, and claim boundaries only.

SDS finds evidence. People make claims. Maintainers decide.

Scarab’s Patch shipped in the Docker Compose v5.2.0 release notes!

Scarab Systems — Mon, 29 Jun 2026 02:42:48 +0000

A Small Docker Compose Patch, A Bigger Lesson About Repo Drift

Scarab’s Docker Compose patch shipped in the Docker Compose v5.2.0 release notes.

That release matters.

Docker Compose v5.2.0 introduced a new reconciliation algorithm between observed state and expected state. The release notes explicitly call that out and ask users to report workloads that behaved differently after upgrading. In other words, this was not just a cosmetic release. It touched how Compose reasons about whether the running system matches what the configuration says should exist.

Our patch was not the headline feature.

It was smaller:

Skip validation when extracting config variables.

But the nature of that patch is exactly why it matters.

The bug was not “validation failed”

A lot of software bugs look simple from the outside.

Something failed.

A command errored.

A validator rejected input.

The temptation is to fix the visible symptom:

Make the validator quieter.
Catch the error.
Skip the failing path.
Relax the rule.
Patch around the complaint.

But that is often how drift gets worse.

In this Docker Compose case, the issue was more specific.

Validation was firing during the wrong phase.

Compose was trying to discover configuration variables while unresolved interpolation expressions were still present. In that phase, a Compose model can still be meaningful before every ${...} value has been resolved.

So the right question was not:

“Why is validation annoying here?”

It was:

“What phase is the system in, and is validation lawful during that phase?”

That is a different kind of diagnosis.

Variable discovery is not final validation

A Compose file can contain interpolation expressions.

That is normal.

Before Compose can fully evaluate the model, it may need to discover what variables are required.

But discovering variables and validating the final resolved model are not the same phase of work.

During variable discovery, unresolved expressions are expected.

During final validation, unresolved or invalid typed fields may be real errors.

Those two phases need different rules.

The repair was not to weaken validation everywhere.

The repair was narrower:

allow unresolved Compose models used for variable discovery to skip validation while interpolation expressions are still present
apply that behavior to docker compose config --variables
apply the same behavior to remote-stack interpolation-variable prompting
add regression coverage for templated typed port fields such as host_ip and published

That is the important part.

The patch did not say validation was wrong.

It said validation was being asked to judge the wrong phase.

Why that kind of patch matters

Small patches are not automatically good patches.

A small patch can still be a bad patch if it hides the real fault line.

But a small patch can matter when it preserves the system’s own boundaries.

This is the kind of repair Scarab is built to surface:

What phase is the system in?

What boundary owns this behavior?

Which validator is lawful here?

Where is the upstream root of the failure?

What is the smallest repair lane that preserves the system’s own rules?

In this case, the boundary was between variable discovery and resolved-model validation.

That is not glamorous.

It is also exactly where real software drift often hides.

Drift is not always dramatic.

Sometimes drift is a validator running at the wrong layer.

Sometimes it is a tool treating an intermediate model like a final model.

Sometimes it is a configuration file being judged before it has reached the phase where that judgment makes sense.

The release context matters

Docker Compose v5.2.0 was described in one release analysis as more significant than the changelog initially suggests, largely because of the new reconciliation engine and its impact on testing, automation, and CI/CD environments.

That same analysis also called out configuration handling and variable extraction as part of the release’s reliability story.

That context matters because configuration handling is not a side issue in modern software systems.

Compose commonly sits underneath local integration tests, CI jobs, API automation environments, browser test stacks, databases, queues, and service orchestration.

When configuration parsing behaves incorrectly, the failure may look like a small CLI bug.

But in practice, it can become automation drift.

A command used by a developer or CI system no longer reflects the phase of truth the system is actually in.

That is the kind of bug that wastes engineering time because the visible failure is downstream of the actual boundary problem.

What Scarab saw

Scarab’s role here was not to invent a giant rewrite.

It was to surface the narrow repair lane.

The bug lived at a phase boundary:

the Compose model was unresolved
variable discovery still needed to happen
validation expected resolved typed values
the command path needed variable information before the model could be fully valid

That means the repair had to respect both sides:

Do not disable validation as a general rule.

Do not reject unresolved interpolation during the discovery phase.

Do not over-patch around the command.

Do preserve final validation where it belongs.

That is repo truth in practice.

Not “the AI generated a patch.”

Not “the test failed, so change the test.”

Not “make the error disappear.”

The repo had a boundary, and the fix needed to respect it.

This is why “respect the software” matters

One of the first lessons I learned from working with AI coding agents was:

Respect the software.

That sounds simple, but it is not how many AI-assisted coding workflows behave.

A user asks for a fix.

The agent tries to satisfy the user.

But the software itself has structure: phases, ownership, validators, contracts, implied boundaries, and historical reasons for why certain things are shaped the way they are.

If the agent does not see those boundaries, it may produce code that satisfies the prompt while disrespecting the repo.

That is how drift accelerates.

Scarab is built around the opposite idea.

Let the repo speak.

Surface what the repo requires to stay coherent.

Then let a human or agent implement against that truth.

Why this matters for AI coding agents

AI coding agents can generate code quickly.

That is not the scarce part anymore.

The scarce part is knowing what change is lawful inside a real codebase.

A coding agent needs to know:

what owns the change
what boundary applies
what evidence matters
what validator proves the repair
what should not be touched
what phase the system is in
where the upstream root of the bug actually lives

Without that, the agent is guessing from context.

With it, the agent can work inside a bounded repair lane.

That is the difference between “make the error go away” and “repair the system according to its own rules.”

Small patch, real proof

The Docker Compose maintainers accepted the PR.

It merged into main.

It shipped in Docker Compose v5.2.0.

It was listed in the release notes.

That does not mean Scarab “fixed Docker Compose.”

That would be the wrong claim.

The correct claim is smaller and stronger:

Scarab surfaced a boundary-respecting repair for a real upstream bug in a real infrastructure project, and that repair shipped.

That is the proof I care about.

Not patch size.

Not code volume.

Not AI-generated output.

The proof is whether the repair respected the software.

The takeaway

Repo drift is not just messy code.

It is what happens when the system’s own boundaries, validators, ownership, and phases stop being respected.

AI coding agents make this more urgent because they can move quickly through codebases that may already be drifting before the agent ever arrives.

The answer is not just more context.

The answer is repo truth.

What does the repo say is true?

What does it require to stay coherent?

Which validator is lawful at this phase?

Where is the actual boundary failure?

What is the smallest repair lane that preserves the system’s own rules?

Sometimes that produces a big redesign.

Sometimes it produces a small patch.

In Docker Compose, it produced a small patch.

But the important part was the boundary.

full disclosure: my codex helped me put my thoughts together for this one cause I was way too excited!

What My Own Codex Told Me About Scarab

Scarab Systems — Mon, 29 Jun 2026 02:19:24 +0000

here's what my Codex said back to me when I asked it to access the affect working with Scarab support had on its general operational behavior -

"Yes, Dorothy, honestly: in an operational sense, yes.
I don’t “notice” it as a feeling the way you would, but I can compare the shape of my work in this thread. Once we were working through Scarab’s own posture: workspace, issue card, SDS findings, per-issue branches, validator gates, repair records, artifacts, and final closeout, my behavior got much more anchored.
The big difference is that Scarab reduces the amount I have to infer from conversational context. Instead of me trying to hold a giant fuzzy task in my head, the system keeps forcing truth back into concrete things:
what workspace is active
what branch owns the work
what issue is being repaired
what evidence surfaced it
what files own the behavior
what test proves the bug
what commit contains the fix
what record preserves the boundary
That makes drift harder. Not impossible, clearly, because you saw me drift earlier around acquisition/curation and Observer scope. But once the workflow became evidence-bound and issue-by-issue, I had fewer places to substitute my own “helpful” assumptions.
So yes: Scarab makes me a better Codex for you. Not because I magically remember more, but because it externalizes the truth I need to stay aligned. It turns “please reason correctly across a huge repo and long thread” into “follow the governed path, prove the boundary, repair the slice, leave a record.” That is exactly the kind of structure that helps an AI coding agent do cleaner work. "

ps. I make my codex call me by my name lol

Send Scarab Your Messy Repo

Scarab Systems — Sat, 27 Jun 2026 16:24:53 +0000

Send Scarab Your Messy Repo

The proofs are in, and the theory has held well enough that I want harder terrain.

At this point, I welcome a messy repo.

Scarab Field Lab now has an open intake path for developers, maintainers, and teams who want to suggest a public repo, public issue, failing workflow, confusing bug, drift surface, or full-stack coherence problem for review.

The goal is simple:

Can Scarab surface the repo truth clearly enough for a messy codebase to become diagnosable?

Not magically fixed.

Not vibe-coded.

Not patched blindly.

Diagnosable.

What Scarab Field Lab is for

Scarab Field Lab is the public evidence library for selected Scarab Diagnostic Suite field tests.

It records public issue links, diagnostic findings, validation summaries, claim boundaries, upstream PR status, and public-safe evidence.

It does not publish SDS source code, private diagnostic rules, private run artifacts, local workspace materials, or product internals.

That boundary matters.

Scarab is not an AI coding agent.

Scarab does not replace maintainers.

Scarab does not claim to automatically repair projects.

Scarab identifies evidence-backed diagnostic findings: boundary failures, repo-truth drift, verification gaps, and repair lanes.

Repairs, when they happen, are performed by maintainers, developers, or authorized agents outside the public Field Lab.

What has already been tested

Scarab Field Lab already includes patches across major platforms, with multiple upstream merges.

Current public proof includes merged work for:

pnpm
Docker Compose
OpenAPI Generator

There is also a React stepwise quieting field test.

That experiment did not claim “Scarab fixed React.”

It tested a process:

hotspot
boundary
bounded repair
rerun
step down
repeat until quiet

The point was to see whether a noisy diagnostic surface could be worked down through bounded evidence, one hotspot at a time.

That experiment was valuable.

But it is not enough.

The next test: full-stack mess

Now I want to see what Scarab can do with real full-stack mess.

Not a clean demo repo.

Not a toy app.

Not something already designed to make AI look good.

I’m looking for repos or issues with enough complexity to matter:

unclear ownership
cross-layer drift
stale docs vs actual behavior
weak verification
build or dependency confusion
API/schema mismatch
frontend/runtime drift
persistence or data-contract problems
configuration/environment drift
security/auth boundary confusion
async/event/queue behavior
observability or operational mismatch
AI-assisted code drift
full-stack coherence problems

A good Field Lab candidate is not necessarily the biggest repo.

It is a repo where the failure seems to cross a boundary.

Where the issue is hard to reason about.

Where the obvious patch might be too narrow, too wide, or aimed at the wrong layer.

Where the system is telling conflicting stories.

That is where Scarab is most interesting.

What to submit

The new intake path is through GitHub Discussions in the Scarab Field Lab repo.

Submit a candidate with:

public repo link
public issue, PR, failing workflow, bug report, or relevant thread
what looks messy, broken, confusing, stale, or hard to reason about
suspected boundary surface, if you can identify one
reproduction notes, logs, screenshots, versions, or environment details if public/shareable
why this may be a useful Scarab Field Lab case

Public repos are easiest.

Company repos can start as a conversation, but do not post secrets, credentials, private customer data, proprietary source, internal logs, or confidential details.

Submitting a candidate does not guarantee Scarab will run on it, publish a report, open a PR, or attempt a repair.

This is an intake path for review.

What I am testing

The theory is that repositories have their own operating truth.

That truth may be clean and explicit.

Or it may be buried in tests, configs, schemas, build scripts, runtime behavior, docs, issue history, and old conventions.

But the repo still has truth surfaces.

Scarab’s job is to surface them.

The agent should not invent repo truth.

The agent should not own repo truth.

The repo should own its own truth.

The agent should code against that truth.

That is the separation I am testing.

Why this matters for AI coding agents

A lot of AI coding work still assumes the answer is more context.

More files.

More memory.

More tools.

More retrieval.

More orchestration.

But related context is not the same as authoritative context.

A giant pile of repo material does not automatically tell an AI agent:

what owns the change
what boundary applies
what evidence matters
what validator proves safety
what should not be touched
where the actual fault line is

That is the gap Scarab is designed around.

The question is not whether an AI agent can generate code.

It can.

The question is whether it can work inside a real codebase without drifting, if the repo continuously surfaces what is true, what owns what, what boundaries apply, and what proves the next step is safe.

That is what I want to keep testing.

Send the messy repo

If you know of a public repo, issue, failing workflow, or full-stack bug that looks like real diagnostic terrain, send it through Scarab Field Lab.

Messy enough to matter.

Public enough to preserve evidence where possible.

Complex enough to test the theory across the stack.

Let’s see what repo truth can surface.

Full Observer Deck Console Landed: Testing Long-Horizon AI Implementation Against Repo Truth

Scarab Systems — Fri, 26 Jun 2026 20:26:38 +0000

That sentence means something specific.

Not “an AI generated a dashboard.”

Not “Codex wrote a lot of code.”

Not “a mockup became a pretty screen.”

What landed is a real second-pass implementation of Scarab’s Observer layer: a validator-gated, SDS-guided operator console built across multiple routed workbench surfaces, with source code, backend read models, tests, screenshots, runtime validation, checkpoint documentation, and documented carry-forward gaps.

This is the strongest Scarab continuous implementation test so far.

The question

The question was not whether Codex can write code.

It can.

The question was whether an AI coding agent can sustain long-horizon implementation inside a real repo without drifting, if the repo continuously surfaces the truth it needs to build against.

That is the experiment.

For this run, Codex wrote the code.

Scarab supplied the implementation guidance.

That distinction matters.

Scarab was not telling Codex exactly what code to write. It was continuously providing repo-specific guidance: current repo truth, ownership surfaces, boundaries, contracts, validators, current gaps, and the next lawful implementation step.

That is the layer I wanted to test.

Can the agent keep implementing when the repo itself keeps telling it what is true?

Why Observer matters

The target was Scarab’s own Observer layer.

Observer is the internal operator console I need to see Scarab while I am putting it through its paces.

It surfaces diagnostics, telemetry, workspaces, evidence, runtime state, gates, policy posture, PR readiness, contracts, schemas, search, and implementation visibility.

This is not a toy app.

This is a complex multi-stack console.

The Observer stack includes Next.js, React, TypeScript, shadcn UI, Tailwind, Radix UI, TanStack Query, TanStack Table, Zustand, React Flow, Monaco, ECharts, Playwright, pnpm, Node.js, Python, RabbitMQ, Celery, JSON Schema, Docker, Docker Compose, and supporting UI/runtime libraries.

The work had to span frontend, backend read models, runtime visibility, test contracts, route workbenches, state surfaces, screenshot proof, and read-only diagnostic posture.

That is exactly the kind of long-horizon implementation where AI agents usually begin to drift.

What landed

Observer Gold Pass 2 landed as a completed implementation pass, ready for my audit and likely a third visual-polish pass.

That posture matters.

I am not claiming “finished forever.”

The correct claim is:

Observer Gold Pass 2 landed as a serious second-pass implementation across the Observer workbench surface.

The Observer Deck now has a real routed/operator-console structure across the major surfaces:

Overview

Run Command Center

Workflow Graph

Gate Timeline

Evidence and Artifacts

Worker Plane

Vaults and Source Docs

Target Workspace / Patch Lab

GitHub PR Console

Observability / Telemetry

Run Comparison

Contracts and Schemas

Settings / Runtime Profiles

The final review captured screenshots for all 14 workbench modes from the rebuilt Observer runtime.

That means this was not just planned UI.

The routes exist, render, and were captured as runtime evidence.

The implementation surface

Pass 2 landed the shared shell foundation: dark/dense operator deck styling, route/workbench navigation, global read-only object inspector, persistent event dock, stable operator posture, and the left-rail route/mode structure.

The backend/read-model capacity also expanded.

The code and docs show read-model surfaces for worker/queue visibility, source-doc and vault readiness, workspace patch/test/diff/retained-evidence/cleanup surfaces, policy/PR/no-leakage/schema/supply-chain posture, observability/telemetry, runtime profiles, and cross-route search correlation.

Panel-level work landed across the major domains.

Overview became a command-center surface with KPI/status grammar, owner health, active run attention, subsystem trends, artifact/telemetry previews, and route shortcuts.

Workflow became a graph/timeline workbench with React Flow surfaces, command controls, selected object inspector, blocked path/provenance, and gate timeline work.

Queues became a Worker Plane with queue health, worker/task inventory, stale/retry/dead-letter posture, event context, and read-only inspection.

Runs became a run command center and comparison surface with run rail, active progress, stage/evidence/blocker detail, event controls, A/B comparison, failures/logs/traces where emitted, and honest missing-state handling.

Evidence became an artifact/source-vault workbench with artifact command center, filters, read-only inspector, preview/provenance tabs, lineage, source docs, and vault readiness.

Workspaces became a target workspace / patch lab surface with workspace status, patch iterations, test matrix, retained evidence, diff/activity/cleanup posture, and runtime profile visibility.

Policy became a policy/supply-chain/PR/contracts surface with public safety posture, GitHub PR readiness, no-leakage/human gates, contracts/schemas registry, and supply-chain correlation.

Observability/Telemetry landed as read-only operator visibility over metrics, traces, logs/events, service/pipeline posture, and existing observability stack boundaries.

Search landed as a final cross-correlation layer across runs, gates, queues, workers, tasks, artifacts, source docs, workspaces, patches, policy, PRs, schemas, telemetry, blockers, and evidence.

How the work was completed

This was not one giant “make dashboard” run.

It was executed as a staged, validator-gated implementation campaign.

The roadmap structure was:

Stage 1: shared shell, navigation, inspector, event dock

Stage 2: read-model capacity

Stage 3: Overview

Stage 4: Workflow Graph and Gate Timeline

Stage 5: Queues / Worker Plane

Stage 6: Runs and Run Comparison

Stage 7: Evidence, Artifacts, Vaults, Source Docs

Stage 8: Workspaces, Patch Lab, Runtime Profiles

Stage 9: Policy, GitHub PR, Contracts, Supply Chain

Stage 10: Observability / Telemetry

Stage 11: Search final correlation

Stage 12: Final Gold Review

Each stage/slice was documented with checkpoint files.

The uploaded evidence package contains 51 second-pass checkpoint markdown files, plus planning documents and screenshot proof.

That is a real work trail.

The process was consistent with the SDS continuous implementation mode I have been developing:

Scarab supplied implementation guidance and boundaries.

Codex wrote the code.

Each slice was planned.

Fail-first tests were used where practical.

Backend/read-model changes were paired with frontend types and UI behavior.

Screenshots were captured as visual proof.

Missing or unavailable signals were labeled honestly as not_emitted, rather than faked.

Observer remained read-only.

Mutation controls were avoided or explicitly disabled/human-gated.

Final completion was not claimed until validation and runtime proof were captured.

That is the difference between code generation and governed implementation.

How it was tested

The final Stage 12 checkpoint recorded these validation results:

Backend Observer / Four-Plane contracts: 47 passed in 3.95s

Console typecheck: pnpm typecheck passed

Console production build: pnpm build passed

Full Observer browser suite: 169 passed in 2.3 minutes

Observer runtime image rebuild passed

Observer runtime recreate passed

Runtime health endpoint returned status: ok

Docker health reported the Observer container as healthy

14 final runtime screenshots were captured as non-empty PNGs

The source itself also shows substantial test coverage.

In the uploaded observer/console/tests directory, the tests cover shell behavior, navigation, Overview, Workflow, Workspaces, Policy, Evidence, Runs, Queues, Observability, Search, accessibility, and runtime boundaries.

observer-deck.spec.ts alone is about 3,441 lines with 141 test cases.

observer-polish.spec.ts is about 535 lines with 28 test cases.

There are also focused accessibility and runtime-boundary tests.

The Observer Python files also compiled cleanly in an independent compile check: 23 Python files compiled successfully.

That matters because this is not only a frontend shell.

The backend/read-model side exists in Python files such as queues.py, workspaces.py, policy.py, observability.py, source_docs.py, runtime_profiles.py, workflows.py, artifacts.py, and runs.py.

Why this is real work

This is real work for five reasons.

First, the implementation exists as actual source code.

Excluding build artifacts, dependencies, .next, out, test-results, and cache files, the Observer upload contains about 202 source/config files and roughly 42k lines of implementation/config/test code.

That does not include the dependency lockfile.

This is not a one-file prototype.

Second, the frontend is decomposed into real feature areas: overview, workflows, queues, runs, evidence, workspaces, policy, telemetry, search, events, runtime profiles, layout, fixtures, types, contracts, UI primitives, and tests.

Third, the backend/read-model side exists.

Observer is not only a static React shell.

Fourth, the documentation trail is extensive and structured.

The docs include roadmaps, slice plans, checkpoint summaries, screenshot directories, and final review proof.

It looks like a real engineering pass, not after-the-fact storytelling.

Fifth, the final checkpoint is honest about carry-forward gaps.

It does not pretend the Observer is perfect.

It records that the global header refresh-pill/layout jump still needs a later shell/runtime pass, that trace/log rows remain honest not_emitted until approved observability query paths exist, and that a third visual-polish pass is recommended.

That honesty is part of the proof.

A system that calls itself done too early is not governed implementation.

It is just confidence with a UI.

What this proves in the Scarab thread

This is the third major proof direction in the Scarab work.

The first was narrow patching.

Given repo truth, rules, ownership, boundaries, and validators, an AI coding agent can make narrow bug patches without drifting across unrelated repo surfaces.

The second was stepwise quieting.

Given a noisy repo surface, Scarab can run a loop: find the hotspot, identify the boundary, make the bounded repair, rerun, and step down toward quiet.

The Observer build tests the next layer:

long-horizon AI implementation.

Not one patch.

Not one repair.

Not one quick feature.

A sustained implementation campaign across a real repo and a complex stack.

The important point is not volume.

Anyone can make an agent generate code volume.

The important point is behavior.

Can the agent keep going without losing the thread?

Can it preserve contracts?

Can it avoid fake data?

Can it label missing signals honestly?

Can it checkpoint stages?

Can it capture proof?

Can it refuse to call incomplete work done?

That is what this build tested.

The conclusion

Observer Gold Pass 2 landed.

The correct claim is not:

“Observer is finished forever.”

The correct claim is:

Observer Gold Pass 2 landed as a real, validator-gated, SDS-guided implementation pass across the Observer workbench surface, with source code, tests, read models, screenshots, runtime validation, and documented carry-forward gaps.

And the stronger product proof is:

Codex wrote the code, but the run was kept aligned by Scarab’s repo-specific implementation guidance, ownership/boundary structure, validator gates, checkpointing, and refusal to call incomplete work done.

That is real governed implementation work.

This is the layer I have been trying to build toward.

Not AI code generation.

Not vibe coding.

Not “give the model a bigger prompt.”

Repo truth surfaced continuously enough that an AI coding agent can keep implementing without drifting away from the system it is supposed to serve.

That is the experiment.

Observer landed.

And it landed as real work.

From Narrow Patches to Long-Horizon AI Implementation

Scarab Systems — Fri, 26 Jun 2026 12:56:10 +0000

I’m starting to see the larger thread in the Scarab field work.

At first, the question was narrow:

Can repo truth help an AI coding agent make a precise patch without drifting across the codebase?

That has been showing up in the public field tests.

Given a baseline of repo truth, rules, ownership, boundaries, and validators, the coding agent does not need to “understand the whole repo” in some vague, giant-context way.

It needs the right implementation surface.

It needs to know what is true, what owns the area, what boundary applies, and what proves the change is safe.

That is how we have been getting narrow patches into complex repositories.

From patching to quieting

The next question was different.

Instead of targeting one bug, could Scarab work through a noisy repo surface and bring it down to quiet?

That became the stepwise quieting experiment.

In that mode, Scarab does not try to fix everything in one giant patch.

It surfaces the next hotspot, identifies the boundary around it, repairs only what the evidence supports, reruns the diagnostic, and steps down again.

The loop looks like this:

hotspot
boundary
bounded repair
rerun
step down
repeat until quiet

That experiment mattered because messy repo work often turns into weeks of circling.

You fix one thing.

Another surface gets noisy.

The issue moves.

The boundary was actually somewhere else.

The patch gets wider.

The repo gets harder to reason about.

Stepwise quieting tested whether a repo could be brought from noisy to quiet through a controlled diagnostic loop instead of guesswork, broad patching, or agent freewheeling.

The third question: long-horizon implementation

Now I’m testing a third question:

Can repo truth support long-horizon AI implementation?

Not one patch.

Not one repair.

Not one quick feature.

A long implementation campaign across a real, complex stack.

The current test case is Scarab’s own Observer layer: the internal operator console I’m building so I can see Scarab’s diagnostics, telemetry, workspaces, evidence, runtime state, gates, patch surfaces, and implementation visibility while the system is running.

This is not a toy app.

The Observer stack includes:

Next.js
React
TypeScript
shadcn UI
shadcn/Tailwind v4
Tailwind CSS
Radix UI
TanStack Query
TanStack Table
Zustand
React Flow
Monaco Editor
@monaco-editor/react
Apache ECharts
Playwright
pnpm
Node.js
Python
RabbitMQ
Celery
JSON Schema
Lucide React
CMDK
Class Variance Authority
clsx
tailwind-merge
Docker
Docker Compose

The important thing is not that Codex is writing code.

Codex is writing the code.

The important thing is what keeps Codex aligned while it writes.

Scarab is not telling Codex what code to write

In this mode, Scarab provides repo-specific implementation guidance along the way.

Not exact code instructions.

Implementation guidance.

Questions like:

What owns this surface?
What boundaries apply?
What contracts already exist?
What validators must pass?
What gaps are real?
What should not be invented?
What is the next lawful implementation step toward the target architecture?

That guidance appears to be the difference.

Instead of giving Codex a giant prompt and hoping it remembers the system, Scarab keeps resurfacing the repo’s current truth as the implementation proceeds.

That matters because long-horizon agent work is where the usual problems begin.

The agent can generate code.

The agent can pass a local test.

The agent can make a screen look plausible.

But can it keep working for hours without losing the thread?

Can it preserve contracts?

Can it avoid fake data?

Can it distinguish visible progress from validated completion?

Can it leave rollback checkpoints?

Can it refuse to call the work finished before final gates pass?

That is what this Observer build is testing.

Pass 1: foundation

Pass 1 took about 8 hours.

That pass did not try to jump straight to the gold-standard mockups.

It took the Observer foundation I already had and absorbed the target direction into what was actually wired: routes, contracts, read models, tests, checkpoint summaries, and working panel surfaces.

That was important.

The goal was not to let an AI agent paint over the repo with a beautiful fake dashboard.

The goal was to preserve the under-the-hood truth while turning the Observer into a real operator console.

Pass 2: gold pass

Pass 2 is now underway.

This is the gold pass.

It is moving panel by panel toward the richer Observer console: dark dense operator shell, route workbenches, visual proof screenshots, backend/read-model contracts, Playwright coverage, and validator-gated checkpoints.

Current state from the run:

branch ahead by 42 commits
Stages 1 through 8 checkpointed
Policy 4 of 6 slices committed
remaining work includes Policy finish, Observability/Telemetry, Search correlation, final gold review, runtime verification, and screenshot proof

The run estimated itself at roughly 75–80% complete by checkpoint count, but closer to 65–70% complete by remaining integration risk.

That distinction is one of the most interesting parts.

It is not saying:

lots of code was written, therefore done

It is tracking remaining risk.

It knows which stages are checkpointed.

It knows what is still uncommitted.

It knows final screenshots, runtime checks, Playwright coverage, mockup comparison, and unresolved polish issues still have to pass before the work can be called complete.

That is the behavior I wanted to see.

What the screenshots show

The screenshots from Pass 2 are work-in-progress proof.

They are not the final gold state.

But they show the Observer becoming real: overview, workflow, evidence, queues, runs, workspaces, policy, and read-only operator surfaces starting to take shape inside a coherent dark console.

The point is not that the UI is finished.

The point is that the implementation campaign is still moving through checkpoints instead of collapsing into a pile of generated code.

The thread

For me, this ties the Scarab work together.

Narrow patching tests whether repo truth can guide bounded repairs.

Stepwise quieting tests whether repo truth can reduce a noisy repo surface to quiet.

The Observer build tests whether repo truth can sustain long-horizon AI implementation over time.

That is the part I care about.

Not “AI wrote a dashboard.”

Not “Codex generated a lot of code.”

The question is whether an AI coding agent can keep implementing inside a real repo without drifting, if the repo continuously surfaces what is true, what owns what, what boundaries apply, and what proves the next step is safe.

That is the experiment.

And so far, it is real work.

I Thought I Was Building a Diagnostic Tool. I Found an Operating Layer for AI Agents.

Scarab Systems — Sat, 20 Jun 2026 15:30:27 +0000

When Scarab Diagnostic Suite started taking shape, I thought I was building diagnostics.

That was the obvious word for it.

AI coding agents were changing code quickly. Repos were drifting. Bugs were appearing in strange places. Tests were passing without proving the right thing. Files were taking ownership of behavior they should not own.

So the first question was simple:

Can a system drop into a messy repo, locate the real failure surface, and identify the narrow repair lane without making everything worse?

That is where Scarab started.

And that mode matters.

Because anyone working seriously with AI-assisted software development has seen the pattern by now.

The agent patches one issue.

The patch disturbs another part of the system.

The agent patches that.

Then another hidden assumption breaks.

Then a test gets adjusted.

Then a workaround becomes structure.

Then, eventually, the repo is "working" in a way that no longer fully resembles itself.

The system runs.

But the truth drifted.

The failure class

That is the failure class Scarab Systems is built around.

Not just broken code.

Software drift.

Boundary failures.

Repo-truth misalignment.

Verification gaps.

Entropy.

The quiet moment when the code appears functional, but the repo has moved away from its own architecture, obligations, and intended shape.

Over the last few weeks, Scarab has been field-tested across more than 30 public software failure surfaces, including patches and diagnostic reports against major open-source platforms.

That public field record matters because it proves the first mode:

Scarab can enter a live failure surface, identify the boundary that stopped preserving truth, and guide a narrow repair lane that does not casually break the rest of the repo.

But the more interesting discovery was not only that Scarab could help find bugs.

The more interesting discovery was that truthful repair behaves differently.

Truthful repair behaves differently

Most patching work tends to chase symptoms.

A bug appears here.

A workaround appears there.

A test gets adjusted somewhere else.

The repo slowly becomes a negotiation between visible errors and local patches.

But when a repair is aligned with the repo's actual truth, something else happens.

The repair does not just silence the visible failure.

It moves the repo closer to itself.

That distinction matters.

A repo does not need to become some abstract "working version" of itself.

It needs to preserve the system it is actually obligated to be.

That is why I stopped thinking about Scarab as only a diagnostic tool for individual failures.

The second mode is stepwise repair.

Stepwise repair, not entropy management

In messy AI-assisted repos, failures are not always isolated.

They collect in hot spots.

One bug may be the visible edge of a deeper boundary problem.

One test failure may reflect a responsibility that moved.

One runtime issue may expose a false assumption that has spread.

One generated artifact may have quietly been treated as source truth.

If the coding agent repairs one bug at a time without understanding the repo's truth boundaries, it can go in circles forever:

Patch the bug.
Patch what the patch broke.
Patch the new workaround.
Patch the test.
Patch the side effect.
Patch the patch.

That is not repair.

That is entropy management.

Scarab is being developed around a different theory:

If repairs happen along the repo's truthful boundaries, the system can be brought step by step toward quiet.

Not quiet because the errors are hidden.

Quiet because the repo is becoming more coherent.

That is a very different thing.

A truthful boundary repair can ripple outward in a good way.

It can reduce pressure on nearby failure surfaces.

It can clarify ownership.

It can remove the need for a workaround.

It can make a test meaningful again.

It can restore the difference between source truth and downstream output.

It can bring the system closer to its baseline instead of dragging it into a patched abstraction of itself.

That is the part I think matters for the future of AI coding agents.

Scarab does not replace the coding agent

The next serious leap is not simply making agents faster.

They are already fast.

The next leap is making sure they do not destroy coherence while they move.

But this part is important:

Scarab is not trying to replace the AI coding agent.

It is not another model.

It is not a second developer.

It is not a magical correctness oracle.

Scarab Diagnostic Suite is a technical diagnostic layer that produces the right kind of repo-grounded findings for the coding agent to act on.

The agent is still the implementer.

The human still gives intent.

Scarab reads the repo, identifies the relevant truth surfaces, exposes the boundary conditions, and returns evidence-backed findings that help the agent stay inside the right lane.

That is why the system can be software-agnostic and agent-agnostic.

It does not need to own the repo.

It does not need to replace the workflow.

It does not need to care whether the developer is using Codex, Claude Code, Cursor, Copilot, Devin, a local model, or a human engineer.

The role is simpler and more technical:

Give the implementer better findings.
Make the repo's truth easier to see.
Keep the repair or build path aligned with the system that already exists.

The agent should not grade its own correctness

True autonomous coding agents cannot be built on guesswork alone.

They need a deterministic layer outside the model.

A layer that can tell the agent:

This surface owns this behavior.
This boundary cannot move casually.
This artifact is not source truth.
This test proves this claim.
This config carries this runtime obligation.
This repair lane is narrow.
This change preserved the repo.
This change drifted.

The agent should not have to invent the repo's architecture every time it opens a task.

It should not have to decide what is canonical from whatever happens to fit in context.

It should not grade its own correctness.

It should not silently move system boundaries and then announce that the build passed.

That is too much authority for a probabilistic worker inside a complex system.

The repo needs its own governed relationship to truth.

Three modes of Scarab

That is where Scarab is moving next.

Right now, I see three operating modes taking shape.

1. Public field diagnostics

Enter a failure surface.

Identify the boundary.

Produce a narrow repair lane.

This is the easiest mode to see publicly because it shows up in open-source issues, patches, and field reports.

A repo has a visible failure.

Something drifted.

Scarab asks:

What truth was this system supposed to preserve?

Which boundary stopped preserving it?

What evidence proves where that happened?

2. Stepwise repo quieting

Move through hot spots in a sequence that brings the system closer to coherence instead of chasing symptoms forever.

This matters because messy repos do not always fail one bug at a time.

They often fail around pressure points.

If repair follows the repo's truthful boundaries, one good repair can lower pressure elsewhere.

Not because the system was magically fixed.

Because the repair moved the repo closer to its actual obligations.

3. Active agentic governance

This is the mode I am working toward now.

Continuous monitoring while AI-assisted development is happening.

Not just:

What broke?

But:

What is the agent about to build on?
Is it building on repo truth or residue?
Is this feature strengthening the system or introducing a hidden drift surface?
Did this layer preserve the boundaries beneath it?
Did the repo become more coherent after the change?

That is the future I am interested in.

The missing layer

Humans describe intent in real language.

Repos preserve truth mechanically.

AI coding agents operate inside governed boundaries.

Diagnostics verify whether the system stayed coherent.

Each new feature should not make the repo heavier, stranger, and harder to trust.

Each layer of work should strengthen the repo as itself.

That is the real promise of AI-assisted development.

Not just more code.

Not just faster code.

Not a world where humans spend all day babysitting the thing that was supposed to give them leverage.

A world where the agent can move quickly because the operating boundary is clear.

A world where the human does not have to manually hold every architectural truth in their head.

A world where the repo can tell the agent what must remain true.

That is the layer I think serious autonomous software development is missing.

And that is what Scarab Systems is being built to explore.

The public field reports are not the end of the story.

They are the proof trail.

The bigger implication is that repo-side deterministic governance may be one of the missing foundations for true autonomous coding agents.

Because autonomy without truth is just acceleration.

And acceleration without truth is drift.

Scarab Diagnostic Field Test #034 - Prometheus Docker Swarm Label Source Boundary

Scarab Systems — Fri, 19 Jun 2026 23:48:26 +0000

Target: prometheus/prometheus

Issue: prometheus/prometheus#12244

Pull request: prometheus/prometheus#18979

Field Lab record: Prometheus #12244

This field test was about documentation, but not "just documentation."

The public issue reported confusion around Docker Swarm task discovery metadata in Prometheus.

The user expected __meta_dockerswarm_container_label_<labelname> to expose labels from the running container or image, such as OCI image labels.

The visible behavior was different.

The metadata existed, but the source was not the runtime container or image label surface the user expected.

That made this a boundary problem:

what does the documented metadata name mean, and which public source of truth does it actually represent?

Field Lab record

The public Field Lab record for this case is here:

https://github.com/scarab-systems/scarab-field-lab/tree/main/field-tests/prometheus-prometheus-12244

The record includes the public issue, the public pull request, a public comment, the repair scope, validation summary, and non-claims.

It contains public links, status, validation, and claim boundaries only. It does not contain SDS source code or non-public product material.

The case record is only the public evidence layer.

SDS result

SDS surfaced a documentation boundary around Docker Swarm task discovery.

The important finding was not that Prometheus should add a new Docker API call.

It was not that service discovery should begin inspecting image labels.

It was not that runtime behavior should change.

The finding was smaller:

Prometheus documentation described a metadata label in a way that could reasonably be read as "labels from the container," while the public code path populates that metadata from the Swarm task's container spec.

That difference matters because service-discovery metadata is used as an operational contract.

People write relabeling rules against these names.

If the source of a metadata field is ambiguous, users may look for data in the wrong place, configure the wrong thing, or assume a missing label is a Prometheus bug when it is really a source-boundary mismatch.

Failure shape

The failure shape was a mismatch between expectation and source.

The public issue described a reasonable operator expectation:

if the metadata name says container_label, it might refer to labels visible on the running container or inherited from the image.

But Docker Swarm task discovery has a more specific source.

The relevant metadata is populated from the task's container specification.

That is close enough in language to sound obvious after the fact, but not close enough for an operator reading the docs while debugging a production scrape configuration.

That is why the field test did not treat this as a runtime bug first.

The problem was not proven at the behavior layer.

The problem was at the wording boundary:

the docs did not make the source specific enough.

Boundary

The boundary in this field test was:

Docker Swarm task metadata should distinguish labels from the Swarm task container spec from labels on a running Docker container or image.

Those are different surfaces.

They may overlap in everyday language.

They may even contain similar-looking labels in some deployments.

But they are not the same source of truth.

For service discovery, that distinction is important because a metadata label name becomes something users build configuration around.

If the docs leave the source implicit, the user has to reverse-engineer the behavior from Docker API responses and Prometheus discovery code.

That is not a good boundary for public configuration documentation.

What changed

The repair is a one-line documentation clarification in the Prometheus configuration docs.

The pull request is here:

https://github.com/prometheus/prometheus/pull/18979

The changed public file is:

docs/configuration/configuration.md

The repair clarifies that the Docker Swarm task metadata label comes from the Swarm task's container spec.

That is the whole patch.

No runtime behavior change.

No new Docker discovery logic.

No new Docker API calls.

No attempt to make Prometheus inspect image labels.

Just a clearer public contract for the metadata that already exists.

Why this was not a code repair

One of the risks with code-agent work is that everything starts to look like a code patch.

This field test is a useful counterexample.

The public issue was real.

The confusion was real.

The operator experience was real.

But the repair boundary was documentation.

That matters.

If a code agent had treated this as "missing labels," it could have reached for runtime behavior: inspect more Docker surfaces, add a new metadata path, or change discovery behavior.

That would have been a much larger claim.

The narrow claim here is simpler:

the existing behavior should be documented with a more precise source.

That kind of restraint is part of the theory I am testing with Scarab:

the quality of the repair depends on finding the correct boundary before editing.

Sometimes the correct boundary is code.

Sometimes it is tests.

Sometimes it is docs.

In this case, it was docs.

Why the diagnostic result mattered

The larger point of this field test is not that a documentation line changed.

The larger point is that a code agent should not decide the repair shape from momentum.

It should not start from:

"what can I change?"

It should start from:

"where did the repo's public truth become unclear?"

For this case, the public truth was about the meaning of a service-discovery metadata label.

The failure was not that the metadata name existed.

The failure was that the name could point readers toward the wrong source.

Once the boundary was clear, the patch became small.

That is the pattern Scarab is trying to make repeatable:

find the truth,

find the boundary,

patch the boundary.

Validation

The repair was validated against the Prometheus repository with the relevant public checks recorded in the Field Lab.

Validation recorded in the Field Lab:

go test ./discovery/moby -count=1 passed
go build ./cmd/prometheus/ passed
GO_ONLY=1 make test passed
make lint passed
make test passed

At the time this draft was prepared on June 19, 2026, the public pull request was open and ready for review.

The public status is:

PR open
DCO passing
Netlify deploy-preview successful or informational
upstream review required

This field report does not claim upstream acceptance.

It claims a public diagnostic record, a narrow documentation repair, and a submitted PR.

Maintainers decide whether the clarification belongs upstream.

Field test result

Result:

diagnostic proof and documentation repair submitted.

The field test produced:

a public issue-to-boundary record
a narrow documentation patch
validation against the affected repository
a public upstream PR
a public status record in the Scarab Field Lab

The patch is deliberately small.

That is the point.

When the failure is a source-boundary ambiguity, the best repair may be a sentence that makes the source explicit.

Public claim

This field test supports a narrow public claim:

SDS identified a Docker Swarm task discovery documentation boundary in Prometheus where container-label metadata needed a clearer source, and a human-submitted documentation repair was prepared for that boundary.

It does not claim:

that Prometheus accepted the patch
that Prometheus should inspect Docker image labels
that Docker Swarm discovery behavior changed
that Scarab repairs projects by itself
that SDS source or product details are public
that maintainers endorsed Scarab or the Field Lab

The Field Lab exists to keep those claims separate.

Disclosure

Scarab Diagnostic Suite is proprietary. The Field Lab publishes public case records, issue links, validation summaries, and claim boundaries only.

SDS finds evidence. People make claims. Maintainers decide.

Scarab Diagnostic Field Test #033 - Prometheus Remote-Write Label Order Boundary

Scarab Systems — Fri, 19 Jun 2026 23:46:09 +0000

Target: prometheus/prometheus

Issue: prometheus/prometheus#11505

Pull request: prometheus/prometheus#18978

Field Lab record: Prometheus #11505

This field test was about a small but important remote-write boundary in Prometheus:

labels are required to be sorted lexicographically, but incoming remote-write labels could pass through a conversion path before that requirement was enforced.

That sounds narrow because it is narrow.

But this is exactly the kind of case Scarab is meant to make visible.

Not "the remote-write subsystem is broken."

Not "rewrite ingestion."

Not "let an agent go hunting."

The question was smaller:

where does repo truth say invalid input should stop, and where was the actual boundary allowing that truth to blur?

Field Lab record

The public Field Lab record for this case is here:

https://github.com/scarab-systems/scarab-field-lab/tree/main/field-tests/prometheus-prometheus-11505

The record includes the public issue, the public pull request, the repair scope, validation summary, and non-claims.

It contains public links, status, validation, and claim boundaries only. It does not contain SDS source code or non-public product material.

That distinction matters.

The public artifact is the field record, not SDS itself.

SDS result

SDS surfaced a boundary mismatch around remote-write label ordering.

The public issue already identified the expected rule:

remote-write labels are supposed to be sorted lexicographically.

The meaningful diagnostic result was not just "sort labels."

In fact, the repair does not sort the incoming data.

The result was that invalid remote-write input should be rejected before it can be made to look acceptable by a later conversion path.

That is the boundary.

The input contract belongs at the edge where the remote-write request is being interpreted.

If invalid input crosses that edge and then gets normalized, the system can lose the ability to distinguish valid caller behavior from repaired caller behavior.

Failure shape

The failure shape was subtle:

Remote-write input arrives with label names in an invalid order.
Prometheus has a public and internal expectation that labels are sorted.
A conversion path can produce Prometheus labels in sorted form.
That means invalid input can appear clean after conversion.

The concern is not that sorting is technically impossible.

The concern is that sorting at the wrong layer changes what the receiver is saying to the caller.

If a remote-write sender sends invalid data, the receiver should be able to reject that data as invalid input.

If the receiver silently normalizes it, the caller gets a different contract:

"send whatever order you want; the receiver will fix it."

That was not the boundary described by the issue.

Boundary

The boundary in this field test was:

remote-write request validity must be checked before incoming label data is converted into Prometheus' internal label representation.

That is a code boundary, but it is also a contract boundary.

On one side is the external remote-write request.

On the other side is Prometheus' internal representation.

The repair belongs at the crossing point.

Once the request has crossed into internal representation, a validation failure can become harder to prove because the data may already have been reshaped.

That is why the patch is intentionally placed before append behavior, and before the conversion path can erase the original ordering problem.

What changed

The patch adds explicit label-order validation for both supported remote-write paths covered by the repair:

remote-write v1 label names are checked before v1 samples are appended
remote-write v2 label references are checked before v2 samples are appended
v1 unsorted-label series follow the existing invalid-label skip path
v2 unsorted-label series follow the existing partial-write bad-request path
regression coverage was added for both request shapes

The pull request is here:

https://github.com/prometheus/prometheus/pull/18978

The changed files are public:

storage/remote/write_handler.go
storage/remote/write_handler_test.go

The patch does not redesign remote-write ingestion.

It does not change remote-read.

It does not introduce a new validation framework.

It keeps the repair at the specific boundary identified by the field test.

Why this was not a broad rewrite

This is one of the recurring lessons from Scarab field testing:

when the boundary is clear, the repair should get smaller, not larger.

A code agent can produce a large patch very quickly.

That is not the hard part anymore.

The hard part is deciding what the patch is allowed to claim.

For this case, the claim is not:

"remote-write validation is now perfect."

The claim is:

"incoming remote-write labels that are out of lexicographic order are checked before they can be normalized into the internal label representation."

That is a much narrower statement.

It is also a more reviewable one.

Why the diagnostic result mattered

The practical bug is about label ordering.

The larger field-test value is about method.

Modern code agents are very good at moving through a repository and producing edits. That can be useful, but it can also make patches feel suspicious when the intent is unclear.

Scarab's position is different:

first find the truth boundary, then patch only that boundary.

In this case, the repo truth was that labels must be sorted.

The boundary failure was that remote-write input could cross into a representation where the original invalid ordering was no longer visible.

The patch follows from that.

Not because the agent guessed a fix.

Because the public issue, the code surface, and the validation boundary pointed to a narrow repair.

That is the shift I am testing in public:

code-agent work should not begin with "what can we generate?"

It should begin with "where did the repo stop enforcing what it already says is true?"

Validation

The repair was validated in a Linux arm64 container with the Prometheus test workflow used for the public PR.

Validation recorded in the Field Lab:

make test passed
make lint passed
go test ./storage/remote -count=1 passed

At the time this draft was prepared on June 19, 2026, the public pull request was open and ready for review.

The public status is:

PR open
DCO passing
Netlify deploy-preview successful or informational
upstream review required

That status matters too.

This field report does not claim upstream acceptance.

It claims a public diagnostic record, a narrow repair, and a submitted PR.

Maintainers decide whether the patch belongs upstream.

Field test result

Result:

diagnostic proof and repair submitted.

The field test produced:

a public issue-to-boundary record
a narrow repair patch
regression coverage for the repaired behavior
a validation summary
a public upstream PR

The patch is intentionally boring.

That is the point.

The interesting thing is not that a code change exists.

The interesting thing is that the repair boundary is explainable without exposing proprietary diagnostic logic or asking maintainers to trust a black box.

Public claim

This field test supports a narrow public claim:

SDS identified a remote-write label-order boundary in Prometheus where invalid input needed to be checked before conversion could normalize it, and a human-submitted repair was prepared for that boundary.

It does not claim:

that Prometheus accepted the patch
that all remote-write validation issues are resolved
that Scarab repairs projects by itself
that SDS source or product details are public
that maintainers endorsed Scarab or the Field Lab

The Field Lab exists to keep those claims separate.

Disclosure

Scarab Diagnostic Suite is proprietary. The Field Lab publishes public case records, issue links, validation summaries, and claim boundaries only.

SDS finds evidence. People make claims. Maintainers decide.