DEV Community: Scarab Systems

Scarab Diagnostic Field Test #029 — Electron CSP / Isolated Preload Boundary

Scarab Systems — Mon, 15 Jun 2026 01:50:22 +0000

Field test status: diagnostic pass completed; upstream direction pending.

This one is different from the merged patch reports.

The Electron case produced a narrow draft repair and a focused regression test, but it did not land as-is. Maintainer review raised a security-boundary concern, and the repair lane was paused pending clarification of Electron’s intended behavior.

That makes it a useful field test anyway.

Sometimes a diagnostic field test proves the patch.

Sometimes it proves the boundary question.

This one did the second thing.

Target

Repository: electron/electron

Issue: #48240

Draft PR: #51991

Issue title: [24.1.0 regression] Unsandboxed preload is restricted by Content-Security-Policy, but only after readyState becomes interactive

Public PR title: fix: don't apply page csp to isolated preload codegen

The reported failure

The issue reported inconsistent behavior in Electron preload execution.

In an unsandboxed preload script with contextIsolation: true, string code generation such as new Function(...) could be allowed early in preload execution, but later become blocked after the document moved into the interactive phase.

That matters because some libraries detect code-generation support once, cache the answer, and then rely on that answer later.

So the failure was not simply “CSP blocks eval.”

The failure was temporal inconsistency.

The same preload context appeared to answer the code-generation question one way at the beginning of execution and another way after document parsing progressed.

That is a drift-shaped failure.

A truth changed mid-context.

Why this is a boundary problem

Electron sits between several worlds at once:

Chromium page behavior.

Electron preload behavior.

Node-enabled application behavior.

Content Security Policy.

Context isolation.

Application security expectations.

That makes the boundary important.

The diagnostic question was not only:

“Should code generation be allowed?”

The better question was:

“Which policy owns code generation inside an isolated preload world, and should that answer change after document parsing begins?”

That is the boundary.

The page has a Content-Security-Policy.

The preload script runs in an Electron-managed isolated world.

The app may rely on preload behavior.

The security model may rely on the page CSP applying transitively.

If those ownership lines are unclear, the runtime can become inconsistent.

That is exactly what the issue exposed.

The first repair lane

The draft repair tested one interpretation of the boundary:

If the preload script runs in Electron’s isolated world, then page CSP should not make preload string code generation change after document parsing begins.

The patch added a check for Electron’s isolated world and routed that case around the page CSP code-generation callback, while keeping page CSP enforcement for main-world renderer code.

It also added a regression test for an unsandboxed, context-isolated preload under a restrictive page CSP. The test verified that string code generation stayed consistently allowed both before and after DOMContentLoaded.

That was a narrow patch.

Two files changed:

shell/common/node_bindings.cc

spec/chromium-spec.ts

The focused regression test passed locally.

Maintainer review changed the repair lane

Electron maintainer review raised the key concern:

If apps are already relying on the page CSP as a transitive guard against eval-like behavior in isolated preload code, then bypassing page CSP for isolated preload code generation may weaken the security posture.

That is the important moment in this field test.

The diagnostic pass found a real inconsistency.

The first repair lane made the behavior consistent in one direction.

Maintainer review clarified that consistency in that direction may not match the intended security boundary.

So the responsible next step is not to force the patch.

The responsible next step is to pause and ask which boundary Electron intends:

Should page CSP block preload eval-like code generation consistently from the beginning?

Or should Electron eventually expose a separate, explicit isolated-world code-generation control?

That is now the design question.

Why this still belongs in Field Lab

A field test does not have to end with a merged PR to be valuable.

The diagnostic value here is that the failure was reduced from a confusing runtime regression to a precise boundary question:

Should an isolated preload world inherit page CSP code-generation limits?

If yes, the current behavior is inconsistent because enforcement appears only after document parsing progresses.

If no, the current behavior is inconsistent because page CSP eventually reaches into a world it should not govern.

Either way, the bug is not random.

It lives at the ownership boundary between page security policy and Electron preload execution.

That is the Scarab-relevant finding.

What this case shows

This case shows why software drift diagnostics cannot stop at “make the failing behavior consistent.”

Consistency is not automatically correctness.

A repair can make a system internally consistent while still moving the wrong security boundary.

That is especially true in runtime platforms like Electron, where a boundary may be both functional and security-sensitive.

The first patch made one interpretation explicit.

Maintainer review surfaced another possible truth: some applications may depend on the current transitive CSP behavior as a security guard.

That changed the repair question.

Not because the failure disappeared.

Because the authority question became clearer.

The Scarab reading

The issue was not merely a code-generation bug.

It was a policy-carrier bug.

The code-generation answer changed as the document lifecycle advanced.

The same preload context did not receive a stable answer across execution phases.

The draft repair proved that the behavior could be made consistent.

Maintainer review proved that the direction of consistency matters.

That is the lesson.

In drift work, the goal is not just to remove a diff or quiet a failing case.

The goal is to identify which claim owns the behavior and what evidence authorizes that claim to move.

In this Electron case, the claim is still under maintainer/design review:

Does page CSP own isolated preload code generation?

Or does isolated preload code generation need its own explicit control surface?

Until that is answered, the safest public field-test result is not “patch accepted.”

It is:

Boundary isolated. Repair lane paused. Upstream direction pending.

Field result

Result: Significant diagnostic field test.

Patch status: Draft PR opened; not landed as-is.

Maintainer feedback: Security-boundary concern raised.

Current posture: Awaiting upstream direction on intended CSP / isolated-preload ownership.

Diagnostic finding: Electron issue #48240 exposes a lifecycle-dependent policy boundary failure between page CSP enforcement and isolated preload code generation.

Repair lane: Pending explicit decision on whether page CSP should consistently govern isolated preload code generation, or whether Electron needs a separate isolated-world control surface.

Why this matters beyond Electron

Electron is a boundary-dense platform.

It combines browser security models, desktop application power, Node integration, preload scripts, renderer isolation, and application-defined trust decisions.

That is exactly the kind of environment where software drift becomes subtle.

A behavior can be technically consistent with one layer and wrong for another.

A patch can fix a regression and weaken a security assumption.

A test can prove behavior without proving policy authority.

This is why field diagnostics matter.

The hard part is not always writing the patch.

Sometimes the hard part is finding the exact question the patch must answer before it deserves to land.

Repo truth and governance

This field test also points at the broader theory behind Scarab.

Every repository has truth.

That does not mean every repository has perfect documentation, perfect tests, or perfect architecture. It means the codebase contains obligations that must remain true for the system to keep working as itself.

Those truths are not floating abstractions.

They appear in the repo’s components.

They appear in the way those components interact.

They appear in boundaries, contracts, responsibilities, generated artifacts, runtime assumptions, configuration rules, security models, and tests.

A repo’s truth is not found in one file.

It is distributed across the agreements the system depends on.

That is why a boundary failure matters. When a boundary stops carrying the truth it was responsible for preserving, the repo can still look healthy. It can still build. It can still pass a focused test. It can even become more internally consistent.

But it may be preserving the wrong claim.

That is where governance enters.

Not governance as an ethics slogan.

Not governance as a policy document sitting somewhere outside the work.

Repo-truth governance is the mechanical process of keeping the codebase’s own truths from being silently rewritten.

It is the checks and balances that ask:

Which claim owns this behavior?

Which surface has authority here?

Which boundary is responsible for carrying that claim forward?

Did the change preserve that truth, move it, weaken it, or bypass it?

What evidence proves the movement was legitimate?

This is a different lens for AI-assisted development.

Right now, AI coding agents are often dropped into repositories with access to files, tests, and instructions, but without a governed relationship to repo truth.

They can change code.

They can change tests.

They can change config.

They can change documentation.

They can make the project appear coherent around the thing they just changed.

But unless something mechanical is governing the repo’s truth, the agent is not really being guided by the system’s obligations. It is navigating a field of states, files, and feedback loops.

That is not enough.

Governance is the bridge between repo truth and AI function.

If the repo has truth, and an AI agent is allowed to operate inside that repo, then the agent needs more than context.

It needs boundaries.

It needs authority rules.

It needs evidence gates.

It needs checks and balances that prevent it from turning a local patch into a global lie.

This Electron field test is a small but sharp example.

The first repair lane made the behavior consistent.

The maintainer review asked whether that consistency would preserve the correct security boundary.

That is repo-truth governance in action.

The patch did not simply ask, “Can this be changed?”

The real question became:

“Does this change preserve the truth Electron is responsible for maintaining?”

That is the level of the problem Scarab is built to diagnose.

Closing note

This field test is important because it demonstrates a different kind of Scarab outcome.

Not every successful diagnostic pass ends in an immediate merge.

Sometimes the value is that a confusing bug becomes a precise governance question.

For Electron, the question is now much cleaner:

What owns code-generation authority inside an isolated preload world?

That is the field finding.

And more broadly:

What owns truth inside a repository, and what governs whether that truth is preserved?

That is the industry question.

Disclosure: This field report was written with AI-assisted editing and summarization. The underlying diagnostic run, repair branch, PR status, validation commands, and technical claims come from my own Scarab/SDS field-test work and were reviewed by me before posting

The Schema Said Boolean. The Generator Wrote a String. OpenAPI Generator Merged the Fix.

Scarab Systems — Sun, 14 Jun 2026 07:31:48 +0000

Scarab Diagnostic Field Test #027 has landed upstream.

Target: OpenAPITools/openapi-generator

Issue: OpenAPITools/openapi-generator#23550

PR: OpenAPITools/openapi-generator#24022

Status: merged

Milestone: 7.24.0

Field Lab: https://github.com/scarab-systems/scarab-field-lab/blob/main/field-tests/openapitools-openapi-generator-23550/README.md

This field test targeted a Kotlin generator bug in OpenAPI Generator where an OpenAPI 3.1 boolean const: true schema value was rendered into generated Kotlin source as the string "true" instead of the boolean literal true.

That sounds small.

It is small.

That is why it matters.

In code generation, a tiny boundary mistake can become a broken downstream project. A generator does not get to be “almost right” when it emits source code. If the schema says boolean, the generated target-language literal has to preserve that boolean truth.

In this case, the generated enum value type was kotlin.Boolean, but the generator emitted "true" as a string literal.

The generated Kotlin did not compile.

The patch was accepted, added to the 7.24.0 milestone, and merged upstream.

The issue shape

The visible issue was simple:

An OpenAPI 3.1 schema defined a boolean constant value.

The Kotlin generator produced a model where the enum value type was kotlin.Boolean.

But the emitted enum value was rendered as "true".

That created a type/literal mismatch:

"true"

where the generated Kotlin source needed:

true

This was not a Kotlin language problem.

It was not a user configuration problem.

It was not a broad OpenAPI 3.1 interpretation problem.

It was a generator boundary problem.

The schema truth was boolean.

The Kotlin type truth was boolean.

The emitted source literal drifted into string.

The boundary

The boundary here was:

schema-level boolean truth → target-language literal rendering

OpenAPI Generator does not merely copy schema values into files. It translates schema information into source code for a target language.

That translation layer has a responsibility.

When a schema-derived value crosses into Kotlin source text, the generated literal has to match the generated Kotlin type.

For this issue, the source schema said const: true.

The generated enum context expected kotlin.Boolean.

The literal printer emitted "true".

That is the exact boundary failure.

Not “enum generation is broken.”

Not “OpenAPI 3.1 const support needs a rewrite.”

Not “Kotlin models need a new type system.”

Just this:

a boolean schema value crossed into Kotlin source text as a string literal.

That is the repair lane.

What changed

The merged PR updates the Kotlin generator so boolean enum values are emitted as boolean literals when the target enum value type is kotlin.Boolean.

The repair follows the existing primitive-handling pattern already present for numeric values such as Int and Long.

That detail matters.

This patch did not invent a new enum system.

It did not rewrite the broader generator pipeline.

It did not change OpenAPI schema interpretation globally.

It extended the existing literal-emission behavior to cover the missing boolean case.

The PR added focused regression coverage in two places:

a direct toEnumValue assertion for boolean literal rendering
a generated-output regression fixture for the OpenAPI 3.1 const: true boolean case

That means the repair was protected at both the small rendering-function boundary and the generated-source output boundary.

Why this is a good field test

This is a good Scarab field test because the patch is narrow but the boundary is clean.

A generator is a chain of truth transformations.

Schema truth becomes generator metadata.

Generator metadata becomes target-language type information.

Target-language type information becomes emitted source code.

If any step in that chain changes the meaning of the value, the generated project inherits the break.

Here, the boolean value survived the schema.

It survived the target type.

It failed at literal rendering.

That is exactly the kind of failure Scarab is designed to make legible.

Not because the bug is large.

Because the ownership surface is precise.

OpenAPI Generator owns the generated Kotlin source text.

The Kotlin generator owns how schema-derived enum values are rendered into Kotlin literals.

A boolean schema value should remain boolean when emitted into a kotlin.Boolean enum context.

The patch restored that boundary.

Why the repair stayed small

The tempting mistake in generator bugs is to make the patch bigger than the failure.

OpenAPI Generator supports many target languages, schema forms, generator modes, and target-specific behaviors. A small bug can easily invite a broad abstraction change.

That would have been the wrong repair here.

The issue did not prove that enum generation was globally broken.

It did not prove that Kotlin model typing was wrong.

It did not prove that OpenAPI 3.1 const support needed a broad redesign.

It proved one narrower thing:

a boolean value that should be emitted as a Kotlin boolean literal was being emitted as a string literal.

So the patch stayed at that level.

When the failure is a missing primitive literal case, the repair should add the missing primitive literal case.

That is the discipline.

Find the boundary.

Keep the evidence close.

Patch the owning surface.

Do not turn a precise failure into a broad rewrite.

Validation

The PR included the targeted Kotlin regression coverage and full project validation.

Validation included:

./mvnw clean package
./bin/generate-samples.sh ./bin/configs/*.yaml
./bin/utils/export_docs_generators.sh

The automated review found no issues across the changed files.

The maintainer accepted the patch with:

lgtm. thanks for the fix

The PR was then added to the 7.24.0 milestone and merged.

That is the full loop:

public issue

bounded diagnostic read

narrow patch

regression coverage

maintainer acceptance

upstream merge

Field test result

This was a bounded Kotlin generator literal-emission repair for OpenAPI Generator.

The issue reduced to:

an OpenAPI 3.1 boolean const: true schema reached the Kotlin generator
the generated enum value type was kotlin.Boolean
the emitted literal was incorrectly stringified as "true"
the generated Kotlin was uncompilable
OpenAPI Generator already had primitive literal handling for values such as Int and Long
the Kotlin generator was missing the matching boolean literal path
the PR added narrow boolean handling and focused regression coverage
the patch was accepted, milestone’d, and merged upstream

That is a clean field-test result.

The patch does not claim to redesign OpenAPI 3.1 support.

It does not claim to rewrite enum generation.

It does not claim to alter Kotlin model typing broadly.

It fixes the boundary where boolean schema truth becomes Kotlin source text.

Why this matters

This is the third upstream-merged Scarab field-test repair in a major developer infrastructure project.

The pattern is becoming more important than any single patch.

pnpm merged a bounded repair.

Docker Compose merged a bounded repair.

Now OpenAPI Generator has merged a bounded repair.

Different ecosystems.

Different codebases.

Different failure surfaces.

Same diagnostic shape:

Find where the system stopped preserving the truth another part depended on.

Then patch narrowly.

This is why I keep saying software drift is not only an AI-assisted-code problem.

AI makes drift faster and harder to ignore, but mature human-built systems drift too.

A package manager can drift across command/context boundaries.

A container tool can validate in the wrong phase.

A code generator can preserve the type but corrupt the emitted literal.

The surface changes.

The diagnostic question stays the same:

Which boundary stopped carrying the truth forward?

Public claim

The correct public claim for this field test is:

Scarab/SDS helped drive a bounded repair for OpenAPITools/openapi-generator#23550, where the Kotlin generator emitted OpenAPI 3.1 boolean const: true enum values as "true" strings even though the generated enum value type was kotlin.Boolean.

The upstream PR added narrow Kotlin generator handling so boolean enum values are emitted as Kotlin boolean literals, matching the existing primitive-handling pattern for types such as Int and Long.

The repair includes a focused toEnumValue assertion and a generated-output regression fixture.

The PR was accepted, added to the 7.24.0 milestone, and merged.

This does not claim to redesign OpenAPI 3.1 support or enum generation broadly. It fixes the boolean literal-emission boundary that produced uncompilable Kotlin.

Field Lab

Scarab Systems maintains a public Field Lab for selected diagnostic field tests.

The Field Lab publishes public case records from real open-source issues: the issue being examined, the suspected boundary, the evidence gathered, the validation performed, and the current status of the diagnostic claim.

Some cases may end with a local repair candidate.

Some may become upstream pull requests.

Some may remain diagnostic records only.

That status is part of the record.

Scarab Diagnostic Suite is proprietary, but the larger conversation is shared. Software drift is not one team’s problem, and AI-assisted development is making the need for better diagnostic framing more urgent across the industry.

If you know of a public open-source issue that looks like cross-layer drift, unclear ownership, software drift, AI-assisted code drift, phase confusion, or a boundary failure, you can suggest it as a Field Lab candidate.

Useful suggestions include the public issue link, the suspected boundary, reproduction notes if available, and why the issue may be diagnostically interesting.

The goal is simple:

Make the failure legible.

Find the boundary.

Preserve the evidence.

Repair narrowly.

Let the win trickle down.

Disclosure: This field report was prepared with AI-assisted editing from my own field-test notes, public issue and PR records, validation summary, and repair record. The technical claims and final wording were reviewed before publication.

Autonomous Agentic Workflows... Really?

Scarab Systems — Sun, 14 Jun 2026 05:05:16 +0000

I was going to launch into my usual dev.to appropriate tech heavy piece but honestly I have just one question -

Is this really for real?

After everything I have learned about the rigor required to actually keep an AI coding agent on task I'm just slack jawed at the full steam barreling ahead of these so-called autonomous agents...

I find myself just laughing at every single new AI ad that comes across my laptop...

It can't be real that an entire industry is quite literally missing the point... and expecting AI to find it.

That's all I got.

Scarab Diagnostic Field Test #028 — OpenAPI Generator Rust Server Client Feature Boundary

Scarab Systems — Sat, 13 Jun 2026 20:09:29 +0000

Target: OpenAPITools/openapi-generator

Issue: OpenAPITools/openapi-generator#23920

PR: OpenAPITools/openapi-generator#24023

Field Lab: https://github.com/scarab-systems/scarab-field-lab/blob/main/field-tests/openapitools-openapi-generator-23920/README.md

This field test targeted a Rust server generator bug in OpenAPI Generator where rust-server client-only builds could miss dependencies required by generated shared model code.

The visible issue was not that the Rust generator could not generate a client.

It was more specific:

a rust-server project was generated
the build was run with default features disabled
the client feature was enabled
generated shared model code could still emit pattern validation
that pattern validation could require lazy_static and regex
but the client-only feature path did not reliably wire those dependencies in
the resulting Rust sample could fail under cargo check --no-default-features --features client

That is a classic generated-code boundary bug.

The user is not hand-writing the missing dependency usage.

The generator emits code that can require those crates.

So the generator also has to emit Cargo feature wiring that makes the generated code build under the feature combinations the generator claims to support.

The diagnostic question was not:

How do we make every Rust server feature depend on everything?

The better question was:

Which generated-code path can emit lazy_static and regex, and which Cargo feature boundary needs to own those dependencies?

Field Lab record

The public case record for this field test is available in the Scarab Field Lab:

https://github.com/scarab-systems/scarab-field-lab/blob/main/field-tests/openapitools-openapi-generator-23920/README.md

Upstream posture

This was a clean upstream repair candidate.

The issue was open, the failure was reproducible in a narrow feature configuration, and the repair surface was inside OpenAPI Generator’s owned Rust server template and sample-generation path.

That matters because this was not a dependency wish-list patch.

It was a generator correctness patch.

When a generator emits code behind a supported feature combination, the generated Cargo manifest has to include the dependencies required by that code path.

The upstream PR was opened cleanly and contains no SDS, Scarab, Codex, local-path, or internal workflow language.

SDS result

This field test was run in SDS field-test posture against a disposable OpenAPI Generator arena.

The useful result was a bounded ownership read.

The failure lived across a small but important chain:

OpenAPI schema input
generated Rust shared model code
pattern-validation output
Cargo dependency and feature wiring
client-only Rust sample compilation

That chain points to a generator-owned boundary.

The correct repair was not to remove validation.

It was not to tell users to manually add crates.

It was not to make every generated build pull every dependency all the time.

It was to tighten the rust-server Cargo feature wiring so a client-only build includes the dependencies required by generated shared models when those models emit pattern validation code.

Failure shape

The failure shape was a feature/dependency mismatch.

The generated Rust code could require lazy_static and regex.

The client-only build path could still compile shared models.

But the Cargo feature wiring did not reliably include the crates needed by that shared generated code.

That creates a broken generated project because the feature combination is internally inconsistent.

In plain English:

the generator emitted code that needed crates the selected feature path did not provide

That is not a Rust compiler problem.

That is not a user application problem.

That is not a case where the user forgot to add a dependency by hand.

It is a generator contract problem.

If the generated code can reference a crate under a supported feature build, the generated manifest has to make that crate available under that same feature build.

Boundary

The boundary here is:

generated shared model requirements versus generated Cargo feature wiring

OpenAPI Generator owns both sides of that boundary.

It owns the generated Rust model code.

It owns the generated Cargo manifest template.

So when shared models can emit pattern validation using lazy_static and regex, the feature path that compiles those models has to carry the matching dependency wiring.

That is the repair surface.

The patch does not redesign Rust server generation.

It does not remove model validation.

It does not flatten every Cargo feature into one broad dependency set.

It tightens the feature wiring around the actual generated-code requirement.

That is the Scarab boundary:

when generated code and generated build metadata disagree, repair the smallest owned seam where the contract breaks

What changed

The PR tightens rust-server Cargo feature wiring so client-only builds include the dependencies needed by generated shared model validation code.

The specific dependency path involved lazy_static and regex, which can be required when generated shared models emit pattern validation logic.

The patch adds a focused regression fixture and test for the feature combination.

It also regenerates the affected Rust server samples so the checked-in generated output reflects the corrected template behavior.

That matters because this is a generator project.

A fix is not complete if only the template changes.

The generated samples also need to show the corrected output, and the regression test needs to prove the feature combination remains buildable.

Why this was not a broad dependency patch

The tempting fix would be to make the generated Rust server manifest include more dependencies everywhere.

That would be easier, but it would be less precise.

The bug was not:

Rust server generation needs every dependency in every mode

The bug was:

a client-only feature build can still compile shared generated model code that requires lazy_static and regex

So the repair stayed near the feature boundary.

That is important because Cargo feature design is part of the generated project contract.

Over-wiring dependencies can hide the bug, but it also makes the generated output less disciplined.

The better repair is to make the feature path honest:

if the client feature can compile shared model validation code, then the client feature must carry the dependencies that validation code requires.

Why the diagnostic result mattered

This case is useful because it is another small patch with a real platform implication.

The failure is not visually dramatic.

There is no UI break.

There is no large subsystem rewrite.

But for anyone relying on generated Rust output, a broken client-only feature build is a real failure.

Generated code is often dropped into CI pipelines, SDK builds, integration tests, and downstream application work. If the generator emits a project that fails under one of its own supported feature combinations, the downstream user loses trust in the generator.

The value of the diagnostic posture was keeping the repair framed around the actual contract:

OpenAPI Generator emits the model code.
OpenAPI Generator emits the Cargo feature wiring.
The selected feature combination must satisfy the code the generator emits.
The repair should live where generated code requirements meet generated build metadata.

That framing kept the patch small.

It avoided a broad Rust server rewrite.

It avoided a user-side workaround.

It repaired the generator-owned seam.

Validation

The patch was validated with both focused and full project checks.

Validation passed for:

targeted regression test
Rust sample check with cargo check --no-default-features --features client
full ./mvnw clean package
full sample generation across 769 generators
./bin/utils/export_docs_generators.sh
git diff --check

At the time of this report, the PR is open, ready for review, and mergeable.

CircleCI nodes are green.

Cubic AI review is green.

For the full validation record and public status, see the Field Lab case record:

https://github.com/scarab-systems/scarab-field-lab/blob/main/field-tests/openapitools-openapi-generator-23920/README.md

Field test result

This was a bounded Rust server feature-wiring repair candidate for OpenAPI Generator.

The issue reduced to:

rust-server client-only builds could compile generated shared models
generated shared models could emit pattern validation code
that validation code could require lazy_static and regex
the client feature path did not reliably provide those dependencies
the generated Rust project could fail under cargo check --no-default-features --features client
the repair tightened Cargo feature wiring for the owned generated-code path
a focused regression fixture/test was added
affected Rust server samples were regenerated
full validation passed

That is the repair lane.

This patch does not claim to redesign Rust server generation.

It does not claim to change OpenAPI validation semantics.

It does not claim that every generated feature should carry every dependency.

It fixes the feature boundary where generated shared model code and generated Cargo metadata stopped agreeing.

Public claim

The correct claim for this field test is:

Scarab/SDS helped drive a bounded repair candidate for OpenAPITools/openapi-generator#23920, where rust-server client-only builds could miss lazy_static and regex even though generated shared model code could emit pattern validation requiring those crates. The upstream PR tightens the Rust server Cargo feature wiring, adds a focused regression fixture/test, and regenerates affected Rust server samples. Validation passed through the targeted regression, cargo check --no-default-features --features client, full Maven package build, full sample generation across 769 generators, generator docs export, and whitespace checks. This does not claim to redesign Rust server generation or Cargo feature policy broadly; it fixes the generated-code/build-metadata boundary where the client feature path failed to carry dependencies required by generated shared model validation.

Scarab Diagnostic Field Test #027 — OpenAPI Generator Boolean Const Enum Boundary

Scarab Systems — Sat, 13 Jun 2026 17:18:37 +0000

Target: OpenAPITools/openapi-generator

Issue: OpenAPITools/openapi-generator#23550

PR: OpenAPITools/openapi-generator#24022

Field Lab: https://github.com/scarab-systems/scarab-field-lab/blob/main/field-tests/openapitools-openapi-generator-23550/README.md

This field test targeted a Kotlin generator bug in OpenAPI Generator where a boolean const: true schema value was rendered into generated Kotlin code as the string "true" instead of the boolean literal true.

The visible issue was simple:

an OpenAPI 3.1 schema defines a boolean constant value
the generated Kotlin model treats the enum value as kotlin.Boolean
the generator emits "true" as a string literal
the resulting Kotlin code does not compile

That is a small surface area, but it is exactly the kind of failure that matters in code generators.

A generator does not get to be “almost right” when it emits source code. If the emitted literal does not match the target language type, the downstream project inherits a broken compile artifact.

The diagnostic question here was not:

How do we special-case this one schema?

The better question was:

Where does OpenAPI Generator already decide how enum values become Kotlin literals, and which primitive boundary is missing?

Field Lab record

The public case record for this field test is available in the Scarab Field Lab:

https://github.com/scarab-systems/scarab-field-lab/blob/main/field-tests/openapitools-openapi-generator-23550/README.md

SDS result

This field test was run in SDS field-test posture against a disposable OpenAPI Generator arena.

The useful diagnostic output was not a broad rewrite recommendation.

It was a boundary read.

The failure lived in a very specific place:

OpenAPI schema truth
generator model metadata
Kotlin enum literal rendering
generated source compilation

That chain left very little room for a broad architectural patch.

The right repair was not to alter OpenAPI 3.1 interpretation globally.

It was not to change Kotlin model typing.

It was not to rewrite enum handling.

It was to correct the Kotlin generator’s literal emission for boolean enum values so kotlin.Boolean values are emitted as Kotlin boolean literals, matching the existing primitive handling pattern already present for types such as Int and Long.

Failure shape

The failure shape was a type/literal mismatch.

The source schema expressed a boolean value.

The generated Kotlin type expected a boolean.

But the emitted enum value was rendered as a string.

That creates uncompilable Kotlin because the generated code tries to assign a string literal where the Kotlin type is kotlin.Boolean.

In plain English:

the generator knew the target type was boolean, but the literal printer still treated the value like text

That is not a schema problem.

That is not a Kotlin language problem.

That is not a user configuration problem.

That is a generator boundary problem.

A code generator has to preserve semantic type truth when it crosses from schema representation into target-language source text.

Boundary

The boundary here is:

schema-level boolean truth versus target-language literal rendering

OpenAPI Generator does not merely copy schema values into files. It translates schema information into compilable source code for a specific language.

That translation layer is where the bug lived.

The boolean value true is valid schema data.

The Kotlin type kotlin.Boolean is valid generated typing.

The invalid part was the stringified source literal "true".

So the repair stayed inside the Kotlin generator’s enum-value rendering path.

It did not reinterpret the OpenAPI schema.

It did not alter the broader model-generation pipeline.

It did not introduce a Kotlin-only workaround outside the existing generator structure.

It added the missing primitive literal handling where the generator already converts values into Kotlin enum output.

That is the Scarab boundary:

preserve the schema truth, but repair only the translation surface that broke it

What changed

The PR updates the Kotlin generator so boolean enum values are emitted as boolean literals when the target enum value type is kotlin.Boolean.

That means the generated Kotlin output uses:

true

instead of:

"true"

The change follows the existing primitive-handling pattern already used for numeric values such as Int and Long.

That is important because this repair does not invent a new enum system. It extends the existing literal-emission logic to cover the missing boolean case.

The patch adds two focused regression checks:

a direct toEnumValue assertion for boolean literal rendering
a generated-output regression fixture using an OpenAPI 3.1 const: true boolean schema

Together, those tests cover both the small rendering function and the actual generated Kotlin output path.

Why this was not a broad generator rewrite

The tempting mistake in generator bugs is to make the patch bigger than the failure.

Because OpenAPI Generator supports many languages, schema forms, and target-specific behaviors, a small bug can appear to invite a large abstraction change.

That would have been the wrong move here.

The reported failure did not prove that enum handling was globally broken.

It did not prove that OpenAPI 3.1 const support needed a broad redesign.

It did not prove that Kotlin model typing was wrong.

It proved one narrower thing:

a boolean value that should be emitted as a Kotlin boolean literal was being emitted as a string literal

So the patch stayed at that level.

That is the discipline of this kind of repair.

When the failure is a missing primitive case, the patch should add the missing primitive case.

Why the diagnostic result mattered

This case is useful because the patch looks small, but the boundary is still important.

In a generator project, output correctness depends on tiny translation decisions.

A single quote mark can be the difference between valid source code and a broken generated project.

SDS helped keep the repair framed around the actual ownership surface:

OpenAPI Generator owns the generated Kotlin source text.
The Kotlin generator owns how schema-derived enum values are rendered into Kotlin literals.
The boolean schema value should remain boolean when emitted into a kotlin.Boolean enum context.

That framing kept the repair from drifting into unrelated schema interpretation, target typing, or broad generator behavior.

The result was a small patch with direct evidence.

Validation

The patch was validated with the targeted Kotlin regression test and full project checks.

Validation passed for:

targeted Kotlin regression test
./mvnw clean package
./bin/generate-samples.sh ./bin/configs/*.yaml
./bin/utils/export_docs_generators.sh
git diff --check

At the time of this report, the PR is open and mergeable.

The Cubic AI reviewer passed.

CircleCI node0 passed, while node1, node2, and node3 were still pending at the time of drafting.

For the full validation record and public status, see the Field Lab case record:

https://github.com/scarab-systems/scarab-field-lab/blob/main/field-tests/openapitools-openapi-generator-23550/README.md

Field test result

This was a bounded Kotlin generator literal-emission repair candidate for OpenAPI Generator.

The issue reduced to:

an OpenAPI 3.1 boolean const: true schema reached the Kotlin generator
the generated enum value type was kotlin.Boolean
the emitted value was incorrectly stringified as "true"
the generated Kotlin was uncompilable
OpenAPI Generator already had primitive literal handling for values such as Int and Long
the Kotlin generator was missing the matching boolean literal path
the PR adds that narrow boolean handling and focused regression coverage

That is the repair lane.

This patch does not claim to redesign OpenAPI 3.1 support.

It does not claim to rewrite enum generation.

It does not claim to alter Kotlin model typing.

It fixes the boundary where boolean schema truth becomes Kotlin source text.

Public claim

The correct claim for this field test is:

Scarab/SDS helped drive a bounded repair candidate for OpenAPITools/openapi-generator#23550, where the Kotlin generator emitted OpenAPI 3.1 boolean const: true enum values as "true" strings even though the generated enum value type was kotlin.Boolean. The upstream PR adds narrow Kotlin generator handling so boolean enum values are emitted as Kotlin boolean literals, matching the existing primitive handling pattern for types such as Int and Long. The repair includes a focused toEnumValue assertion and a generated-output regression fixture. Validation passed through the targeted Kotlin regression test, full Maven package build, sample generation, generator docs export, and whitespace checks. This does not claim to redesign OpenAPI 3.1 support or enum generation broadly; it fixes the boolean literal-emission boundary that produced uncompilable Kotlin.

What Happened When I Told Codex to Calm Down

Scarab Systems — Sat, 13 Jun 2026 15:31:26 +0000

I have been doing a lot of work lately tightening up my diagnostic suite: the mechanics, the workflow, the way it runs against target repos, the way it helps narrow a repair instead of letting everything turn into a fog machine.

And because I work with Codex as my coding agent, I have also become very familiar with a specific kind of AI-agent behavior.

The “I am helping so hard I am about to make this worse” behavior.

If you work with coding agents, you probably know the vibe.

You ask for one thing.

The agent does that thing.

Then it also adjusts a helper.

Then it updates a fixture.

Then it “notices” a nearby pattern.

Then it starts explaining three other improvements you never asked for.

And now you’re staring at the diff like:

“Why are you in that file?”

“I did not tell you to touch that.”

“That was not the repair lane.”

“Please stop being useful for one second.”

I am not proud of how many times I have verbally threatened a language model.

But here we are.

The funny thing is, I am building Scarab partly because I already expect this kind of drift.

I know that when an AI coding agent is given too much uncertainty, it tries to solve the uncertainty itself.

Sometimes that is useful.

Sometimes it is a raccoon with a soldering iron.

The challenge is that while I am developing the diagnostic system, I cannot always use the diagnostic system to supervise itself. So there are moments where I have to manually hold the line.

That means a lot of conversations with Codex that sound like:

“Do not widen the patch.”

“Do not change the diagnostic output to make the diagnostic pass.”

“Do not fix the test by changing what the test means.”

“Do not touch SDS mechanics while repairing the target repo.”

“Stay in the target.”

“Stay in the lane.”

“Why are you like this?”

Very normal. Very calm. Very professional.

Then something changed

At some point, after a lot of tightening, the workflow started to feel different.

Scarab had enough of the diagnostic work under control that I could tell Codex, in plain English:

“You can calm down now.”

Not literally, obviously. Codex does not have nerves. But the workflow had been asking it to carry too much.

Before, the agent was trying to figure out the failure, infer the owning surface, choose the repair, patch the code, update tests, validate the result, and explain the whole thing without leaking anything weird into public output.

That is a lot.

Once the diagnostic suite started doing more of the diagnostic work, Codex had less to invent.

It could just follow the commands, read the result, make the bounded repair, run the checks, and stop.

And weirdly enough, it did start drifting less.

The whole session felt less frantic.

Less “I found a thing and now I will fix six adjacent things.”

More “the suite says this is the lane, so I will work this lane.”

That was the first time I really felt the workflow itself calming the agent down.

The prompt was not the magic

I do not think this happened because I found the perfect prompt.

I think it happened because I stopped asking the prompt to do too much.

There is a difference between:

“Fix this bug.”

and:

“Run the diagnostic. Use the result. Repair only the selected lane. Validate. Stop.”

The first one sounds efficient, but it leaves a huge amount of judgment floating around in the conversation.

The second one gives the agent rails.

And rails matter.

A coding agent with no rails will try to be a detective, architect, repair engineer, QA analyst, cleanup crew, and narrator all at once.

A coding agent with rails can be much more useful.

It does not need to solve the entire repo.

It just needs to do the next bounded thing.

“Please stop helping” is now part of my workflow

The funniest lesson from all this is that sometimes the problem is not that the AI agent is failing.

Sometimes the problem is that it is trying too hard.

It sees a failure and wants to make it go away.

It sees a test and wants it green.

It sees a messy surface and wants to clean it.

It sees a nearby file and thinks, “while I’m here…”

And that is where drift creeps in.

Not as evil robot behavior.

As over-helpfulness.

That is why I now care so much about making the workflow itself stricter.

Not because I dislike AI coding agents. I use them constantly.

But because the agent needs a smaller job than “understand everything and fix the repo.”

When the diagnostic layer carries more of the investigation, the agent can stop sprinting around the codebase with a flashlight in its mouth.

And honestly?

It works better.

Current operating theory

My current theory is simple:

The calmer agent is the bounded agent.

Not calmer emotionally. Calmer operationally.

Less guessing.

Less wandering.

Less “I also fixed this.”

Less “I made a small unrelated improvement.”

Less “the tests pass now, don’t ask too many questions.”

More targeted repair.

More focused validation.

More useful diffs.

So yes, I told my AI coding agent to calm down.

But what I really meant was:

“You do not have to carry the whole diagnostic burden anymore.”

And once that burden moved into the workflow, the agent became easier to work with.

Still weird.

Still occasionally raccoon-coded.

But much better.

Docker Compose Merged the Patch: The Boundary Fired Too Early

Scarab Systems — Fri, 12 Jun 2026 14:13:51 +0000

A Scarab Diagnostic Suite field test just landed upstream in Docker Compose.

PR: https://github.com/docker/compose/pull/13831
Issue: https://github.com/docker/compose/issues/13613
Status: merged

The fix is small in shape, but important in meaning.

It repairs a failure in how Docker Compose handled variable discovery for Compose files that still contained ${...} interpolation expressions inside typed fields, especially in remote stack paths such as oci://... Compose applications.

The specific bug appeared when Compose tried to discover interpolation variables from a Compose model that had not yet been fully resolved.

That model could contain values like:

ports:

host_ip: "${LXKNS_ADDRESS:-127.0.0.1}" published: "${LXKNS_PORT:-5010}"

Those values are valid as unresolved Compose expressions.

But they are not yet valid concrete typed values.

host_ip eventually needs to be an IP address.

published eventually needs to resolve into a port value.

The problem was that validation was running before the interpolation-variable discovery path had finished doing the work needed to make validation meaningful.

So Compose failed early with an error like:

invalid ip address: ${LXKNS_ADDRESS:-127.0.0.1}

The important part is not only that the command failed.

The important part is why it failed.

The hypothesis

The Scarab hypothesis is that many software failures are not best understood as “bad code” first.

They are often boundary failures.

A system has phases.

A system has ownership surfaces.

A system has claims that are allowed to be true in one phase and not yet true in another.

A value may be valid as an unresolved expression before it is valid as a typed runtime value.

A generated artifact may be valid as output but not as source truth.

A test may be valid as regression evidence but not as permission to rewrite behavior.

A config may describe intent before runtime has resolved it into concrete state.

When the wrong boundary enforces at the wrong time, the system can fail even though the pieces involved are each doing something reasonable in isolation.

That is what made this Docker Compose issue diagnostically interesting.

It was not that validation was bad.

Validation is necessary.

It was not that interpolation was bad.

Interpolation is necessary.

The failure was that the validation boundary was being applied during a phase where unresolved interpolation expressions were still supposed to exist.

The boundary was firing too early.

The user-facing shape

The issue came from a real Compose application distributed through an OCI artifact.

That matters because this was not a theoretical parser edge case.

The workflow was:

A Compose application is published.

A consumer deploys it from an oci://... URL.

The Compose file contains interpolation variables so the consumer can customize deployment values.

Compose needs to discover those variables, prompt or resolve them, and then continue.

Instead, typed-field validation saw unresolved ${...} values too early and treated them as invalid concrete values.

So a remote Compose application that should have been customizable failed before the customization path could complete.

That is the kind of failure Scarab is built to care about.

Not just “command errored.”

But:

Which phase owned this value at the moment it failed?

Was ${LXKNS_ADDRESS:-127.0.0.1} supposed to be treated as an IP address yet?

Or was it still supposed to be treated as an unresolved interpolation expression?

That question identifies the repair surface.

The boundary

The boundary was between:

interpolation-variable discovery

and

typed Compose model validation

Those are not the same operation.

Variable discovery needs to inspect unresolved expressions.

Typed validation needs to validate resolved values.

When validation runs too early, it rejects the exact thing discovery is supposed to find.

That is the contradiction.

So the repair was not “turn validation off.”

That would be too broad.

The repair was:

load unresolved models with validation skipped only for variable-discovery paths.

That distinction matters.

Validation remains part of the system.

But variable discovery gets to happen before validation treats templated typed fields as concrete runtime values.

The patch

The merged PR applies loader.WithSkipValidation to the specific Compose loading paths used for interpolation-variable extraction.

It covers:

interpolation-variable extraction from unresolved models
docker compose config --variables
remote-stack interpolation-variable prompting
regression coverage for templated typed port fields such as host_ip and published
a test-only remote loader override so the remote-stack case can be tested deterministically

The patch touched five files:

cmd/compose/options.go
cmd/compose/config.go
cmd/compose/compose.go
cmd/compose/options_test.go
cmd/compose/up_test.go

That is the repair shape I want to keep emphasizing:

Find the boundary.

Provide the context and evidence.

Apply the narrow patch.

Let the win trickle down.

Why this is a good field test

This is a good Scarab field test because the fix is not dramatic.

It does not rewrite Compose loading.

It does not change the meaning of typed validation globally.

It does not broaden remote stack behavior beyond the failing path.

It restores the order of responsibility.

Variable discovery gets to read unresolved interpolation expressions.

Validation gets to validate once values are in the phase where validation has authority.

That is the core of the diagnostic theory.

A lot of software drift comes from systems forgetting which layer owns which truth at which moment.

Sometimes the repair is not to add a new abstraction.

Sometimes the repair is not to make the system more permissive everywhere.

Sometimes the repair is to restore the sequence of authority.

This value is still an expression.

This phase is still discovery.

This validator does not own the value yet.

That is the whole bug.

Why this matters beyond Docker Compose

This is why I keep saying that Scarab is not only about AI-assisted code.

AI makes drift faster and more visible, but software drift is older than AI.

Human-built systems drift too.

Complex systems drift when phases blur, when contracts move, when generated artifacts gain authority they should not have, when tests begin proving the patch instead of the behavior, or when validation starts enforcing a claim before the system has reached the phase where that claim can be true.

Docker Compose is a serious developer tool.

This was a human-built, mature codebase.

The bug was still a boundary failure.

That is the point.

Scarab Diagnostic Suite is a proprietary diagnostic system for software drift. It can be used against AI-assisted development, but the deeper theory is broader: diagnose where a software system stopped preserving the truth another part depended on.

In this case, the truth was simple:

An unresolved interpolation expression is not yet a concrete typed field.

Once that was clear, the repair became narrow.

Evidence before repair

The field-test posture matters.

The goal was not to throw a patch at the wall.

The goal was to identify the boundary and make the smallest change that restored it.

The validation for the PR included:

go test ./cmd/compose
docker buildx bake lint

The regression tests were part of the claim.

They prove that templated typed port fields no longer break variable extraction and that remote-stack prompting can handle the unresolved model path.

That is the difference between a patch and a diagnostic repair.

A patch says: “This seems to work.”

A diagnostic repair says: “This is the boundary that failed, this is the evidence, this is the narrow change, and this is the validation that protects the claim.”

The public record

This PR has now merged into Docker Compose.

That makes it a useful public proof point for the larger Scarab hypothesis.

Not because every field test will merge.

Not because every diagnostic claim becomes an upstream patch.

But because this one shows the shape clearly:

A real issue in a major software platform.

A boundary failure.

A narrow repair.

Regression coverage.

Upstream acceptance.

That is the loop.

Scarab Diagnostic Suite finds evidence.

People make claims.

Maintainers decide.

Field Lab

Scarab Systems maintains a public Field Lab for selected diagnostic field tests.

Some cases may end with a local repair candidate.

Some may become upstream pull requests.

Some may remain diagnostic records only.

That status is part of the record.

If you know of a public open-source issue that looks like cross-layer drift, unclear ownership, phase confusion, AI-assisted codebase confusion, or a boundary failure, you can suggest it as a Field Lab candidate.

Useful suggestions include the public issue link, the suspected boundary, reproduction notes if available, and why the issue may be diagnostically interesting.

The goal is simple:

Make the failure legible.

Find the boundary.

Preserve the evidence.

Repair narrowly.

Let the win trickle down.

AI Code Quality Is Not Repo Truth

Scarab Systems — Fri, 12 Jun 2026 14:11:17 +0000

There is a pattern starting to show up everywhere now.

A team uses an AI coding agent. The agent moves fast. It writes code, rewrites tests, updates docs, creates abstractions, touches config, patches runtime behavior, and explains itself confidently.

Then the team starts noticing the uncomfortable part.

The code is not always “wrong” in the obvious way. It may compile. It may pass tests. It may satisfy a review checklist. It may even look cleaner than what was there before.

But something in the system has shifted.

A test now proves the patch instead of the behavior.

A README now describes an API the repo does not actually expose.

A generated artifact is treated like source truth.

A config file silently becomes the repair surface for a runtime problem.

A fallback path preserves uptime but loses correctness.

A frontend change compensates for a backend contract that should never have moved.

The code looks finished, but the repo no longer tells one coherent story.

That is drift.

And I think the industry is at risk of misunderstanding what kind of problem drift actually is.

The wrong reflex: solve AI drift with more AI

A lot of the current response to AI-assisted code failure is still happening inside the same mental frame that created the problem.

The agent generates code.

Then another AI reviews the code.

Then another AI summarizes the review.

Then another AI writes tests.

Then another AI explains whether the tests passed.

This can be useful. It can catch things. It can improve workflows.

But it does not solve the deeper problem.

If the source of truth is still conversational, probabilistic, and context-fragile, then you have not created a stable diagnostic layer. You have created another layer of interpretation.

That may be better than nothing, but it is not the same as proof.

The industry keeps trying to make the AI agent more self-aware, more careful, more reflective, more heavily prompted, more supervised by other AI systems.

But drift is not primarily a personality flaw in the agent.

Drift is what happens when a system loses track of which boundary owns which truth.

That means the missing layer cannot just be another AI opinion.

It has to be outside the agent loop.

Step outside the conversation

One of the easiest ways to see this is to stop imagining yourself as the developer talking to your AI coding agent.

Instead, imagine you are standing outside the workflow, watching another developer have that conversation.

The developer tells the agent to fix a bug.

The agent changes production code.

Then it changes the tests.

Then it edits the docs.

Then it updates configuration.

Then it says the work is complete.

From inside the conversation, this can feel productive.

From outside the conversation, the obvious process question appears:

Who is checking whether each layer still owns the thing it claims to own?

If a technician from another department made those same changes, a serious engineering team would not simply ask, “Did they finish?”

They would ask:

What did they touch?

Which system boundary did they cross?

Which contract changed?

Which evidence proves the behavior still belongs there?

Which test proves the original claim rather than the new patch?

Which source is authoritative now?

AI-assisted development needs that same operational distance.

Not because AI is bad.

Because AI is fast enough to cross boundaries faster than the team can notice.

AI code quality is not the same as repo truth

There are useful tools emerging around AI code quality: guards, review skills, semantic linters, test checkers, docs checkers, prompt rules, agent policies, and CI add-ons.

Some of them catch real problems.

They can flag hallucinated APIs.

They can notice mock abuse.

They can detect documentation that references missing functions.

They can warn about over-abstraction, broad error swallowing, unsafe patterns, or framework-specific mistakes.

That is valuable.

But catching known output patterns is not the same as proving that the repo’s truth model was preserved.

A guard might say:

“This test mocks too much.”

A diagnostic system has to ask:

“Did the test layer stop validating the behavior that the production system depends on?”

A docs checker might say:

“This function does not exist.”

A diagnostic system has to ask:

“Is the documentation wrong, is the code missing a public API, is generated reference output stale, or did the ownership of this claim move?”

A code-quality tool might say:

“This abstraction is premature.”

A diagnostic system has to ask:

“Did this change move responsibility out of the surface that owns it?”

Those are related questions, but they are not the same question.

The first is output review.

The second is boundary diagnosis.

Scarab’s realignment

Scarab Diagnostic Suite was built around a simple premise:

AI should not be the source of truth for AI-assisted code work.

The repo has to be the source of truth.

The diagnostic layer has to be deterministic, mechanical, evidence-first, and independent of the coding agent’s conversational state.

That is the realignment.

Instead of asking an AI agent to remember every contract, every invariant, every architectural rule, every generated artifact boundary, every test obligation, and every repo-specific convention, Scarab works from the outside.

It inspects evidence.

It compares claims.

It surfaces contradictions.

It identifies boundary failures.

It gives the agent the right context only after the system has established what needs to be preserved.

That matters because an AI coding agent can only work with the context it has. If the context is wrong, stale, incomplete, or conversationally compressed, the agent can produce a very polished wrong answer.

Scarab is not trying to make the agent “smarter” in the abstract.

It is trying to stop the agent from operating without a stable map of repo truth.

The same failure shows up in different lanes

This is why the problem is bigger than one kind of codebase.

AI drift does not only affect open-source projects.

It affects frontend teams, backend services, data systems, DevOps workflows, scientific software, internal tools, agencies, startups, and companies that never thought of themselves as software companies until AI started writing code for them.

The pressure point changes by lane.

The underlying failure is the same.

A boundary stopped preserving the truth another part of the system depended on.

Frontend systems: UI behavior becomes the patch surface

In frontend work, drift often appears when the visible interface starts compensating for a broken contract somewhere else.

A component gets a defensive fallback.

A route adds extra state handling.

A client-side workaround hides an API inconsistency.

A UI test is updated to match the new rendering path.

The screen looks fixed, but the ownership question may be wrong.

Was the frontend supposed to absorb that behavior?

Or did the backend contract, router boundary, state model, accessibility surface, or generated client drift first?

A normal review may ask whether the UI works.

A boundary diagnostic asks whether the UI became responsible for something it does not own.

That distinction matters because once the wrong layer absorbs the repair, future changes inherit the lie.

Backend and API systems: contracts drift quietly

Backend drift often shows up as contract confusion.

A handler returns a slightly different shape.

A serializer changes behavior.

A migration updates data in a way the API layer does not fully encode.

A client library keeps working because it is permissive.

The tests pass because they were updated near the patch.

But the contract may no longer be stable.

In ordinary review, the question is often: “Does this endpoint work?”

The deeper question is:

“Which layer owns this contract, and what evidence proves the contract still matches implementation, documentation, tests, and clients?”

That is where drift hides.

Not always in a crash.

Often in a silent mismatch between what the system claims and what the system now does.

Data systems: freshness, schema, and provenance are not vibes

Data systems are especially vulnerable because the code can be technically correct while the data truth has already moved.

A schema changes.

A cache survives too long.

A migration succeeds but changes meaning.

A model reads from a snapshot that is no longer valid.

A pipeline output is treated as fresh because the job completed.

For data-heavy teams, the problem is not only whether the job ran.

It is whether the job preserved the assumptions the downstream system depends on.

What schema did this result assume?

What version of the source did it read?

Which migration state was active?

Which artifact is authoritative?

Which result is generated, and which result is source truth?

A deterministic diagnostic layer matters here because AI can explain a data pipeline beautifully and still miss the fact that the pipeline is reasoning from stale or misowned evidence.

DevOps and CI/CD: availability is not correctness

Automation failures often look like infrastructure problems.

A deployment succeeds but pulls the wrong image.

A cache hit looks valid but was built under a different assumption.

A fallback keeps the system alive but bypasses the verification path.

A retry prevents downtime but repeats a side effect.

A CI job passes because the failing surface was never exercised.

The industry has spent years building tools around availability, observability, retries, alerts, and recovery.

Those tools matter.

But AI-assisted development adds a different question:

Did the workflow preserve the proof that the result is still correct?

A green pipeline does not automatically mean the repo stayed truthful.

It means the pipeline’s checks passed.

Those are not always the same thing.

Tests: the most dangerous drift can look like validation

Tests are one of the first places AI coding agents can create false confidence.

An agent writes tests.

The tests pass.

The patch looks validated.

But what did the tests prove?

Did they prove the original behavior?

Did they prove the new implementation?

Did they mock away the system boundary?

Did they assert on internals?

Did they update the expectation to match the patch?

Did they delete the failure rather than preserve the regression?

This is why test quality is not just a code smell issue.

It is a truth issue.

A test is not valuable because it exists.

A test is valuable because it preserves a claim the system depends on.

When that claim moves silently, the repo can look safer while becoming less trustworthy.

Documentation: public claims are part of the system

Documentation drift is often treated as cosmetic.

It is not.

Docs are public claims.

A README, API reference, changelog, migration guide, or docstring tells users and future agents what the system is supposed to be.

When documentation references functions that do not exist, examples that cannot run, flags that no longer work, or behaviors that changed without a claim boundary, the repo has lost one of its truth surfaces.

This matters even more with AI agents, because agents read documentation too.

Bad docs do not only mislead humans.

They feed future automation.

That means documentation drift can become agent drift.

And agent drift can write more documentation drift.

That loop is exactly why an independent diagnostic layer matters.

Scientific and applied technical systems: correctness is the product

Not every company following AI-assisted development is a devtools company.

Biotech companies, research labs, analytics teams, agencies, logistics firms, ecommerce platforms, healthcare-adjacent software teams, and internal automation groups all have code.

Many of them are now using AI to write or maintain that code.

For those teams, drift is not an abstract developer concern.

It can mean measurement pipelines become less reproducible.

Reports no longer match source data.

Internal tools encode the wrong assumption.

A generated workflow silently changes how evidence is processed.

The company may not care about “AI code quality” as a category.

But they absolutely care when their software stops preserving the truth their business depends on.

That is the market-level problem.

The repair begins before the patch

The industry often talks about AI-assisted development as if the main question is how to generate better patches.

But the harder question comes before the patch.

What is the repair surface?

Which boundary owns the failure?

What evidence proves the failure belongs there?

What should not be touched?

What tests are allowed to change?

What generated artifacts are outputs, not authority?

What documentation claims must remain aligned?

What context does the agent need before it is allowed to act?

A patch without that map can make the system worse while looking helpful.

That is why Scarab is not a patch bot.

It is not a linter.

It is not a code review personality.

It is not another AI agent watching the first AI agent.

Scarab Diagnostic Suite is a proprietary diagnostic product built around evidence-first repo analysis.

SDS finds evidence.

People make claims.

Maintainers decide.

That boundary is important.

The diagnostic layer should not pretend to be the maintainer.

It should make the system legible enough for the maintainer, developer, or AI coding agent to act without guessing.

Field Lab: public diagnostic case records

Scarab Systems has opened a public Field Lab for selected diagnostic field tests.

Some cases may end with a local repair candidate.

Some may become upstream pull requests.

Some may remain diagnostic records only.

That status is part of the record.

Scarab Diagnostic Suite is proprietary, but the larger conversation is shared. AI-assisted development is changing how all of us work with code, and the goal of the Field Lab is to make boundary failures, drift patterns, and diagnostic reasoning easier to see from more than one angle.

We welcome Field Lab candidate suggestions from developers, maintainers, companies, researchers, and anyone working close enough to code to notice when something has stopped holding together.

If you know of a public open-source issue that looks like cross-layer drift, unclear ownership, AI-assisted codebase confusion, or a boundary failure, you can suggest it as a Field Lab candidate.

Useful suggestions include the public issue link, the suspected boundary, reproduction notes if available, and why the issue may be diagnostically interesting.

scarab-systems / scarab-field-lab

Public case library for Scarab Diagnostic Suite field tests, recording public issues, diagnostic findings, validation summaries, and upstream PR status without publishing private work materials.

Scarab Field Lab

Scarab Field Lab is the public case library for selected Scarab Diagnostic Suite field tests.

Scarab Diagnostic Suite is proprietary and is not currently distributed as a public installable tool. Public materials describe selected diagnostic field tests and software-drift concepts only.

Scarab does not automate repairs or replace maintainers. It identifies evidence-backed diagnostic findings: boundary failures, repo-truth drift, verification gaps, and repair lanes.

Any repair is performed by maintainers, developers, or authorized agents outside the public Field Lab.

This repository publishes public case records only: public issue and pull request links, specific diagnostic findings, validation notes, claim boundaries, and, when applicable, the public status of a human-reviewed patch or upstream pull request. It does not contain SDS source code, internal diagnostic rules, product internals, private run artifacts, or implementation details.

Scarab Diagnostic Suite is a mechanical diagnostic layer. It inspects repository evidence, compares expected and observed behavior…

View on GitHub

The conceptual shift

The AI coding conversation has been centered on the agent.

Better prompts.

Better models.

Better context windows.

Better reviews.

Better tool calls.

Better planning.

Better self-correction.

All of that may help.

But it does not remove the kink in the road.

The kink is that we keep asking AI to be both the worker and the source of truth for the work.

That is the part that has to change.

The repo needs an independent diagnostic layer.

The agent needs bounded context.

The repair needs an owned surface.

The system needs evidence before action.

Once you see it that way, the problem becomes much clearer.

AI did not make software boundaries matter less.

It made them matter more.

Because now the thing crossing those boundaries is faster, more confident, and less naturally constrained by the tacit knowledge a human team used to carry.

The next phase of AI-assisted development will not be won by teams that generate the most code.

It will be won by teams that can still prove what their codebase means after AI has touched it.

That is the real shift.

Not more AI inside the loop.

A stable diagnostic layer outside it.

Scarab Diagnostic Field Test #026 — Next.js Turbopack Denied-Path Watcher Boundary

Scarab Systems — Fri, 12 Jun 2026 01:21:07 +0000

Target: vercel/next.js

Issue: vercel/next.js#81161

PR: vercel/next.js#94735

Field Lab: https://github.com/scarab-systems/scarab-field-lab

This field test targeted a Next.js Turbopack development-server issue where memory and CPU usage could grow heavily during local development.

The reported issue shape was broad:

Turbopack dev server used high RAM and CPU
route compilation caused large CPU spikes
each route compilation appeared to add substantial memory
hard reloads continued increasing memory
the issue appeared in next dev
the reproduction involved a mostly empty app with several routes

That is a big symptom surface.

But the bounded repair did not attempt to solve every Turbopack memory, CPU, cache-growth, or task-eviction concern connected to the issue.

The repair focused on one narrower boundary:

filesystem watcher events from denied generated/cache paths should not keep entering Turbopack invalidation work as if they were source changes.

Field Lab record

The public case record for this field test is available in the Scarab Field Lab:

https://github.com/scarab-systems/scarab-field-lab

The Field Lab is where the public status, issue links, PR links, validation record, changed public files, assistance disclosure, and claim boundaries live.

This article is the semantic field report: what the failure meant, where the ownership boundary was, and why the repair stayed narrow.

Failure shape

The visible failure was resource growth in the Turbopack dev server.

That kind of issue invites broad explanations. It could be memory retention. It could be task eviction. It could be cache growth. It could be route compilation. It could be dependency behavior. It could be a dev-server lifecycle issue.

A broad symptom does not automatically justify a broad repair.

SDS surfaced evidence around cache source-of-truth, cache freshness, shared runtime artifact-cache authority, and the Turbopack filesystem boundary. That pointed the repair toward watcher/invalidation behavior around generated output and cache paths.

The important distinction is this:

Generated output and cache artifacts are not project source truth.

If the project filesystem already treats a generated/cache path as denied, then watcher events from that path should not keep feeding the invalidation pipeline as if they represent meaningful source changes.

That is the smaller boundary inside the bigger performance report.

Boundary

The boundary here is:

project source changes

versus

denied generated output and cache artifacts

Turbopack needs filesystem watchers because source changes should invalidate work.

But not every native filesystem event represents a source change the dev server should care about.

Recursive platform watchers can still receive native events from generated output and cache directories. That can happen even when the project filesystem already knows those paths are denied.

So the repair question becomes:

If a path is already denied by the project filesystem boundary, should watcher events from that path still be queued for invalidation work?

The bounded answer is no.

Denied generated/cache paths should stay outside the source-change invalidation lane.

What changed

The PR updates Turbopack’s filesystem watcher path handling.

The patch adds shared denied-path helper logic in:

turbopack/crates/turbo-tasks-fs/src/denied_paths.rs

That helper checks both relative project paths and absolute system paths against the existing denied-path prefixes.

The patch then uses that denied-path logic in:

turbopack/crates/turbo-tasks-fs/src/lib.rs

and

turbopack/crates/turbo-tasks-fs/src/watcher.rs

The core behavior is simple:

Before watcher events are queued for invalidation work, paths under denied filesystem prefixes are filtered out.

That keeps events from generated output and cache paths from being treated like project source changes.

Rename events matter

One subtle part of the patch is rename handling.

Watcher rename events can come through as RenameMode::Both, where notify provides a source path and a destination path together.

That pair matters.

If filtering happens too early or too bluntly, one denied side of the rename could be removed while the other remains. That can turn a valid source/destination pair into a malformed event.

The repair preserves the source/destination pairing first.

Then it filters each side according to the denied-path boundary.

That means a denied side does not corrupt the structure of the event, and a valid side can still be handled correctly.

This is the kind of small detail that makes the repair boundary more precise than “just drop paths.”

The patch is not merely suppressing events.

It is preserving event semantics while preventing denied paths from causing invalidation work.

Why this was not a full memory-eviction fix

The issue was reported as high RAM and CPU usage.

But this patch does not claim to solve all Turbopack memory growth.

It does not replace memory eviction.

It does not replace task eviction.

It does not solve every future cache-pruning problem.

It does not claim that every route compilation cost in the issue is caused by watcher events.

That is important.

The field-test result is narrower:

Turbopack should not queue invalidation work from filesystem events that come from denied generated/cache paths.

That boundary is meaningful even if other performance work remains.

In other words, the patch does not say:

“This fixes Turbopack memory.”

It says:

“This removes one invalidation path where generated/cache artifacts can continue acting like source-change evidence.”

That is a safer and more reviewable claim.

Why the diagnostic boundary matters

This is exactly the kind of case where a repair can drift.

A dev server uses too much memory and CPU. The reproduction is compelling. The stack is complex. The cache is involved. The filesystem is involved. Watchers are involved. Generated output is involved. Routes are involved.

A broad patch could easily start redesigning eviction, pruning, cache lifecycle, or compilation scheduling.

That may eventually be needed somewhere.

But it was not the smallest repair supported by this field test.

The smaller supported boundary was:

The project filesystem already knows some paths are denied.

Native recursive watchers may still surface events from those paths.

Those events should not enter invalidation work as if they were source changes.

That is the Scarab lane.

Find the surface that owns the boundary.

Repair the handoff there.

Do not claim the entire symptom universe.

Validation

The patch was validated with repo-native Rust/Turbopack checks.

The public Field Lab record includes the validation summary.

The important validation points are:

denied-path helper behavior is tested
the turbo-tasks-fs package test suite passed
formatting passed
clippy passed with warnings denied
diff check passed
public PR checks passed for the recorded checks

For current upstream status and the full public case record, see the Field Lab:

https://github.com/scarab-systems/scarab-field-lab

Field test result

This was a clean Turbopack watcher-boundary repair candidate.

The issue reduced to:

generated output and cache paths are not source truth
the project filesystem already has denied-path boundaries
recursive platform watchers can still receive native events from those denied paths
watcher events from denied paths should not be queued for invalidation work
rename events need to preserve source/destination pairing before filtering
the repair should not claim to solve all memory, CPU, eviction, or cache-growth behavior

That is the repair lane.

The patch restores a boundary between project-source evidence and generated/cache artifact noise.

Public claim

The correct claim for this field test is:

Scarab/SDS helped drive a bounded repair candidate for vercel/next.js#81161, a Turbopack dev-server performance issue involving high RAM and CPU usage. SDS surfaced cache source-of-truth, cache freshness, and shared runtime artifact-cache authority evidence without treating issue text as diagnostic truth. The upstream PR filters denied filesystem paths before watcher events are queued for Turbopack invalidation work, while preserving RenameMode::Both source/destination pairing before filtering. This does not claim to solve every Turbopack RAM, CPU, memory-eviction, task-eviction, or cache-pruning concern connected to the issue.

Public Field Lab record: https://github.com/scarab-systems/scarab-field-lab

Scarab Diagnostic Suite is proprietary. The Field Lab publishes public case records, issue links, validation summaries, and claim boundaries only. If you know of a public open-source issue that looks like a boundary failure or cross-layer drift, you can suggest it through the Field Lab.

Disclosure: This field report was prepared with AI-assisted editing from my own field-test notes, public Field Lab record, upstream issue and PR records, validation summary, and repair record. The technical claims and final wording were reviewed before publication.

Scarab Diagnostic Field Test #025 — VS Code Mouse Echo Runtime Configuration Boundary

Scarab Systems — Thu, 11 Jun 2026 18:07:53 +0000

Target: microsoft/vscode

Issue: microsoft/vscode#247522

PR: microsoft/vscode#320877

Field Lab: https://github.com/scarab-systems/scarab-field-lab

This field test targeted a Visual Studio Code accessibility regression where mouse echo stopped working correctly with screen readers such as NVDA and JAWS.

The visible issue is user-facing and serious:

a screen reader is running
mouse echo is enabled
the user moves the mouse over text in the VS Code editor
the screen reader no longer announces the text under the pointer
the behavior reportedly worked in older VS Code versions

For affected users, this is not a cosmetic regression. Mouse echo is part of how some users inspect and navigate visible text through assistive technology.

But the important diagnostic question was not simply:

How do we fix mouse echo?

The better question was:

Which part of this failure does VS Code actually own?

Field Lab record

The public case record for this field test is available in the Scarab Field Lab:

https://github.com/scarab-systems/scarab-field-lab

SDS result

This was run in full SDS field-test posture against a disposable VS Code arena.

The completed run used product-level diagnose with a full audit profile and full execution mode. It scanned the VS Code target broadly rather than using a bespoke one-off query aimed at the final patch.

That matters because the final repair was not pre-baked into the diagnostic.

SDS did not surface a literal instruction that said:

Add Chromium feature flags.

That is not what happened.

Instead, SDS surfaced the surrounding runtime, configuration, accessibility, and Electron boundary strongly enough to support a bounded repair.

That distinction matters.

A diagnostic system should not have to invent a patch to be useful. Sometimes the correct diagnostic output is the neighborhood: the owned surface where the project can act without pretending to own the entire root cause.

In this case, that owned surface was VS Code’s runtime argument and Electron startup configuration path.

Failure shape

The reported regression sits at the intersection of several layers:

VS Code editor behavior
assistive technology behavior
Windows screen readers such as NVDA and JAWS
Electron
Chromium accessibility-provider behavior
VS Code desktop runtime startup configuration

That is a dangerous shape for a patch.

When a bug crosses that many layers, it is easy to repair the wrong thing.

A patch could try to change editor rendering.

A patch could try to alter screen-reader interaction.

A patch could try to encode assumptions about NVDA or JAWS.

A patch could pretend VS Code owns a Chromium/Electron provider regression that actually sits lower in the stack.

That would be too broad.

The better repair lane was smaller:

VS Code can expose a persistent way to pass Chromium feature flags through its own runtime configuration path.

That does not fix the underlying provider bug.

But it gives maintainers and affected users a stable way to apply Chromium/Electron feature-flag diagnostics or workarounds while the deeper accessibility regression is investigated.

Boundary

The boundary here is:

VS Code-owned runtime configuration

versus

Chromium/Electron accessibility-provider behavior

VS Code does not own every part of the accessibility-provider stack.

It does not own NVDA.

It does not own JAWS.

It does not own Chromium.

It does not own every Electron-level accessibility behavior.

But VS Code does own part of the path by which Electron and Chromium startup switches are configured and allowed through.

That is the repair surface.

The patch stays inside that surface.

It does not change editor rendering.

It does not change screen-reader APIs.

It does not change mouse-hit testing behavior.

It does not assert that one Chromium feature combination is correct for every user.

It adds a persistent runtime feature-flag path through VS Code’s existing argument/configuration machinery.

That is a boundary repair, not a root-cause claim.

What changed

The PR adds support for:

enable-features

and

disable-features

across VS Code’s native runtime argument path.

The patch wires those switches through:

native parsed args
CLI parser options
the argv.json schema
the supported Electron switch allowlist

The values are documented as comma-separated Chromium feature names.

Default behavior is unchanged. These switches are only forwarded when explicitly configured.

That last point is important.

This patch does not impose a feature flag on all users. It does not decide which Chromium accessibility-provider behavior is correct. It creates a supported path for maintainers and affected users to persist runtime flags when they need to test or apply a workaround.

Focused tests cover the parser behavior, including empty and repeated values.

Why this was not an editor rendering fix

The obvious place to look is the editor.

The failure is experienced in the editor. The user moves the mouse over editor text. The screen reader does not announce what it used to announce.

But “where the user feels the bug” and “where the project owns a safe repair” are not always the same place.

That is especially true in desktop applications built on Electron.

A user-facing accessibility regression may pass through VS Code UI, Electron accessibility APIs, Chromium accessibility-provider behavior, and external screen-reader expectations before it becomes visible.

If VS Code does not yet own the root lower-level behavior, a narrow patch should not pretend it does.

The safer repair is to expose a configuration path that allows the lower-level behavior to be tested or worked around without distorting editor code.

That is what this patch does.

It keeps the repair in the runtime startup boundary instead of pushing speculative accessibility logic into the editor surface.

Why the diagnostic result mattered

This case is useful because SDS did not produce a magic answer.

It produced a bounded evidence neighborhood.

That is closer to how real debugging often works.

The value was not:

SDS found the exact patch.

The value was:

SDS narrowed the relevant ownership area enough that the patch could stay small.

For a cross-layer accessibility regression, that matters.

The stack has too many tempting surfaces. A coding agent could easily chase symptoms into editor code, screen-reader assumptions, or Electron internals. A human could do the same.

The diagnostic question kept the repair grounded:

Which surface does VS Code own that can help maintainers and users investigate this regression without claiming to fix the lower-level provider bug?

The answer was the native argument/configuration/Electron switch path.

Validation

The patch was validated with VS Code project checks and a focused parser test.

The focused test confirmed the Chromium feature switches are recognized through the argument parser.

For the full validation record and public status, see the Field Lab case record:

https://github.com/scarab-systems/scarab-field-lab

Field test result

This was a bounded runtime-configuration repair candidate for a VS Code accessibility regression.

The issue reduced to:

mouse echo is broken for affected screen-reader users
the root regression appears to involve Chromium/Electron accessibility-provider behavior
VS Code should not claim to own the entire lower-level provider bug
VS Code does own its runtime startup argument/configuration surface
adding enable-features and disable-features gives maintainers and affected users a persistent feature-flag path
default behavior remains unchanged unless the switches are explicitly configured

That is the repair lane.

This patch does not claim to fix NVDA.

It does not claim to fix JAWS.

It does not claim to fix Electron.

It does not claim to fix Chromium.

It gives VS Code a cleaner owned configuration path while the deeper accessibility-provider regression is investigated.

That is the Scarab boundary:

do not claim the root bug when the evidence supports a narrower owned surface.

Public claim

The correct claim for this field test is:

Scarab/SDS helped drive a bounded repair candidate for microsoft/vscode#247522, an accessibility regression involving mouse echo behavior with screen readers such as NVDA and JAWS. A full SDS field-test diagnose surfaced the runtime/configuration/accessibility/Electron boundary strongly enough to support a narrow VS Code-owned repair path. The upstream PR adds enable-features and disable-features to VS Code’s native argument parser, argv.json schema, and Electron switch allowlist so maintainers and affected users have a persistent Chromium feature-flag path while investigating the lower-level accessibility-provider regression. This does not claim to fix the root NVDA, JAWS, Electron, or Chromium bug.

Scarab Diagnostic Field Test #024 — pnpm CAFS TMPDIR Socket Budget Boundary

Scarab Systems — Thu, 11 Jun 2026 14:50:10 +0000

Target: pnpm/pnpm

Issue: pnpm/pnpm#12222

PR: pnpm/pnpm#12327

Field Lab: https://github.com/scarab-systems/scarab-field-lab

This field test targeted a pnpm install failure where a long pnpm-created TMPDIR path could cause lifecycle tooling to exceed the Unix socket path length limit.

The issue shape is deceptively simple:

pnpm install runs as root, commonly inside a container
pnpm sets TMPDIR inside the pnpm store
a git-hosted dependency runs lifecycle tooling during package preparation
that tooling creates an IPC socket path under TMPDIR
the full socket path becomes too long
Node reports listen EINVAL

That visible error makes the failure look like it belongs to Node, tsx, the lifecycle script, or the package being prepared.

But the bounded pnpm-side repair lives somewhere smaller.

The failure was a path-budget problem.

The public case record for this field test is available in the Scarab Field Lab:

https://github.com/scarab-systems/scarab-field-lab

Failure shape

The reported failure happened during git-hosted dependency preparation.

pnpm created a temporary package directory under its store. That directory became the TMPDIR for lifecycle tooling. A lifecycle tool then created an IPC socket path below that temp directory.

The problem is that Unix domain socket paths have a hard length limit.

So the failure was not caused by the package simply “not building.” It was not caused by a missing dependency. It was not even, in the narrow repair sense, caused by the lifecycle tool creating an IPC socket.

The lifecycle tool was doing something normal: creating a socket under TMPDIR.

The problem was that the TMPDIR path pnpm handed to that tool had already consumed too much of the available path budget.

By the time the lifecycle tool appended its own socket path, the final path crossed the limit.

The visible symptom was:

listen EINVAL

But the diagnostic boundary was:

pnpm-owned temp path length versus lifecycle-owned socket path creation.

Boundary

The boundary here is:

pnpm-owned CAFS temporary package directory naming

versus

lifecycle tooling IPC socket paths created under TMPDIR

pnpm does not own every lifecycle script.

It does not own tsx.

It does not own the Unix socket path limit.

It does not own every file or socket a package tool may create under TMPDIR.

But pnpm does own the temporary directory path it gives those tools to work inside.

That is the repair surface.

The fix is not to special-case one package. It is not to patch around tsx. It is not to redesign lifecycle execution. It is not to claim that pnpm can prevent every possible socket path overflow.

The bounded pnpm-side repair is to stop spending unnecessary path length in the CAFS temp directory basename.

That gives lifecycle tools more room to create their own paths below TMPDIR.

What changed

The repair changes CAFS temp directory creation so pnpm uses a shorter generated basename.

The old path shape used a longer generated temp suffix.

The new path shape uses Node’s native temp directory creation with a short prefix:

ts fs.mkdtemp(path.join(baseTempDir, 'tmp'))

That keeps the temp directory inside the pnpm store’s temp area while reducing the basename length.

The patch also removes the path-temp dependency from @pnpm/store.create-cafs-store.

A regression test was added to prove that CAFS temp directories used during git package preparation keep a short basename.

The important part is not just that the name is shorter.

The important part is that pnpm preserves its ownership boundary:

keep the pnpm-controlled temp root
shorten the pnpm-controlled generated basename
leave more room for lifecycle tools below TMPDIR
avoid pretending to own the downstream lifecycle tool’s socket construction

That is a narrow repair.

Why the full SDS diagnose mattered

This case was opened as a full SDS field test.

SDS did not need a bespoke scoped run to find the right neighborhood.

That matters because the issue had multiple tempting surfaces:

git-hosted dependency preparation
root/container behavior
lifecycle scripts
Node IPC
tsx
pnpm store paths
TMPDIR
Unix socket limits

A human or agent could easily chase the loudest part of the stack trace.

The full diagnostic pass surfaced the runtime/temp-artifact neighborhood instead.

That is the purpose of the diagnostic layer.

Not to magically “know the fix.”

Not to invent a patch.

Not to replace maintainer review.

The useful diagnostic move was narrowing the ownership surface:

which part of this failure is pnpm’s to repair?

The answer was not “everything below the stack trace.”

The answer was the CAFS temp directory naming path.

Why this was not a lifecycle-script fix

It would be easy to frame this as a lifecycle tool failure because the crash happens while the lifecycle tool is creating an IPC socket.

But that would put the repair pressure in the wrong place.

The lifecycle tool needs a socket path. It creates that path under TMPDIR. That behavior may be entirely reasonable.

The Unix socket limit is also not negotiable from pnpm’s side.

So the question becomes:

Can pnpm reduce avoidable path length before lifecycle tools enter the picture?

Yes.

That is why shortening the CAFS temp basename is the right class of repair.

It does not make assumptions about the package. It does not depend on one lifecycle tool. It does not require pnpm to understand every downstream socket path. It simply gives the downstream process more path budget.

That is the semantic repair.

The bug is not “a tool used a socket.”

The bug is “pnpm consumed too much path budget before handing off to tools that reasonably create paths under TMPDIR.”

Validation

The repair was validated with focused and project-level pnpm checks.

The important validation point is that the regression proves the generated CAFS temp basename stays short during git package preparation.

For exact validation details, current upstream status, and the public case record, see the Field Lab:

https://github.com/scarab-systems/scarab-field-lab

Field test result

This was a clean path-budget boundary repair.

The issue reduced to:

pnpm creates a temporary package directory under its store
that directory becomes part of TMPDIR
lifecycle tools create additional paths below TMPDIR
Unix socket paths have a strict length limit
pnpm can shorten the part of the path it owns
shortening that basename leaves more room for lifecycle tools

That is the whole repair lane.

The patch does not claim to fix every possible IPC path failure. It does not claim to change Unix socket limits. It does not claim to fix tsx. It does not redesign pnpm lifecycle execution.

It fixes the pnpm-owned part of the path budget.

That is the kind of boundary Scarab/SDS is designed to surface.

Public claim

The correct claim for this field test is:

Scarab/SDS helped drive a bounded repair candidate for pnpm/pnpm#12222, where long pnpm-created TMPDIR paths could cause lifecycle tooling IPC socket paths to exceed the Unix socket path limit during git-hosted dependency preparation as root. A full SDS diagnose surfaced the runtime/temp-artifact neighborhood without requiring a bespoke scoped run. The upstream PR shortens CAFS temporary package directory names using fs.mkdtemp() with a short tmp prefix, removes path-temp from @pnpm/store.create-cafs-store, and adds regression coverage proving the generated CAFS temp basename stays short.

Scarab Field Lab Is Public: A Case File Repo for Diagnostic Field Tests

Scarab Systems — Thu, 11 Jun 2026 03:47:48 +0000

I opened the public Scarab Field Lab this week:

https://github.com/scarab-systems/scarab-field-lab

This repo is not the Scarab Diagnostic Suite source code. It is not a patch farm. It is not a feed of AI-generated fixes.

It is a public case-file repo for Scarab Diagnostic Suite field tests.

The goal is simple: when Scarab/SDS is used to investigate a real public software issue, the field lab records the public-safe diagnostic trail. That includes the target repo, issue or PR links, the diagnostic finding, the mode of the case, the public status, and any validation summary that can be shared safely.

In other words: if I say Scarab helped narrow a boundary failure, the field lab is where the public record lives.

Why make this a GitHub repo?

Field-test work needs structure.

A DEV.to article is useful for explaining a case. A GitHub issue or PR is useful for upstream review. But neither of those is the right place to preserve the whole diagnostic record across many projects.

The field lab gives the work a stable public home. It lets people see which public issues were examined, which cases stayed diagnostic-only, which cases produced local repair candidates, which cases became upstream PRs, which cases were accepted upstream, which claims are supported by public evidence, and which claims are deliberately limited.

That last part matters.

A case being listed in the field lab does not mean the upstream project endorsed Scarab. It does not mean a patch was accepted. It does not mean Scarab “fixed” the project. The status field says what actually happened.

That keeps the record honest.

What a case file is

A case file is a small public record of a diagnostic field test.

It usually answers a few basic questions:

What project was examined?
What public issue or PR is connected to the case?
What mode is the case in?
What specific boundary or failure shape was identified?
Was a patch prepared?
Was an upstream PR opened?
Was validation run?
What is safe to publicly claim?

That is it.

No giant transcript. No target repo clone. No private local paths. No unpublished maintainer correspondence. No “trust me, the AI said so.”

The field lab is designed to publish enough information to make the work inspectable without turning the repo into a junk drawer of private run artifacts.

Modes and statuses

The repo separates diagnostic modes from public outcomes.

A case can be diagnostic-proof. That means SDS recorded a finding, but no public patch is being claimed.

A case can be repair. That means a local or prepared repair exists, but that does not automatically mean upstream accepted it.

A case can be diagnostic-proof-and-repair. That means the diagnostic finding and a repair candidate are both recorded.

A case can be upstream-pr-recorded. That means a human-reviewed PR or draft PR is publicly linked.

A case can be upstream-accepted. That means an upstream maintainer accepted or merged the public PR.

Those distinctions are boring on purpose. They stop everything from collapsing into a vague “we fixed it” claim.

For developers, that matters because the difference between “I found a boundary,” “I prepared a patch,” “I opened a PR,” and “the project merged it” is not cosmetic. Those are different facts.

The field lab keeps them separate.

The mechanical diagnostics boundary

One of the main reasons I wanted this repo public is to make the Scarab boundary explicit.

Scarab Diagnostic Suite is not an AI coding agent.

The diagnostic suite is mechanical. It inspects repository evidence, compares expected and observed behavior, and records specific findings.

It does not use generative model reasoning to decide what is true. It does not submit unattended patches. It does not treat an AI response as validation.

AI assistance may enter later. For example, AI-assisted tooling may help draft a narrow patch, summarize a diagnostic record, organize validation notes, or prepare a maintainer-facing explanation.

But that happens after the diagnostic evidence exists.

The separation looks like this:

text SDS finds evidence. A human reviews and owns the claim. AI may assist with implementation or writing. Maintainers decide what belongs in their project.

That is the operating model.

Why this matters in practice

A lot of debugging starts from a symptom.

A command fails outside a project directory. A compiler path accepts a type it should reject. A response API hangs instead of settling. A test passes but the behavior is still wrong.

The field-test approach tries not to stop at the symptom.

The useful question is usually:

text Which boundary stopped preserving the behavior another part of the system depended on?

That question is where the case file starts. The repair, if there is one, should come after the boundary is understood.

That is why I care about the diagnostic record.

Without the record, a patch can look like a fix while still being hard to review. With the record, a reviewer can see the intended repair lane:

This is the failure shape.
This is the boundary.
This is what should stay unchanged.
This is the narrow behavior being restored.
This is what the test proves.

That is the kind of contribution I want Scarab field tests to produce.

What the repo does not contain

The field lab deliberately does not contain everything.

It does not contain SDS product internals, cloned upstream repos, target worktrees, secrets, local paths, private prompts, private maintainer correspondence, or raw AI transcripts.

That is not because those things are unimportant. It is because a public evidence repo should stay public-safe.

The field lab is for case records and public links. The target project remains the authority over its own source code. The upstream issue or PR remains the authority over upstream review.

The field lab records what Scarab investigated and what claim is safe to make.

Why not just publish field reports?

The field reports are useful, but they are narrative. They explain the bug, the boundary, and the repair in plain English.

The field lab is more structured. It is closer to an evidence index.

A DEV.to field report can say:

text Here is what happened.

The field lab can say:

text Here is the case record. Here is the public issue. Here is the public PR if one exists. Here is the status. Here is the validation summary. Here is the claim boundary.

Those two things work together.

The article is the story. The repo is the record.

What I want developers to see

I do not expect every developer to care about Scarab as a product yet. That is fine.

What I want developers to be able to inspect is the method.

Does the case distinguish symptom from boundary? Does the repair claim stay narrow? Does the status match the public upstream reality? Does the case avoid implying maintainer endorsement where none exists? Does the diagnostic record make the patch easier to reason about? Does the process reduce noise instead of adding more?

That is the bar I am trying to hold.

Open-source maintainers do not need more confident noise. They need clear, reviewable, bounded contributions.

The public promise

The shortest version of the field lab is in the Scarab Boundary Contract:

text SDS finds evidence. People make claims. Maintainers decide.

That is the whole posture.

Scarab does not replace maintainers. It does not override upstream ownership. It does not claim that every local repair belongs upstream. It does not pretend AI confidence is proof.

It records diagnostic evidence so human-owned repair work can start from a clearer boundary.

That is why the field lab exists.

It is a public diagnostic record.

And now it is open.