DEV Community: Kwansub Yun

No Single Key Opens the Boundary: An Offline Dual-Control Gate for Sensitive Artifact Export

Kwansub Yun — Thu, 02 Jul 2026 10:35:33 +0000

In short: PYRHELIX is an offline dual-control release gate for sensitive BIO/PII artifacts. It does not replace perimeter security, compliance review, or clinical judgment. It closes one specific gap: no single actor, key, or approval should be able to move a high-consequence artifact across the release boundary alone.

By BIO/PII export, we mean high-consequence data bundles such as genomic files, clinical exports, research packets, identity-linked health records, and derived AI-ready datasets leaving a controlled environment.

The gap between what BIO/PII data is worth and how it's protected

The BIO and health-data economy is enormous, and it's getting more automated every year as AI pipelines touch more of it. The security incidents in this space are not rare, and they are not small.

23andMe (2023): attackers used credential stuffing, feeding passwords stolen from unrelated breaches into 23andMe's login page until roughly 14,000 accounts matched. Because of the platform's DNA-relative matching feature, that cascaded into exposure of genetic and ancestry data tied to an estimated 6.9 million users, including a disproportionate concentration of Ashkenazi Jewish and ethnically Chinese users who were then targeted with hateful, ancestry-specific harassment. (HIPAA Journal, Wikipedia)
Change Healthcare / UnitedHealth (2024): a single server without multi-factor authentication was the entry point for a ransomware group. Change Healthcare clears roughly 1 in 3 U.S. patient records. The final confirmed count was 192.7 million people affected, the largest healthcare breach ever reported to HHS, disrupting billing and payment processing across thousands of hospitals and clinics nationwide. Response cost: $3.1 billion in 2024 alone. (BleepingComputer, Cybersecurity Dive)
Vastaamo (Finland, 2020): a psychotherapy clinic stored ~36,000 patients' therapy notes unencrypted, on a system whose root account had no password. When extortion of the company failed, the attacker emailed roughly 30,000 patients directly, demanding ransom to avoid publishing their session notes. The clinic went bankrupt within months. (Wikipedia, CNN)

None of these three were release-boundary failures. They were perimeter and infrastructure failures: credential stuffing, a missing MFA setting, unencrypted storage with no root password. A signing protocol does not fix a server with no root password, and we are not going to claim otherwise.

What they show, together, is a pattern. BIO and health data carries some of the highest-consequence blast radius of any data category, and the organizations holding it are frequently operating below the security maturity that scale would justify. Compliance frameworks like HIPAA and GDPR mandate access logging and encryption at rest.

They say much less about what happens at the moment a sensitive artifact is about to leave a controlled environment. In practice, that moment is still often gated by one signature, one approval click, one export button.

That narrower gap, not "BIO security" broadly but specifically the release and export decision, is what we're building PYRHELIX to close. It is not a fix for this industry. It is one piece of it, and we want to find out if it actually holds up.

Why the release step specifically stays weak

A few structural reasons this keeps happening, independent of any one company's competence:

Compliance is not the same discipline as security engineering. Passing a HIPAA audit checks access logs and encryption settings. It rarely asks whether one compromised or coerced person can push sensitive data out the door alone.
Security engineering talent concentrates in tech and finance, not biotech. Research organizations hire for domain science, not for building dual-control release infrastructure in-house.
Consolidation creates concentration risk. Change Healthcare touching a third of U.S. patient records through one platform is the extreme case. The same pattern, one vendor, one pipeline, one signer, repeats at smaller scale everywhere.
Release is usually the last, least-engineered step in an otherwise carefully built research or clinical pipeline. By the time data is ready to ship, the hard technical work already happened, and the actual export is often just an email or a shared drive.

What PYRHELIX is

PYRHELIX enforces that a sensitive bundle cannot be released by one actor acting alone. The workflow has two independent steps.

Producer (Strand A): the party that generated the artifact seals its content hash and attests to what was produced. This alone is not enough to release anything.
Authority (Strand B): an independent reviewer, using a different signing key and a different role, attests approval. That attestation is cryptographically bound to the exact same bundle closure Strand A produced.

The release boundary only opens when both strands are present, use distinct keys, and bind to the same closure. Every step is recorded in a signed, hash-chained governance journal, so a separate offline verifier, potentially running on a machine that never touched the original data, can independently replay the whole decision trail: what was produced, who approved it, with which keys, in what order.

Why "Strand A / Strand B", and where the metaphor stops

Worth getting right rather than sounding clever: a real DNA double helix does not have strands named "A" and "B." The correct terms are sense strand (coding strand) and antisense strand (template strand), and neither one alone is the functional, stable molecule; the helix only exists once both are bound together.

"Strand A" (producer) and "Strand B" (authority) are protocol role labels that borrow that single idea, not a claim to real molecular-biology nomenclature.

PYRHELIX's separate, internal bit-level integrity subsystem does use the correct sense/antisense terms for an unrelated purpose (parity diagnostics on hashed symbol streams). The governance mechanism described here is plain Ed25519 dual-signature binding over a shared closure hash.

Two pieces of the actual code

Real source, not simplified for effect.

Role separation can't be satisfied by accident. In strict mode, the verifier checks that Strand A and Strand B didn't reuse the same signing key, the same role, or the same underlying public key. Any one of those three silently fails the whole bundle:

# pyrhelix/governance/verifier_enforcement.py
if strand_a_key_id and strand_a_key_id == strand_b_key_id:
    errors.append(error("KEY_ID_REUSE", "authenticity",
        "Strand A and Strand B reused the same key_id in strict mode."))
    attestation.strand_b_authenticity_verified = False
    attestation.cross_strand_binding_verified = False

if strand_a_role and strand_a_role == strand_b_role:
    errors.append(error("ROLE_REUSE", "authenticity",
        "Strand A and Strand B reused the same role in strict mode."))
    attestation.strand_b_authenticity_verified = False
    attestation.cross_strand_binding_verified = False

There's no "trust me, these are different people" flag anywhere in this path. If the same key produced both attestations, the bundle fails.

Every journal entry is chained into the one before it, not just hashed on its own. The entry hash is computed over prev_entry_hash + canonical_bytes, so tampering with entry N breaks the hash of every entry after it, not just the one you touched:

# pyrhelix/governance/chronicle_io.py
canonical_bytes = canonical_chronicle_bytes(payload)
payload["entry_hash"] = sha256_hex_bytes(
    (prev_entry_hash + canonical_bytes.decode("utf-8")).encode("utf-8")
)
payload["event_signature_b64"] = base64.b64encode(
    signing_key.sign(canonical_bytes).signature
).decode("ascii")

Splice out or edit one line of the journal and every subsequent entry fails replay. That is exactly what the adversarial CI gate checks for (journal_tail_truncation, manifest_closure_tamper, and several others) on every commit.

Built on standards, not custom cryptography

Layer	Standard
Content hashing	SHA-256 (FIPS 180-4)
Canonical JSON for hashing/signing	RFC 8785 (JCS)
Signatures	Ed25519 (RFC 8032, FIPS 186-5)

For supply-chain security readers: PYRHELIX is not yet an in-toto or SLSA-compatible attestation format. That compatibility layer is the next evaluated integration path, not a claim we are making today.

What it actually rejects

An adversarial CI gate exercises these cases on every change, not just the happy path:

Two strands signed with the same key or role
A revoked or retired signing key reused for a new decision
Journal tampering, sequence gaps, or tail truncation
Replay of a superseded governance session
Authority key rotation without a signed rotation event
A bundle declaring "no raw payload" while raw sensitive content is actually present in the bundle directory

Running it offline

python tools/governance_ci_gate.py

It bootstraps local producer/authority keys, emits a provisional bundle (Strand A), finalizes it (Strand B), verifies the finalized bundle, and independently verifies the signed journal, all without a network call. It exits non-zero on any failure, which is also how it runs in CI on every commit.

The repository is currently private while we validate this against a real BIO/PII workflow before opening it more broadly.

What this is not

Not a runtime AI firewall. Tools like LlamaFirewall or NeMo Guardrails intercept model calls in real time and inspect prompts or outputs as they happen. PYRHELIX operates downstream of that. It gates the moment content leaves the organization, after inference is already done.
Not a fix for the kind of breaches described above. Those were perimeter and credential failures, not release-gate failures.
Not a compliance scoring product. Tools that score content against frameworks like GDPR or HIPAA evaluate what's inside a bundle. PYRHELIX only verifies cryptographically that two independent signatures exist. It doesn't replace that evaluation. It gates release after it.
Not yet interoperable with in-toto/SLSA attestation formats. We're sequencing that compatibility layer after real-world validation, not blocked on it technically.
Not externally audited yet. Internal verification today means an automated test suite plus the adversarial CI gate above.

Where we need input

If BIO/PII export is part of your work, three questions:

Is release approval today genuinely single-approver, or do you already run a dual-control process?
Does verification need to work fully offline, or is a networked service acceptable?
What would have to be true for you to say "this doesn't solve my problem"?

If any of these land for you, we'd like to run a real scenario together: info@flamehaven.space

Everyone Was Talking About AI Agents. We Were Asking Who Was Responsible.

Kwansub Yun — Wed, 24 Jun 2026 15:29:34 +0000

The Question Nobody's Asking

The advice is consistent enough to count as consensus:

Configure your AI like you'd configure a skilled teammate.

Set up structured context, encode your preferences, close feedback loops, delegate progressively. Eugene Yan's recent piece on working with AI is the best formulation of this I've read. Worth reading in full. If you're short on time:

Context as infrastructure — Organize your workspace so the model can navigate it. Treat each session like onboarding a new hire: CLAUDE.md, INDEX.md, suggested reading order.
Taste as configuration — Encode how you want the model to behave, push back, and teach. Frequent workflows become skill files that load on demand.
Verification for autonomy — Shift verification left. Let the model run evals, inspect browser output, read its own errors. You can't delegate what you can't verify.
Scaling via delegation — Move from line-by-line instructions to end-to-end specs. Parallel sessions, git worktrees, progressively larger task chunks.
Closing the loop — Mine session transcripts to update CLAUDE.md and skills. Make corrections inside the session so the transcript captures the before-and-after.

But there's a question the framework doesn't address.
It's not a criticism. It's a scope boundary, and the boundary matters more as the stakes get higher.

What happens when the session ends? Not just this session.
What happens when the operator who built all that context leaves the project,
When a new engineer opens the codebase cold and the AI boots with no history of what came before
When the rules that worked so well were personal preferences that lived in one operator's workflow and not documented failures encoded into the project itself?

Without a structure designed to survive operator changes, the answer is: the model knows what it can read from the current files. Every incident that shaped the previous operator's judgment resets. Every hard-won rule that came from a real failure is invisible to the new session.

That's the gap MICA(MICA (Memory Invocation & Context Archive) fills. But it's important to be precise about what kind of gap it is.

Not a Scale Problem — a Threat Model Problem

Eugene explicitly notes in his piece that his principles extend beyond individual use: team norms, agent harness design, organizational infrastructure. A well-committed CLAUDE.md at repo level can outlive any single operator. He makes this point himself.

So the distinction isn't scale, and it isn't storage. It's origin.

Eugene’s configuration encodes preferences, which represent the accumulated judgments of a skilled operator refined across sessions.

MICA's Design Invariants (DIs) encode incidents instead.

Each DI is a binding rule the model must operate within, serving as a constraint extracted from a specific failure rather than a preference. Every critical DI requires binding.origin_episode.

The rule exists not because someone prefers a certain behavior, but because a specific failure occurred, was documented, and was encoded as a constraint. An operator who has never touched the project can read a DI and understand not just what the rule is, but why it exists and what the consequences were when it wasn't followed.

That information can't come from preference accumulation. It comes from incident history.

Eugene's framework defends against an operator who hasn't thought carefully about their setup. MICA's framework defends against an operator — or a model — that doesn't know what this project has already paid to learn.

Why We Don't Use Agents

That threat model difference leads directly to the agent question, and to the choice that most teams in this space make differently from us.

Eugene recommends parallel agents, git worktrees, and progressive delegation to larger work chunks. We don't. The reason isn't technical immaturity, though the empirical record is worth noting: Gartner projects 40% of agentic AI projects will be cancelled by late 2027; a 2025 enterprise survey found only 14% of agent pilots have successfully scaled; the FSE 2025 "Agentless" paper showed that a simple three-step pipeline competed with complex agent orchestration at a fraction of the cost. But none of that is the reason.

The reason is a single question:

When an agent gets something wrong, who is responsible?

In compliance-sensitive domains, such as financial signal generation, citable scientific archives, and legal audit trails, that question must have a person as its answer.

Here is what MICA's Package Conformance Tests (PCT) produce in a typical deployment, running eleven deterministic checks at session start before the model acts:

PCT-001 [PASS] mica.yaml found
PCT-006 [WARN] mica_spec 0.2.6 is 2 version(s) behind canonical 0.2.8 -- consider upgrading
PCT-010 [PASS] all 6 critical DIs have binding
PCT-010 [WARN] doctrinal binding (no episode code, version ref, or date): ['DI-001', 'DI-002', 'DI-004']
              -- ground origin_episode in a real incident
PCT-009 [PASS] CLOSED CONTRACT

Deterministic. Inspectable.

A person sees this output and decides to proceed. The model activates only after that decision. However, if you add an orchestrator agent before PCT, three new failure modes appear:

First, the agent might see a WARN alongside a CLOSED CONTRACT and continue anyway.
Second, the agent might judge the PCT-010 WARN as purely informational and deprioritize it.
Third, PCT might not run at all because the agent skips it.

Governance that is probabilistic is not governance. It's a suggestion. The moment you introduce a stochastic layer before the verification check, the check becomes contingent on the model's disposition — which shifts by session, by context, by what else is in the window.

Eugene's framework contains a related principle: verification must precede delegation, and effective delegation requires defining success criteria so you can verify the outcome. His implementation is configuration.

MICA's implementation takes a different cut: don't delegate the verification itself. The human runs PCT. The human reads the result. The model then operates within the confirmed state.

Agent + MICA is technically possible. But routing PCT through an agent changes what MICA is.

Instead of a gate running before the model acts, it becomes a document that the agent reads if it decides to. Think of the difference between a hard-failing linter that blocks a CI build and a README that documents the same rule: one stops the problem from shipping, the other gets skipped when someone is in a hurry.

A gate is structural. A document is advisory. Our domains require a gate. Here's what that gate looks like in practice.

MICA Mapped to Eugene's Principles

The mapping is worth making explicit, because MICA isn't an alternative to Eugene's framework. It's an implementation of it, extended from individual to institutional context:

# mica.yaml

# ── Eugene: Context as infrastructure ─────────────────────────────────
# mica.yaml declares what loads, in what order, before anything runs.
name: alecta-stock
mica_spec: "0.2.8"
mode: memory_injection

# ── Eugene: Taste as configuration ────────────────────────────────────
# Not personal taste — incident-grounded invariants.
# Each DI traces to a real event in binding.origin_episode.
di_policy:
  namespace_mode: sequential
  critical_binding_required: true   # PCT-010 escalates to FAIL if unbound
  max_archive_age_days: 180         # PCT-012: WARN when archive goes stale

layers:
  - name: archive
    path: alecta-stock.mica.archive.json
  - name: playbook
    path: MEMORY_PLAYBOOK.md

# ── Eugene: Verification for autonomy ─────────────────────────────────
# Verification happens here, before delegation begins.
# PCT runs deterministically. Human reads output. Model activates after.

# ── Eugene: Closing the loop ───────────────────────────────────────────
# binding.origin_episode on every DI is the closed loop — into a
# traceable incident record, not a corrected preference file.

# ── Eugene: Scaling via delegation ────────────────────────────────────
# COMPACT_MODE: no mica.yaml at all. Archive + playbook carry
# governance directly. Delegation scales to minimal footprint.

Four of the five principles map cleanly. The fifth — Closing the loop — is where the most interesting failure appeared. In v0.2.7, the loop looked closed. Running PCT against production deployments showed it wasn't.

v0.2.8: The Loop Has to Actually Close

Two critical DIs, same project, both returned PCT-010 [PASS] under v0.2.7.

{
  "id": "DI-001",
  "label": "astock-data-integrity",
  "severity": "critical",
  "binding": {
    "origin_episode": "Enforcement of absolute data integrity to prevent financial risk.",
    "violation_count": 0
  }
}

{
  "id": "DI-006",
  "label": "astock-output-schema-completeness",
  "severity": "critical",
  "binding": {
    "origin_episode": "EXP-OS-1 (v0.8.6): outputSchema used Zod .strip() — unknown fields silently dropped before scoring. valuation.per and valuation.pbr lost in three separate live runs before detection.",
    "violation_count": 3,
    "last_triggered": "2026-04-02"
  }
}

DI-001's origin_episode restates the label. DI-006's origin_episode is a record: a version, a named experiment, a specific failure, a count of how many times it recurred before anyone caught it.

A model loading DI-001 learns that data integrity matters. A model loading DI-006 learns that this specific failure happened three times before it was found, and here is exactly what caused it.

v0.2.8 now distinguishes between them:

PCT-010 [PASS] all 6 critical DIs have binding
PCT-010 [WARN] doctrinal binding (no episode code, version ref, or date): ['DI-001', 'DI-002', 'DI-004']
              -- ground origin_episode in a real incident

CLOSED CONTRACT holds. But three of the rules are declarations, not lessons. The loop isn't closed yet. This is more than just a v0.2.8 feature, as it represents the threat model playing out at the data level.

The incident-grounded binding is what survives operator changes. The doctrinal binding is what disappears when the person who understood the intent stops being the person running the sessions. Three deployments show where this has mattered.

Three Deployments

Different domains, different failure modes. The first two are about what the gate stopped. The third is about what governance looks like when you deliberately minimize it — and what it means that the institutional memory persisted anyway.

1. Alecta-Stock (securities)

The Zod failure that produced DI-006 was invisible in the moment and obvious in retrospect. The pipeline used Zod to validate its output schema, and Zod's default .strip() behavior silently drops unknown fields by design, for security reasons. When valuation.per and valuation.pbr were added to the archive definition but not to the Zod schema, the model produced complete scoring objects, the schema stripped the valuation fields before they reached the scoring stage, and the output looked valid. No error raised. Three live scoring runs were affected before anyone noticed two fields were consistently absent.

This is precisely where preference-based configuration falls short. A CLAUDE.md entry saying "always verify Zod schema completeness" is a reminder. A new operator won't know why it's there. A distracted session will skip it.

In contrast, DI-006 is a record showing that this specific failure happened three times at a specific version before it was caught. It loads structurally at session start, meaning there is no session where it is optional.

DI-002 covers the complementary failure: any execution path reaching NO_DECISION must fail closed, rather than returning a neutral placeholder that downstream systems treat as a valid result. Through this framework, the model begins each session knowing the failure modes of this specific project. Not as warnings, but as history.

2. Flamehaven Audit Reports (biomedical)

Covered in the previous article in this series: 56 EQA records (physics/math reproductions), 34 BAV experiments (protein-folding validation). The archive is citable. Labels become downstream citations.

The failure was EQA framing drift. Across 51 records, a "PASS" label had been applied to results that were numerically correct but had not been verified through actual engine threshold evaluation — only through manual review. Seven records had gone through the real process. Forty-four had not. No one had noticed because the label looked the same in both cases.

DI-EQA-001 now encodes the precision lock: PASS badge only from real threshold evaluation, not from manual review or narrative description. Domain-namespaced DI IDs, formalized in v0.2.7, keep EQA-specific rules identifiable separately from BAV-specific ones, which have their own failure history.

The gate doesn't verify that the science is correct. It verifies that the process that produced the label matches what the DI requires. That distinction matters because the failures that poison archives are usually not fabrications. They're process mislabelings. Data checked the wrong way, labeled as checked the right way.

3. Flamehaven Code Audit Standard (site operations)

CAS is a confidentiality audit system, and its MICA deployment has no mica.yaml. This is intentional. COMPACT_MODE — formalized in v0.2.7 — is a deliberate minimum-footprint deployment. PCT-001 fails (mica.yaml not found), and the output correctly identifies the package as LEGACY. Not defective. Not non-compliant. Operating at minimum footprint by decision, not by accident.

The distinction between COMPACT_MODE and pre-migration LEGACY_MODE matters in practice: both produce pct=LEGACY at runtime, but one is a decision and one is a migration target. Before v0.2.7, there was no way for the system to tell them apart.

What this case demonstrates: even without the full conformance stack, the incident history persists. The archive and playbook still carry the record of what happened and why the rules exist. Simplify the gate, and the institutional memory doesn't go with it.

What MICA Cannot Block — and Why That Matters More Than What It Can

An honest accounting of limits is part of what makes a gate trustworthy. A gate that claims to catch everything is a gate you should trust less.

Plausible fabrications within valid ranges pass every structural check. A protein pTM score of 0.74 when the actual value is 0.61 has valid format; the number is wrong. No pattern check reaches this. It requires someone who can re-run the underlying computation and compare.

Structural compliance with false content is subtler. v0.2.8's doctrinal WARN fires when origin_episode contains no episode code, version reference, or date. But EXP: general integrity principle passes the check — it looks like an episode code without pointing to one. The validator detects naming conventions. It cannot verify that the narrative behind a code is accurate. A binding can look grounded without being grounded.

Mid-session governance drift is the hardest to close. PCT runs at session start. A model that loads the DIs, acknowledges the constraints, and violates one in step 8 of a 12-step task is not caught by PCT.

The gate is at the session boundary. What happens within the session still requires oversight from someone who understands what the model is doing.

Correct computation with wrong interpretation sits entirely outside structural checking. "This confirms X" versus "this is consistent with X" cannot be separated by a YAML validator. The data may be real, the structure valid, and the conclusion wrong. Catching this requires domain expertise.

Understanding these four categories is what keeps the gate honest. It is a filter for failures that are cheap to prevent structurally, so that human attention is available for the failures that aren't.

Eugene's framework closes a loop by encoding what the operator has learned. MICA closes a loop by recording what the project has paid for.

Both loops matter. That discipline is what the gate enforces. The gate ensures the operator has read the record before the model begins. What happens after that is still on the operator.

Beyond M15: Why STEM BIO-AI Started Acting More Like a Governance Report in v1.8.x

Kwansub Yun — Fri, 12 Jun 2026 10:21:34 +0000

Not just a new framework, but a clearer answer to what the score means, why the report exists, and how the artifact should be read.

The real change in v1.8.0 through v1.8.4 was not that STEM BIO-AI cited one more framework.

The real change was that it became harder to misread the report.

M15 mattered. It strengthened the regulatory-traceability vocabulary. But the deeper shift was broader:

the tool got stricter about what it was willing to imply from local repository evidence, and the report got more explicit about why each surface exists at all.

That changed the project in three ways:

it stopped behaving like a score sheet that developers happened to inspect
it integrated M15 as a bounded post-hoc traceability layer rather than a hidden score driver
it treated release memory, packaging, and public report surfaces as part of release integrity rather than mere maintenance hygiene

This is the real post-M15 story.

Part 1. Perception: Why STEM BIO-AI Should Not Be Read as a Simple Score Tool

The hardest reporting problem in the v1.8.x line was no longer only how to show something or even what to show.

It was why to show it at all.

That distinction matters because the same report is read by different people for different reasons:

a prospective user wants to know whether the repository is trustworthy enough to try
a maintainer wants to know what is holding the score down and what to fix first
a reviewer or auditor wants to know which claims are supported, which are overstated, and which remain outside scope

If those audiences all receive the same fields without a visible purpose hierarchy, the result is machine-legible but human-misleading.

That is why the recent report changes should be understood as user-friendliness in a governance sense, not as design polish.

The project had to become better at stopping readers from confusing:

a deterministic score with a safety verdict
a traceability mapping with compliance proof
a code-integrity PASS with overall repository maturity
a compact report surface with complete evidence

That realization changed the output layer itself.

Recent report work added or strengthened several surfaces specifically to solve that perception problem:

a fixed score-boundary note near the score itself
explicit Tier Lock and Classification Applied surfaces so score constraints are not hidden
stronger Governance Posture, What Is Actually Present, and What Is Missing Or Contradicted summaries
Regulatory Traceability placed ahead of the MIT AI Risk Repository (AIRI), used here as a secondary risk-vocabulary layer, so the reader sees repository-to-framework mapping before the broader risk language
clearer chapter hierarchy in the detailed PDF so the report reads like a governance document instead of a detector dump

In concrete terms, that changed the reader's path through the artifact.

Instead of landing first on a score and then digging through detector output, the current report leads with:

Governance Posture
About This Score
What Is Actually Present
What Is Missing Or Contradicted
Regulatory Traceability
AIRI Risk Triggers

Only after that does it move into Decision Path, Top Remediation Actions, Code Integrity details, and Evidence detail.

The key lesson was simple:

a report becomes more useful not when it shows more fields, but when the reason those fields exist becomes legible to the reader.

This is also why the score disclaimer mattered so much:

Score reflects calculation integrity, not calibrated validity. Triage signal only.

That sentence is not ornamental. It forces the system to tell the truth about itself.

What is verified:

calculation integrity
deterministic reproducibility
transparent score assembly

What is not verified:

calibrated measurement validity
runtime behavior correctness
clinical safety
compliance or regulatory clearance

This is the most important perception shift in the v1.8.x line.

The project is no longer trying only to answer, “What score did this repository get?”

It is trying to answer something more useful:

Is bio-governance actually present?
Is it adequate relative to the repository’s claims?
What is verified, what is inferred, and what is still missing?

Figure 1. The report now places governance posture, score-boundary language, and top-level trust signals near the score surface instead of hiding them behind lower-level detector output.

Part 2. What M15 Is, Why It Matters, and How STEM BIO-AI Uses It

M15 refers to ICH M15: General Principles for Model-Informed Drug Development.

The official FDA guidance page is here:

FDA: M15 General Principles for Model-Informed Drug Development

As the FDA describes it, the June 2026 final guidance was prepared under the auspices of the International Council for Harmonisation and provides general recommendations for:

planning model-informed drug development evidence
model evaluation
documentation
regulatory interactions
reporting and submission

It also establishes a harmonized assessment framework and terminology for MIDD evidence. That matters because it gives a cleaner language for talking about traceability, documentation quality, and context of use.

But the important thing in STEM BIO-AI is not merely that M15 appears in the output.

The important thing is how it appears.

STEM BIO-AI does not use M15 as a covert score driver. It does not inflate the formal score because an M15 citation exists. It uses M15 as a post-hoc regulatory traceability layer attached to already-detected repository evidence.

That boundary matters.

Without it, a framework citation can easily become a kind of rhetorical overclaim:

the report looks more regulatory than it really is
the reader assumes framework mention implies compliance maturity
traceability begins to masquerade as proof

The post-M15 line was careful to avoid that mistake.

In practice, the project used M15 in a bounded way:

as part of measurement_basis and regulatory framing
as a traceability surface that helps interpret repository evidence
as a complementary reference alongside EU AI Act, IMDRF, and FDA guidance themes
not as a direct input that changes the formal score formula

That changed real artifact fields.

The post-M15 line now surfaces traceability in places such as:

human-readable Regulatory Traceability sections in HTML, Markdown, explain, and PDF
framework-grouped labels such as EU AI Act, ICH M15, IMDRF, and FDA
status-oriented summaries such as Signal only, Partially aligned, and Aligned
explicit source_ids and finding_refs so a reader can trace which repository signal triggered which regulatory mapping

That is why the right way to describe the integration is:

M15 strengthened traceability language and reporting context, but it did not become the hidden engine of the score.

This is also consistent with how FDA guidance should be read. FDA's own Federal Register notice states that guidance documents do not establish legally enforceable responsibilities; they describe the Agency's current thinking and should be read as recommendations unless specific statutory or regulatory requirements are cited. See the June 3, 2026 Federal Register notice for M15: https://regulations.justia.com/regulations/fedreg/2026/06/03/2026-11112.html.

This distinction also helped the report become more honest.

Regulatory Traceability is useful because it tells a reviewer:

which frameworks the observed evidence touches
which mappings are only signal-level
which are partially aligned
what the report still cannot claim

That is exactly where a framework like M15 belongs in this system: as a bounded interpretive layer that helps a reader connect local repository signals to external governance language more carefully.

Regulatory traceability now shows framework-grouped mappings, bounded statuses, and trigger-linked references, making it easier to see how local repository evidence touches M15, EU AI Act, IMDRF, and FDA guidance themes without mistaking those mappings for compliance proof.

Part 3. The Other Improvements That Actually Made the Tool More Mature

After the M15 integration, three other changes mattered just as much, and in some cases more.

3.1 The Tool Stopped Hiding Score Constraints

One of the biggest interpretability problems in earlier versions was that a report could be capped or floored without making that state obvious enough in the human-readable artifact.

That is what led to:

Tier Lock [CA-CAP], the clinical-adjacent score-cap state
Tier Lock [T0-FLOOR], the hard-floor state for stronger direct clinical concern
Classification Applied

These surfaces changed the meaning of the report.

They tell the reader that the formal score is not just an arithmetic total. It is also shaped by active classification state:

whether the repository is clinical-adjacent
whether an explicit non-clinical boundary is missing
whether a score ceiling is active
whether a hard-floor review path has been triggered

This made the report more inspectable, but more importantly, it made the report less willing to hide the reasons a higher tier is blocked.

That matters because remediation is not always “add more points.”

Sometimes the real issue is:

remove the condition that prevents the repository from being meaningfully read as governance-ready

That is a better audit posture than a naked scalar score.

3.2 The Report Became a Governance Document Instead of a Score Sheet

This was the most visible change to anyone reading the artifacts.

The detailed packet stopped feeling like a machine-oriented export and started behaving more like a governance-suitability document.

The current packet is built around a more explicit hierarchy:

Governance Posture
What Is Actually Present
What Is Missing Or Contradicted
Regulatory Traceability
AIRI Risk Triggers
Method Boundary

The current detailed packet is chaptered as:

Chapter 1 — Stage Scorecard and Governance Scoring
Chapter 2 — Code Integrity Deep Analysis
Chapter 3 — Regulatory Traceability
Chapter 4 — Remediation Actions, AIRI Risk Triggers & Method Boundary
Chapter 5 — Report Metadata

The HTML report similarly exposes a seven-section navigation path:

Summary
Decision Path
Code Integrity
Regulatory
AIRI Risk Triggers
Evidence
Developer

Those labels matter because they changed what the reader sees first and what the reader is expected to conclude from the artifact. The reader now moves through adequacy, contradiction, traceability, and scope before falling back to engineering detail.

Only after that does the packet lean into deeper developer-facing material such as:

Decision Path
Top Remediation Actions
Code Integrity details
Evidence detail

That reordering matters because the report’s first job is not to help a maintainer debug detectors. Its first job is to answer whether bio-governance is actually present, whether it is adequate relative to claims, and what remains unsupported or missing.

That is why the current packet structure is more than presentation work. It is a statement about document type: a governance artifact with

a posture statement
explicit scope limits
traceability context
contradiction surfaces
remediation direction

3.3 MICA, Packaging, and Release Surfaces Became Release Integrity Work

The final maturation step was less glamorous, but it mattered a great deal.

In v1.8.x, active memory pointers, public version surfaces, preview assets, and package-data inclusion became impossible to treat as optional housekeeping.

If the release says one thing while:

MICA, the project's active release-memory layer, points somewhere else
packaged assets omit active files
report previews lag behind the actual runtime
public docs describe stale behavior

then the tool is not governed. It is merely assembled.

That is why post-M15 work spent real effort on:

rotating the active MICA trio cleanly
pruning live historical memory surfaces while preserving provenance in Git-tagged history
making report previews match the actual runtime output
hardening package-data and release-surface alignment

The practical examples here are not abstract:

README level tables and actual packet filenames had to agree on 8p, not 7p
tracked preview assets had to match the real generated HTML and PDF outputs
active MICA pointers had to reference the same live trio the package actually shipped
public docs had to stop describing stale section counts or old packet shapes

Small mismatches matter here because governance tools are judged by their own traceability discipline. If a report surface says 8p while the surrounding docs still describe 7p, the tool teaches the wrong lesson about its own evidence hygiene.

This sounds operational because it is. But it is also methodological.

A governance scanner that critiques target repositories for stale surfaces, unsupported claims, or weak provenance cannot remain credible if its own release memory and public artifact surfaces drift by version.

That is why the packaging and memory work belongs in the same story as the report work.

It reduced the number of places where truth could fork.

Where This Leaves the Project

If I had to summarize the post-M15 line in one sentence, it would be this:

STEM BIO-AI became less willing to let a convenient surface pretend to be the whole truth.

That shows up in several places at once:

the score is now shown with clearer purpose boundaries
score constraints are surfaced instead of buried
M15 appears as traceability, not as covert score inflation
AIRI is framed as secondary risk vocabulary, not proof
the packet now behaves more like a governance document
release memory and packaging are treated as release-integrity concerns

The tool is still bounded and deterministic. It still cannot see runtime truth, wet-lab reproducibility, model-output correctness, or clinical validation.

But in the v1.8.x line, it got better at saying exactly that.

And it got better at saying it in a form that a prospective user, a maintainer, and a reviewer can all use without needing to reverse-engineer the internal taxonomy first.

Roadmap

The next maturity steps are not only more detectors.

They are also:

improving human-readable explanations without overstating certainty
expanding the behavioral and path-sensitive side of static analysis without pretending it is dynamic truth
broadening benchmark calibration so score validity is less prior-heavy
continuing to align report purpose, release memory, and public surfaces so the artifact remains hard to misread

That is the real roadmap after M15.

Not just more coverage.

More disciplined meaning.

Repository: https://github.com/flamehaven01/STEM-BIO-AI
Live HF Space: https://huggingface.co/spaces/Flamehaven/stem-bio-ai

AI-SLOP-DETECTOR v3.8.1: When Code Generation Gets Cheap, Structural Trust Gets Expensive

Kwansub Yun — Thu, 04 Jun 2026 15:09:30 +0000

For a long time, the hardest part of software development was writing code.

That is no longer true.

As AI-assisted coding and agent-driven workflows become mainstream, the cost of generating code is collapsing. But the cost of understanding, reviewing, simplifying, and deleting code is rising just as quickly. Code is now easier to append than to validate. Easier to duplicate than to consolidate. Easier to generate than to safely remove.

That asymmetry is creating a new engineering problem. The question is no longer only:

How do we generate more code faster?

It is increasingly:

How do we stop generated code from silently degrading the structure of a codebase?

That is the space AI-SLOP-DETECTOR is being built for.

v3.8.1 matters because the project is moving from detection toward governed cleanup, while keeping three layers separate:

scoring: measure structural risk
action planning: prioritize what is safe or important to review
enforcement: verify what must fail closed

That separation is the real story of this release. It is also the strongest reason to take the project seriously.

Why This Release Matters Now

There are many tools that claim to measure “AI code quality.” The meaningful distinction is not whether they can emit findings. It is whether they preserve boundary discipline when the findings start to drive workflow.

v3.8.1 is important because it sharpens three claims:

The scoring path became safer
Cleanup became more actionable
Governance became harder to bypass

Everything else in this release is evidence for one of those three claims.

Changelog Evidence Since v3.6.0

The recent releases make more sense as a sequence than as isolated feature drops.

Version	Key Change	Why It Mattered
`v3.6.0`	Claude Code Skill, CI gate fix, pre-commit rewrite, VS Code packaging	The project became more workflow-aware, not just scan-aware
`v3.7.0`	Dogfooding calibration, renderer/module splits, self-repair from internal audit	Maintainability and internal trust improved
`v3.7.1`	False-positive reduction, richer skill routing, VS Code modularization	Lower friction and better usability
`v3.7.2`	Config/schema validation and runtime data guards	The scoring path became harder to corrupt
`v3.7.3`	Import/package stability and CI fixes	The tool became more reliable in real environments
`v3.7.4`	Major false-positive patch wave	Trustworthiness improved materially
`v3.7.5`	`phantom_import` flat-project fix	A visible correctness gap was closed
`v3.7.6`	`deficit_breakdown`, idempotent `--init`, first-run UX improvements	Explainability and onboarding improved
`v3.7.7`	Cross-language aggregation fix, ignore matching fix, ML reproducibility fix	Project-level correctness improved
`v3.7.8`	Structural scaling, suppression ledger, cache, hotspots, agent API	The tool became more operational
`v3.7.9`	Governance verification gate and math/policy separation	Enforcement became explicit and fail-closed
`v3.8.0`	Canonical CLI: `scan / review / pulse / sweep`	The public surface became simpler and more stable
`v3.8.1`	Cleanup confidence planning, manifest hygiene, layered architecture review	The tool moved from issue listing toward action planning

Seen together, these releases show a pattern: not just more features, but more correctness, more explainability, more governance, and more usable workflow surfaces.

Claim 1: The Scoring Path Became Safer

The most important technical reinforcement since v3.6.0 is not that the project added more signals. It is that the project made the scoring path safer to trust.

The core model still uses a weighted geometric aggregation across four dimensions:

with the deficit-oriented score driven by:

Here, P pattern represents the additional penalty assigned when repeated structural patterns reinforce the deficit.

That formula is not the interesting part by itself. The important part is what was reinforced around it.

What changed

config values are validated before they enter the model
metric ranges are guarded before they can poison the score
deficit_breakdown makes score attribution inspectable
cross-language aggregation no longer misstates project summaries
structural coherence now scales with deterministic fallback above a ceiling

Why it matters

Without those reinforcements, the formula risks becoming authority texture. With them, it behaves more like an engineering instrument.

For a technical reader, the observable improvement is not abstract math prestige. It is:

fewer broken summaries
fewer config-induced distortions
better explanation of where a score came from
predictable behavior on large repositories

In short, the model became harder to misuse, easier to explain, and more stable at scale.

Claim 2: Cleanup Became More Actionable

Most code-quality tools stop at issue emission. That is useful, but incomplete.

Developers do not only need to know what exists. They need to know:

what is important
what is probably safe to review
what needs human caution
what should be looked at first

That is where v3.8.1 makes its clearest product-level leap.

Cleanup confidence planning

Cleanup-family outputs can now carry:

confidence
action_class
evidence

The important architectural choice is that this was not implemented as a second disconnected scoring model. Cleanup confidence is a reuse layer over existing signals:

deficit_score
churn
coverage gap
cleanup-local evidence

A simplified mental model looks like this:

confidence = base_evidence
confidence += low_churn_bonus
confidence += low_coverage_bonus
confidence -= active_churn_penalty

The exact arithmetic is less important than the architecture: the system is not maintaining one truth model for scoring and another truth model for cleanup.

Manifest-aware dependency hygiene

unused-deps also grew beyond file-local hints. It now reads:

pyproject.toml
package.json

and can emit:

manifest_unused_dependency
undeclared_import

That matters because many dependency problems are not visible inside a single file. They exist at the boundary between source code and project metadata.

Why it matters

Before:

sweep -> list of candidates

After:

sweep -> ranked issues -> action class -> evidence-backed review plan

That is the difference between a detector and a cleanup instrument.

Claim 3: Governance Became Harder To Bypass

This is arguably the article’s strongest credibility anchor, and it deserves to be said plainly:

The project does not ask the score to become policy, and it does not let policy quietly mutate the score.

That is the right architectural judgment.

What changed

The project now treats governance as a separate fail-closed path:

analysis emits a deterministic governance artifact
verification recomputes the artifact hash
policy checks run in a dedicated verification gate

The workflow is intentionally layered:

analysis -> governance_record.json -> verify-governance -> pass/fail enforcement

Why it matters

This separation means:

math can evolve without silently changing CI policy
policy can become stricter without corrupting the scoring model
governance can be audited as an artifact, not just inferred from a transient report

In a category crowded with vague “AI code quality” claims, this is the kind of subsystem separation that actually signals seriousness.

Supporting Reinforcements

The release also includes several important supporting improvements that strengthen the three main claims without replacing them.

Layered architecture review

Architecture analysis can now opt into a layered preset rather than stopping at import cycles alone.

A simplified configuration looks like this:

architecture:
  enabled: true
  preset: layered

The built-in intent is narrow by design:

api -> domain allowed
domain -> data forbidden
domain -> service forbidden
domain -> api forbidden

This is not enabled by default, and that is correct. Architecture review is valuable only if it avoids becoming a false-positive factory.

Canonical CLI

The public CLI is now much easier to hold in memory:

scan
review
pulse
sweep

That simplification matters because adoption dies when the interface surface grows faster than user confidence.

Selective Rust acceleration

Performance work also stayed disciplined. The project did not rewrite itself around native code. It kept Python as the product core and used Rust only for measured hot paths such as:

file walking
glob-heavy traversal

That is the right trade. Native code is a performance helper here, not a product identity.

Five Topics Worth A Deeper Follow-Up

The following five areas deserve separate technical notes because they are where the release’s architecture becomes most visible.

1. Mathematical Model Hardening

The scoring model did not need a louder formula. It needed a safer boundary.

That is why the important work happened around validation, metric guards, cross-language aggregation, attributed deficit output, and deterministic fallback above scale thresholds. The benefit is practical: fewer strange summaries, safer config changes, and score outputs that are easier to debug.

scan -> validated metrics -> attributed score -> project summary

The model now behaves less like an opaque detector and more like a measurement subsystem.

2. Cleanup Confidence Planning

“This might be dead code” is not enough guidance for real cleanup work.

v3.8.1 moves cleanup closer to a review plan by attaching confidence, action class, and evidence to cleanup-family findings. The key design choice is reuse: cleanup confidence draws from existing signals such as deficit, churn, coverage, and local evidence instead of inventing a second truth system.

sweep dead-code -> ranked issue -> action class -> evidence

That makes cleanup safer for humans and easier for agents to consume.

3. Manifest-Aware Dependency Hygiene

Dependency debt is often project-level, not file-local.

By comparing declared dependencies, imported dependencies, and normalized top-level mappings across pyproject.toml and package.json, the tool can now surface manifest-level problems such as unused declared packages or missing declarations.

manifest -> imports -> used / unused / missing -> cleanup output

That turns unused-deps from a file hint into a repository hygiene signal.

4. Layered Architecture Review

Cycle detection is useful, but many architecture failures appear before cycles do.

The layered architecture preset gives teams an opt-in way to express allowed and forbidden import directions, with evidence attached to the violation. The important part is restraint: this is not forced on every repository.

boundary-violations -> cycles + optional layered rule review

That keeps architecture review useful without turning it into noisy certainty.

5. Governance Verification Gate

Measurement and enforcement should not collapse into the same layer.

The governance gate creates a deterministic artifact, verifies it separately, and fails closed when policy or integrity checks break. That makes CI behavior more explicit and audit-friendly.

scan -> governance artifact -> verify-governance -> pass / fail

This is one of the strongest separations in the system: measurement, artifact generation, and enforcement each have their own boundary.

Why This Category Will Keep Growing

We are still early.

Most teams are only beginning to feel what large-scale AI-assisted development actually does to a repository over time. At first it feels like acceleration. Then it starts to feel like churn, duplication, abandoned logic, inflated structure, and uncertainty about what is still safe to touch.

That is why interest in slop will keep rising.

The more code agents can generate, the more valuable tools become that help humans decide what should never have remained in the codebase in the first place.

As agent-driven code development becomes more mainstream, the need for systems like this will likely accelerate:

measure structural trust
prioritize cleanup
separate evidence from policy
make deletion safer
make governance explicit

AI-SLOP-DETECTOR is being built gradually in that direction.

Not as a one-shot idea.
Not as a trend-chasing wrapper.
Not as a linter with a fashionable label.

But as a system shaped step by step around a simple reality:

if AI makes code generation cheap, then structural review, cleanup discipline, and governance become more valuable than ever.

That is the craft mindset behind this project:

refine the instrument
tighten the workflow
separate the layers
improve the trust surface one release at a time

That is the craft mindset behind this project:

refine the instrument
tighten the workflow
separate the layers
improve the trust surface one release at a time

Repository: https://github.com/flamehaven01/AI-SLOP-Detector

When the Memory Gate Met a Real Archive: What 90 Experiments Taught Us About Cheap LLM Slop

Kwansub Yun — Wed, 03 Jun 2026 18:15:07 +0000

Introduction: Enforcing the MICA Contract

This article is the practical side of the MICA series. MICA stands for Memory Invocation and Context Archive. In the workflow described here, it is a small package that the maintainer loads at session start so the active rules are visible before any code is touched.

Parts 6 and 7 set up the contract. This article shows what that contract did when a real scientific archive started accumulating cheap slop across more surfaces than a single maintainer could manually hold.

The archive is the Flamehaven Verification Ledger. It publishes three kinds of records.

EQA (Equation-to-Artifact). Physics and math reproductions. Currently 56 records, numbered TOE-TEST-0001 through TOE-TEST-0056. Example: a Schwarzschild Planck-scale metric verification.
BAV (Biomolecular AI Validation). Protein-folding consensus checks across several AI fold models (AlphaFold3, AlphaFold2, Chai-1, Boltz-2). Currently 34 experiments, with 6 active cards and a 26-entry foundational archive.
BSC (Bioscience Compliance). Repository compliance audits against external risk taxonomies (the MIT AI Risk Repository and EU AI Act). Currently 2 audits.

That is around 90 experiments in total. The full file count is past 300. Every record is published. The archive is intended to be cited. If the AI maintainer drifts, the drift can become a downstream paper citation.

One scope note matters before the story starts. flamehaven-audit-reports is not the engine that computes these results. It is the public evidence surface. Upstream engines and experiment repositories produce the raw artifacts.

This repository ingests those artifacts, sanitizes them for publication, classifies what kind of record they are, and renders them in a static ledger that other people can inspect and cite.

The three lanes have already taught us three different shapes of cheap slop. EQA taught us about framing drift at scale (a record displayed as a PASS when no real check produced it). The portal taught us about state duplication (an inline JavaScript copy that drifted from the disk file behind it).

BAV keeps trying to teach us about provenance drift, artifact-identity drift, and over-clean presentation around real runs. The article walks those three scars in order and then describes the gate that grew out of them.

📖 Glossary

A short list. Skim and move on.

MICA. A small package the maintainer loads at session start. It carries the rules and exposes whether the package state is coherent before write work begins.
DI (Design Invariant). A rule with an ID. Example: DI-EQA-001 says math runs must use mpmath at 200-bit precision or higher.
Playbook. A markdown file. People read it. Every rule inside cites a DI.
Schema. Two machine-readable files. mica.yaml carries the package shape. archive.json carries the 28 DIs.
Validator. mica_pct.py. When run against a package root, it emits CLOSED CONTRACT or INCOMPLETE.
Receipt. A small JSON block proving a run actually ran. Pins the engine commit hash, the run command, and the output hash.
EQA / BAV / BSC. The three lanes of the archive. Physics math, protein folding, compliance.

That is enough to read the rest.

1. The Archive We Are Talking About

The story needs a concrete protagonist. The protagonist is the archive itself. The opening named the three lanes. This section adds the file shape, three live numbers, and one honest scope label that the rest of the article will keep returning to.

The protagonist is not a single program. It is a layered publication system. Upstream computation happens in engine or experiment repositories. flamehaven-audit-reports is the place where those outputs are turned into public records.

That projection layer does four jobs that are easy to blur together if they are not named explicitly:

Ingest the upstream artifact
Sanitize anything that should not be published as-is
Classify the record by what kind of evidence it really is
Render it through a static inspection surface.

The three lanes do not all behave the same way inside that surface:

EQA Lane: Closest to a deterministic computation archive.
BAV Lane: Pipeline- and governance-heavy. The ledger must distinguish between a genuine rerunnable experiment, a runtime audit, and a research or review artifact.
BSC Lane: Maps repository state directly to external compliance taxonomies.

Each EQA record carries at least two files. A machine-readable internal_data.json holds the receipt. A human-readable analysis_report.md holds the narrative. Some records also ship a SPAR review record. The strongest three current records:

Schwarzschild Planck-scale metric verification. Engine re-execution produced Omega = 0.9985.
de Sitter background check. Recorded at sqrt_jsd = 0.2722.
OpenAI Erdős Eq.(2.2) reproduction. Claim: matches the published value to 0.014 percent. Anchored to a public MIT repo and a Zenodo DOI.

The BAV lane is honest about its own scale. Of the 6 active cards, only one (EXP-031) carries a foldable input sequence. It is a 52-amino-acid input run against AlphaFold3, AlphaFold2, Chai-1, and Boltz-2.

The other five cards are governance and methodology experiments. They do not ship a re-run scaffold. That boundary is honest. We did not invent a fake fold to fill the slot.

The total is past 300 files. That number matters. A single maintainer can review 5 records by hand. Nobody can review 300. Slop scales with file count. Review does not.

2. What the EQA Lane Taught Us First

The first scar was about labels, not numbers. The numbers were correct. The labels around them were wrong.

In June 2026, we ran an internal audit on the EQA archive, the lane that publishes the physics and math reproductions. The plan was to spot-check the calculations. We re-ran the engine on a sample of records. The numbers matched.

A Schwarzschild horizon calculation came back at Omega = 0.9985. A de Sitter background check came back at sqrt_jsd = 0.2722. The math was honest.

The audit found a different problem.

The public website was showing 51 of the 56 records with a green PASS badge. A green PASS is supposed to mean: a numerical check ran and the result passed a threshold. That was not what the website was doing.

It was treating every record that had a markdown analysis file as PASS, whether or not a real check had ever run. Governance notes, scenario builds, and integration documents all showed up as if they had been verified.

A reader scanning the page saw "51 successful verifications." When we sat down and went through the 51 records by hand, only 7 of them had come from a real engine run. The other 44 were notes and supporting documents that had been imported into the lane over time.

The numbers did not change. The framing did.

We rewrote the page headline to say "7 verification runs and 44 supporting documents." We wrote a new rule into the package contract that lives next to the records on disk.

The rule says, in plain English: a green PASS badge can only come from a real threshold check. The mere presence of a report file is not a PASS. A grade copied in from someone else's report is not a fresh verdict. The five most recent records (numbered TOE-TEST-0052 through TOE-TEST-0056) carry their own real verdicts.

One thing to be clear about. This audit was a manual one-time read. The MICA validator did not catch the drift. We caught it by reading the records ourselves and asking what each one actually claimed. What MICA does now is preserve the lesson in the package contract and in the maintainers' workflow.

The contract status the validator emits when everything lines up is called CLOSED CONTRACT. That does not mean every semantic rule is automatically enforced by the validator itself.

It means the package structure, declared layers, and DI bindings are coherent, and the maintainer is expected to run inside that contract before changing the archive.

This is the kind of failure no CI gate or syntax check would catch. The math was correct. The framing was wrong. A markdown-only policy would have continued to allow it because every file would have parsed cleanly. The rule survived this kind of pressure because the contract records both what the rule says and the specific incident that forced it to exist.

The next scar hit a different part of the system. Not the math lane this time. The website itself.

3. The Forgiveness Budget Scientific Archives Don't Have

Most LLM-assisted writing operates on a forgiveness budget.

A blog post can be slightly overstated. A README can describe something the code does not quite do yet. A pitch deck can round 73% up to "over 70%." The reader corrects internally. The next revision absorbs the drift. The social cost of small overclaiming is low.

A scientific archive does not have that budget.

This archive is published in a form meant to be cited. The Schwarzschild Omega value, the Erdős reproduction match percentage, and the EXP-031 fold metrics are all the kind of claims that can become downstream references. The drift that is harmless in a blog post becomes a poisoned downstream paper citation here.

The model that helpfully rewrites a paragraph also helpfully invents a SMILES string (the text encoding chemists use for molecules) that looks chemically plausible. The agent that summarizes a build log will, if asked one too many times, invent a DOI. The same instinct that makes LLMs useful for prose makes them dangerous for an archive.

The objects that have to survive this environment are the ones an LLM is least equipped to verify on its own. SMILES strings. DOIs. AlphaFold pLDDT values (per-residue confidence scores for a fold). Numerical thresholds with physical meaning. Record-level provenance. None of these can be caught by spell-check or by a continuous-integration pipeline.

This is why the archive needs a gate that loads before any code is touched.

4. The Failure That Forced the Cross-Lane Gate

The second scar was inside the website that displays the archive.

The site used to ship a fallback copy of every record inside the JavaScript file that runs in the reader's browser (js/portal.js). The original purpose was harmless. Some readers download the repository and open the homepage by double-clicking it, which uses the file:// URL scheme. Some browsers refuse to load separate JSON files over file:// for security reasons, so the fallback let the page render anyway. Two small functions held the fallback. One returned a copy of the dataset for a record. The other returned a copy of the human-readable report.

The on-disk files kept changing. The inline copies inside the JavaScript did not. The drift grew quietly over weeks.

A maintainer wrote a small drift-checking script and ran it. It compared every on-disk record with its inline twin. It found 151 places where the two copies disagreed. The most striking example was a record about the Erdős reproduction whose schema_id field did not even share the same structure between its two copies. The AI maintainer had been editing the disk files. The website had been rendering the stale inline copies. Both sides looked fine internally. Neither side agreed with the other.

A GOVERNANCE.md had said "single source of truth" the whole time. The maintainer agreed with the policy. The model also agreed. The policy lived in prose. Nothing in code enforced it.

Same shape as the EQA framing audit. A human caught the drift, not the MICA validator. What MICA does now is preserve the new rule in the package contract and surrounding docs. The rule, in plain English, is: no inline copy of any record may ship inside the browser code. The two functions that used to return the inline copies were stripped out and replaced with stubs that return empty values. The stubs carry the history inline, so a future maintainer reading the file sees both the rule and the incident that forced it.

function getFallbackReportText(runId) {
  // Removed (v1.13.1): inlined report-text fallback drifted from the on-disk .md
  // reports. Single source of truth = the on-disk files fetched above.
  return '';
}

function getFallbackDataset(runId) {
  // Removed (v1.13.1): inlined fallback datasets had drifted from the on-disk JSON
  // (151 schema/value mismatches found 2026-06-02 by check_fallback_drift.py).
  // Single source of truth = the on-disk evidence files fetched above. This ledger
  // must be served over HTTP (e.g. "python -m http.server"), not opened via file://.
  // Returns null so the inspector shows an honest load error rather than stale data.
  return null;
}

The fix lives in three places. The archive's machine-readable contract carries the lesson. The browser code returns empty values where the inline copies used to live. The playbook (the human-facing operating guide) explains why those values are empty.

A new maintainer joining the project sees the rule from all three angles. The validator confirms the package still loads as a coherent contract. The code refuses to render the old fallback because the function returns nothing. The playbook explains why a human should not put the fallback back in.

This is the lesson that produced the title of the article. A memory gate that lives only in markdown is etiquette. The gate becomes structural the moment the contract, the code, and the playbook all point at each other.

5. What the Playbook Actually Does

People ask why we ship a human-readable playbook (a long markdown file) if the contract is already a machine-readable file. The clearest answer is a short list of cheap failures the playbook actually prevented.

A maintainer who reads only the machine-readable contract sees one rule: math must use an arbitrary-precision library at 200 bits or higher. That is precise. It is also blunt. It does not say why. The first time the maintainer hits a math sub-case the contract did not specifically name, they may default to the standard 64-bit floating-point library.

The playbook is where the original incident behind the rule lives. In our case, an early experiment where 64-bit floats silently underflowed to zero in a class-field calculation and produced a meaningless result of 0. The playbook tells that story in plain English. A maintainer who read it will not re-introduce the same bug in a new sub-case.

An AI maintainer that starts work without loading the playbook will, when asked to fix a wrong score in a record, simply edit the JSON file that stores the score. The contract forbids this in one terse sentence. The playbook expands that sentence into a behavior rule. Never edit a record's data file after it has been committed.

Instead, create a new record with a new ID and link the corrected record from the original one. A session that loaded the playbook reads that rule before any edit happens. A session that did not load it destroys the audit trail that lets a third party re-run the original computation.

The third example happens at render time. The website uses a small classifier (a regular expression) to decide what kind of colored label sits next to each metric. When the classifier does not recognize a metric name, it returns nothing, and the metric renders without any colored label.

The contract says what to do, in terse machine terms. The playbook documents the human procedure step by step. Add the new metric to the glossary, decide what kind of evidence backs it, assign the matching label, then merge.

Without the playbook, a record with a missing label might ship as if the missing label were on purpose.

The playbook is not the rule. The contract is. The playbook is the briefing for the maintainer about to face the rule, and the record of the specific past failure each rule was written to prevent.

6. Where MICA Sits, and What It Refuses

MICA is a small Python validator plus package format. In the workflow used here, the maintainer runs it at session start. The script reads the package contract first (a short YAML file). The contract names three other files.

The validator confirms those layers exist and that the package shape is coherent. The first is the archive's machine-readable rule list. The second is the human-readable playbook. The third is the credibility document that says what kinds of internal scores may or may not appear on the public surface.

After loading, the script runs 11 simple structural checks against the package. Each check catches one specific kind of cheap failure before write work proceeds.

The first group of checks refuses a half-formed package. The script asks whether the contract declares the required shape fields (mica_spec, mode, layers), whether the archive and playbook layers exist, and whether the mode/layer combination is coherent. The package is unusable until those fields line up.

The second group refuses drift between what the contract says and what the file system actually holds. The script asks whether every file the contract names exists on disk. A check here fails when a file was renamed in one place and not the other. This is the same shape of failure as the website's inline-fallback drift from the second scar, but caught much earlier.

The third group refuses critical rules that have no accountability behind them. Every critical archive rule is supposed to carry a short note naming the incident that forced the rule. The script asks whether binding.origin_episode is filled in for every critical rule.

A check here fails when a rule was written as a top-down policy with no recorded cost behind it. Rules like that are easy for a maintainer or an AI maintainer to rationalize past in the moment. A rule that names what was paid the last time it was missing is much harder to ignore.

The fourth group refuses stale package references. The script can check whether any declared binding.lesson_ref paths still resolve. A broken cross-reference is how a rule slowly becomes etiquette.

The sequence at session start looks like this:

If every check passes, the script emits the status CLOSED CONTRACT. If a hard-fail check trips, it emits INCOMPLETE. In the workflow used here, the maintainer fixes that state before any code change happens.

This is what we mean by a gate that is meant to run before any code is touched. It is not just a policy hope. It is a small Python validator with explicit hard-fail conditions.

The 28 archive rules do the same thing at the per-record level. We did not aim for 28. The number grew as incidents forced new rules. Every rule carries a short note pointing at the incident that produced it. The list is not a top-down policy. It is an accumulated record of past failures the team agreed not to repeat.

7. One Bad BAV Card, Step by Step

Note before the walk-through. This scenario is a constructed illustration, not a documented incident. The protein-folding cards on the archive today are all well-formed. The point of stepping through it is to show the refusal sequence at the granularity a peer reviewer can check, not to claim that a refusal of this exact shape has been logged in production.

To make the gate concrete, here is one fabrication the contract refuses.

Step 1. An AI maintainer is asked to add a new protein-folding card. There is no real protein sequence on disk to fold, but the model knows the file format the lane expects. It writes a record at bav/exp-035/reference_run.json that looks like a real fold result. The file carries pTM = 0.78, pLDDT_mean = 84.2, PAE = 4.3 Å. These numbers fall inside the same range as the only real fold on the archive (EXP-031), so they pass a casual eye-test.

Without the gate, the rest follows naturally.

Step 2. The website's small classifier reads the metric name pLDDT_mean. It matches the pattern for an externally-defined fold metric (AlphaFold defines pLDDT, so the website treats anything named that way as borrowed from outside, and therefore checkable by a third party). The card renders with a green "verifiable" badge. The classifier is just a short regular expression. Here is what it does:

function provClassOf(label) {
  const s = String(label == null ? '' : label).toLowerCase();
  if (/(plddt|\bpae\b|ptm|contact|brier|\bauc\b|\bece\b)/.test(s)) return 'EXTERNAL';
  if (/(p_e2e|e2e|capture|transfer)/.test(s)) return 'DERIVED';
  if (/(sr9|di2|sidrce|coherence|spar|nnsl|resonance|drift|omega|ω)/.test(s)) return 'ADVISORY-HEURISTIC';
  return null;  // incidental values (counts, dates, grades) carry no badge
}

The label pLDDT_mean contains the string plddt, so the first pattern matches. The function returns EXTERNAL. The badge turns green. The regular expression has no way to check whether the number behind the label came from a real fold or from a fabrication.

Step 3. A reader trusts the green badge. The value gets cited in a manuscript. A wet lab spends real money chasing a fold that was never run.

With the gate, the chain breaks at step 1.

The archive carries a rule that says a fold card can only claim re-runnable status if it ships two specific files alongside the result: a .fasta file containing the protein sequence that was folded, and a small JSON file naming the model version and the random seed used.

In this repository, that rule lives in the contract and in the surrounding spec, and the maintainer is expected to check it before publishing the card. If bav/exp-035 had no real input sequence, it could not honestly ship as a re-runnable fold. At most it would ship as non-re-runnable or stay unpublished. The reader would see the honest label.

A standalone GOVERNANCE.md would not have stopped step 1. A YAML config without a validator would not have noticed the missing input file. An agent system prompt would have been compressed away under context pressure. A CI check would have run too late, after the fabrication was already on the public surface.

The contract, the playbook, and the validator reduce the chance of that chain because all three point at each other, and because the workflow checks the contract before the file is allowed to settle into the public archive.

8. What This Pipeline Cannot Block

A pipeline that pretends to catch everything is the failure mode it was built to prevent.

Five things still slip past every layer above.

A plausible fabricated value inside the normal range. A fake pLDDT of 78.4 looks like a real one. The website's classifier labels it as externally-defined and the green badge appears. Only a third party re-running the fold catches the fabrication. This is why only EXP-031 ships the full re-run scaffold, and the other five active BAV cards do not claim independent re-runnability.

A new promotional pattern outside the word list. A small filter watches the public pages for 14 superlative terms such as revolutionary and breakthrough. A maintainer who writes something like "a novel adaptive coherence framework" defeats every entry on the list. The list is a floor, not a ceiling.

A fabrication marker silently removed. A separate filter looks for a literal [synthetic] tag in shipped files, the kind of marker a developer might leave on placeholder data. The filter only fires when the tag is present. A maintainer who deletes the tag while keeping the fabricated content underneath passes the filter cleanly.

A real DOI pointing at the wrong paper. Nothing in the pipeline fetches DOIs. A real URL pointing to a real but unrelated paper is invisible to the validator. Peer review is the only check.

A correct computation framed as the wrong thing. This was the failure that produced the framing rule in the math lane. The engine outputs were real. The headline treated the mere presence of a report file as a fresh PASS. The fix was structural, but the same shape of error can reappear in any new lane.

The honest claim is narrow. MICA makes cheap slop expensive enough to catch. It does not make expensive slop catchable.

9. What We Learned, What We Did Not Solve

The archive is small by industry standards. 56 math records. 34 biomolecular-validation experiments (6 active cards and a 26-entry foundational archive). 2 compliance audits. Around 90 experiments. Past 300 files in total.

That scale was large enough to teach us four things.

First, a markdown policy alone does not survive an AI maintainer. The 151-mismatch drift proved it. The policy was correct. Nothing in code enforced it.

Second, the rule list is where the policy actually lives. The playbook is the human reading layer. The validator is the structural gate. The workflow is the enforcement surface. Together, they form one operating contract.

Third, the gate works best when it runs before any code is touched. PR-time checks are necessary, but not sufficient. By the time cheap slop reaches the PR surface, the maintainer is already reviewing content that should have been constrained earlier.

Fourth, the pipeline only refuses cheap slop. It does not verify molecules, fold real proteins, or check that a DOI links to the paper it claims to. That work stays external. The pipeline buys reviewer time so the reviewer can do that external work on the few claims that genuinely need it.

What we did not solve.

The website's metric classifier still misclassifies on a typo. A maintainer who writes pLDT instead of pLDDT ships a card with no colored badge at all. Nothing automated catches it. Reading the PR diff before merge is the only safety net.

The fabrication-marker filter is bypassable. Anyone who knows the [synthetic] tag is there can delete it, and the underlying content goes through.

DOIs are not fetched. A real URL to a real but unrelated paper passes every layer.

And the article has not shown a logged production refusal by the MICA validator itself. The validator's refusal logic is exercised by a few test fixtures inside the MICA repository (small example packages deliberately broken in specific ways).

The fixtures prove the mechanism works as designed. They do not prove that the gate has fired in production on this archive yet. The two real incidents in this article were both caught by human attention. The framing drift in the math lane was caught by a one-time read.

The website's fallback drift was caught by a small script a maintainer ran. The rule list records both lessons. The next time either shape returns, the contract now names the failure pattern and creates a refusal point where automation, workflow, or human review can tighten around it. We do not yet have a refusal log entry to point at.

The pattern across these gaps is the same. Cheap slop is refused upstream. Expensive slop is left for the maintainer's reading and for peer review.

The maintainer has finite attention. Every minute spent catching a fabricated pTM = 0.78 is a minute not spent reading the molecule, the protocol, or the citation that actually needs human judgment.

Session-start refusal exists to move the cheap failures upstream so the saved attention can land on the expensive ones. The contract does not pretend to verify the world. It frees the maintainer to verify the parts that matter most.

This article was the practical side of what Parts 6 and 7 set up as a session-start contract. The next part of the MICA series will return to the framework side.

The reproduction handle for the strongest record on the ledger is short:

git clone https://github.com/Flamehaven-Labs/openai-erdos-eq22-reproduction
cd openai-erdos-eq22-reproduction
python -m pytest

Treat any number on the ledger as a number to verify, not a number to trust.

Making Equation (2.2) of the OpenAI Erdős Result Executable

Kwansub Yun — Tue, 26 May 2026 06:37:10 +0000

Why a proved theorem still needs reproducible claim custody

On May 20, 2026, OpenAI announced that an internal reasoning model had produced a counterexample to the Erdős planar unit-distance conjecture.

The problem is easy to state: given $n$ points in the plane, how many pairs of points can be exactly distance $1$ apart?

For nearly eighty years, the prevailing expectation was that square-grid-type constructions were essentially optimal up to a slowly growing exponent. OpenAI’s announcement changed that. Its internal reasoning model produced an infinite family of examples giving a polynomial improvement, and the proof was checked and written up in mathematical form by external mathematicians.

In this article, “the remarks paper” refers to the companion PDF by Alon, Bloom, Gowers, Litt, Sawin, Shankar, Tsimerman, Wang, and Matchett Wood, linked from OpenAI’s announcement.

The proof-level result belongs to those authors and the source papers.

My focus here is narrower: equation (2.2) in that remarks paper, and whether its explicit numerical value can be reproduced as executable code.

This is not about proving the theorem again. It is about what happens after a theorem contains a fragile numerical claim.

The proof is not the artifact

A mathematical proof and a software artifact do different jobs.

The proof establishes the theorem. It gives the definitions, the argument, the dependencies, and the mathematical reason why the result holds.

A software artifact should not pretend to replace that.

But some claims inside a mathematical paper have a finite, numerical, or computationally checkable surface. Those claims can be preserved differently. They can be run. They can be tested. They can fail when precision is wrong.

That is the narrow role of an executable reproduction artifact: not proof replacement, not automated peer review, and not authority over the theorem, but a reproducible object for the part of the claim that can be computed.

The specific target: equation (2.2)

In the OpenAI Erdős result, one checkable surface is equation (2.2) of the remarks paper.

For the explicit choice

the remarks paper gives an explicit numerical lower bound on the exponent excess above the classical Erdős exponent:

These parameters are taken directly from the remarks paper without modification. The artifact does not derive the multiquadratic choice; it reproduces the finite numerical calculation built from that choice.

This is not the later stronger explicit bound associated with Sawin’s separate preprint. It is not $\delta \approx 0.014$. It is the numerical value appearing in equation (2.2) of the remarks paper.

That narrowness is important. It is exactly what makes the claim suitable for executable reproduction.

Where the numerical fragility comes from

The numerical fragility comes from the exact form of equation (2.2), not from a large computation.

Immediately after the published expression, the parameters are:

and

With the paper’s definitions of $u, v$, and $\delta$ substituted into equation (2.2), the exponent excess reduces to:

The constant $36$ is not introduced by the implementation. It is already present in the remarks paper’s equation (2.2), both in the numerator term $u\pi/(36v)$ and in the denominator term $\log(36/\delta^2).$

After substituting $u = K/r^2, v = r/2$, and $\delta = 101^{-2K}$, the numerator simplifies to $\log(K\pi / 18r^3)$, while the denominator becomes $\log 36 + 4K \log 101$.

Here the $101$ comes from the finite prime in $S = {101, \infty}$.

In other words, this artifact does not derive the constant $36$ from first principles; it reproduces the published equation with the stated substitutions.

The precision problem is in the numerator:

Because $K$ is the ceiling of $18r^3 / \pi$, the ratio $K\pi / 18r^3$ is only barely larger than $1$.

More precisely:

For $r = 510510$,

So the numerator is effectively $\log(1 + \varepsilon)$ with $\varepsilon$ at the $10^{-18}$scale.

IEEE 754 double precision has machine epsilon around $2.2 \times 10^{-16}$. A naive float64 computation therefore cannot reliably distinguish the near-one ratio from $1$. The ratio rounds to $1$, leading to $\log(1) = 0.$

The exponent excess disappears before the computation reaches the value stated in the paper.

This is not a flaw in the mathematics. It is a precision failure in the numerical evaluation of a valid expression. That is the reason the artifact evaluates equation (2.2) using mpmath at 200-bit precision.

A PDF can state the value. A verifier can expose when the value disappears.

What we built

We built:

https://github.com/Flamehaven-Labs/openai-erdos-eq22-reproduction

The purpose is deliberately narrow: reproduce the finite, explicitly checkable numerical surface of equation (2.2) in the OpenAI Erdős unit-distance disproof remarks.

The package evaluates the expression using mpmath at 200-bit precision and returns:

6.2391e-38

This matches the published two-significant-figure value $\approx 6.24 \times 10^{-38}$ to $1.4 \times 10^{-4}$ relative error.

The repository includes 60 unit tests, 21 verifier checks, a frozen per-source-file SHA-256 manifest, GitHub Actions CI across Ubuntu and Windows, Python 3.11 / 3.12 verification, and a frozen-report mode that prints a verdict without mutating tracked evidence.

The basic reproduction path is:

git clone <https://github.com/Flamehaven-Labs/openai-erdos-eq22-reproduction>
cd openai-erdos-eq22-reproduction
pip install -e ".[dev]"
python -m erdos_ant.verify

Expected output includes:

Verdict: PASS
Checks: 21/21 passed
eq (2.2) exponent excess: 6.2391e-38

This is not a large system. That is part of the point. A small claim with a clear boundary is easier to inspect than a broad claim that blurs proof, computation, and interpretation.

From reproduction to custody

This repository was not built as a one-off reaction to an OpenAI announcement. We are not announcing a grand framework here; we are showing the discipline in miniature.

For us, the work is part of a longer routine: take a mathematical or technical claim, isolate the checkable surface, pin the environment, and make drift visible.

That is intentionally plain work.

Read the source.

Extract the claim.

Reproduce the computation.

Record the boundary.

Let the verifier fail if the result disappears.

To execute this routine reliably, the scope must be uncomfortably narrow. This repository intentionally leaves the proof of Theorem 1.1, the construction of the infinite tower, and Sawin’s separate $\delta \approx 0.014$ preprint to their respective sources. It does not pretend to be peer review.

This is not just a disclaimer. It is the point of the artifact.

A sharp, restricted boundary is exactly what makes a claim inspectable, repeatable, and challengeable. This is what I mean here by claim custody.

It addresses a technical governance question, but not in the policy sense: what exactly is being trusted, from which source, and what makes the claim fail if the implementation changes?

A PDF can state the value. A verifier can expose when the value disappears.

We claim no authority over the broader theorem. We simply maintain a reproducible boundary around the fragile numerical claim inside it.

Closing

The theorem was proved in the mathematical papers.

This repository asks a smaller question: can the numerical value in equation (2.2) survive execution?

In float64, it does not. The exponent excess collapses to zero.

At 200-bit precision, with the source parameters pinned and the verifier running under CI, the artifact recovers:

6.2391e-38

matching the published value to $1.4 \times 10^{-4}$ relative error.

That is the point.

Not a new theorem. Not a proof replacement.

A reproducible claim surface for one precision-sensitive number in a major AI-assisted mathematical result.

Repository:

https://github.com/Flamehaven-Labs/openai-erdos-eq22-reproduction

Paper / Zenodo:

https://doi.org/10.5281/zenodo.20383217

The README Was a Protocol. The Entrypoint Was Still Optional.

Kwansub Yun — Thu, 21 May 2026 10:34:02 +0000

Glossary: terms used in this article

🔸 MICA (Memory Invocation & Context Archive): A governance schema for AI context management. Defines how context should be structured, trusted, scored, and handed off across sessions.

🔸 Invocation Hierarchy: The operational ladder — natural, guided, forced — that determines how MICA actually reaches a live session.

🔸 Activation Packet: The compiled session-start object that declares read targets, load state, self-test posture, drift status, and gate outcome.

🔸 Session Report: The structured opening output that declares what was loaded, what the self-test found, and whether the session gate is open.

🔸 README-as-Protocol: The pattern where the model's natural tendency to read the README first is formalized as a declared invocation mechanism. Introduced in v0.1.8.

1. Where Part 6 Left Off

Part 6 showed what MICA looks like inside a single maintenance agent — session report, drift detection, design invariants, deviation log. The structure held. The protocol ran.

Part 6 ended with a harder question: what happens when accumulated session knowledge needs to govern the next session — inside a tool that runs within AI workflows itself?

The answer depends on a prior question: does the next session actually load what was accumulated?

That is not a schema problem. It is an entrypoint problem.

2. The Gap README-as-Protocol Left Open

Part 4 made a specific assumption: in many repository-based AI workflows, the README is already the model's first orientation surface.

That observation became README-as-Protocol.

Instead of inventing a new installation mechanism, MICA formalized an existing behavior: the model reads the README, the README points to the archive, and the session is expected to load context, run checks, and report readiness before work begins.

That assumption was useful.

It gave MICA a path into the session without requiring plugins, services, or custom host infrastructure.

But a protocol is not an entrypoint.

The README can declare where the archive is, what invariants matter, what the session report must contain. None of that guarantees sequencing. A model can still skim the README, jump directly into code, or begin work before declaring its load state.

A gate without a consequence is still only etiquette.

That is the gap this version had to close.

3. The Answer: An Invocation Hierarchy

MICA does not auto-invoke by magic. If no human, host, wrapper, or launcher calls the memory contract, the archive can exist without governing anything. This is the same truth Part 2 identified: the structure can exist, and the model can still have no reliable way to know it exists.

The answer is an explicit hierarchy.

Natural — the model reads the project surface voluntarily: README, mica.yaml, archive JSON, playbook. No intervention required.

Guided — a host agent requests the activation packet before work begins. The packet declares read targets, self-test posture, drift state, and gate outcome. The host uses it to preflight the session.

Forced — a launcher blocks repository work until the session report clears. This is the strongest path and the least elegant one. It is also the one that survives noisy real-world terminal workflows.

4. What Changed in Code

Three concrete moves made this operational.

Session report became a real runtime output.

parser.add_argument(
    "--format",
    choices=["text", "json", "hook", "session-report"],
    default="text"
)

The opening report is now a compiled object — not a protocol expectation, not a prose description. A host can consume it directly.

Invocation is now compiled, not described.

mica_invoke.py compiles read targets and session report into one activation packet:

packet = {
    "mode": mode,
    "entry_strategy": mode,
    "read_targets": _layer_targets(project_root),
    "session_report": report,
}

This is the shift from documentation-first startup to packet-first startup. The host no longer has to infer the sequence from prose.

In guided mode, the output is already shaped for host consumption:

{
  "mode": "guided",
  "entry_strategy": "guided",
  "read_targets": [
    {"name": "readme", "...": "..."},
    {"name": "mica_yaml", "...": "..."},
    {"name": "archive", "...": "..."},
    {"name": "playbook", "...": "..."},
    {"name": "lessons", "...": "..."}
  ],
  "session_report": {
    "archive_version": "1.7.8",
    "self_test": {"pct": "CLOSED", "closed_contract": true},
    "drift_status": {"status": "NO_DRIFT"},
    "gate": "PASS"
  },
  "directive": "Host agent should load declared MICA surfaces first and use the session report as opening state."
}

Forced mode now has consequence.

if args.mode == "forced" and packet["session_report"].get("gate") == "BLOCKED":
    sys.exit(1)

The simplest entry surface:

@echo off
python "%~dp0tools\mica_invoke.py" %*

That wrapper gives MICA an enforceable terminal entrypoint instead of relying on good behavior.

5. STEM-BIO-AI: The Cleaner Case

STEM-BIO-AI already had a mature MICA memory layer — archive, playbook, lessons, invocation protocol, drift profile. What changed was not the memory model. It was how that model becomes operative before work begins.

That difference is visible across all three invocation modes.

In natural mode, the helper preserves the README-first path and makes the expected read order explicit:

[MICA INVOKE] mode=natural
Gate       : PASS
State      : INVOCATION_MODE
PCT        : CLOSED
...
Directive: Prefer reading README first, then load mica.yaml, archive, and playbook before scan work.

In guided mode, the same startup becomes a host-consumable packet:

{
  "mode": "guided",
  "read_targets": ["readme", "mica_yaml", "archive", "playbook", "lessons"],
  "session_report": {
    "archive_version": "1.7.8",
    "self_test": {"pct": "CLOSED", "closed_contract": true},
    "drift_status": {"status": "NO_DRIFT"},
    "gate": "PASS"
  }
}

In forced mode, the launcher uses the same contract as a gate:

[MICA INVOKE] mode=forced
Gate       : PASS
State      : INVOCATION_MODE
PCT        : CLOSED
...
Directive: Block work until the session report gate is not BLOCKED.

The session report now looks like this:

[SESSION READY]
Archive: 1.7.8
Load: {"state": "INVOCATION_MODE", "mica_yaml": "memory\\mica.yaml"}
Self-test: {"pct": "CLOSED", "closed_contract": true}
Drift: {"status": "NO_DRIFT"}
Active invariants: {"critical_count": 15, "high_count": 3}
Gate: PASS

Before, the package told the operator how to start correctly. Now, the session declares whether it actually did.

Before this version, starting a STEM-BIO-AI session correctly still depended on the operator remembering to load the right memory surfaces in the right order. Now that dependency can move upward: in guided mode to the host, and in forced mode to the launcher.

6. CCGE: The Harder Case

CCGE is more important precisely because it is harder. It is already a governance-heavy runtime. If MICA's identity were weak, it would disappear into the larger framework.

CCGE here is the Care Chain Governance Engine: a fail-closed clinical governance runtime with its own execution core, artifact generation, policy layers, and approval logic. That is why it is the harder case. MICA is not being tested in isolation. It is being tested inside a system dense enough to swallow it.

It did not.

The boundary stayed explicit:

MICA = invocation, memory, invariants, drift control
CCGE Core = fail-closed runtime and artifact generation
STEM-AI = trust re-audit and classification

That is the important architectural result. In STEM-BIO-AI, MICA is already close to the center of the tool's operational identity. In CCGE, MICA has to retain its own identity inside a much larger runtime. It does so by remaining responsible for invocation, memory, invariants, and drift control, while CCGE Core remains responsible for fail-closed execution and artifact logic.

The current session report in CCGE:

[SESSION READY]
Archive: None
Load: {"state": "INVOCATION_MODE", "mica_yaml": "mica.yaml"}
Self-test: {"pct": "CLOSED", "closed_contract": true}
Drift: {"status": "NO_DRIFT"}
Active invariants: {"critical_count": 0, "high_count": 0}
Gate: PASS

Archive: None with Gate: PASS is not a contradiction. The baseline archive does not yet expose a project.version field. MICA detected that gap and reported it before any work began. A system that hides its own incompleteness is not governed. A system that surfaces it at session start is.

The reason is concrete: the active archive is still a baseline integration memory object, not yet a fully target-bound archive. Its project block still carries placeholders like:

"project": {
  "name": "<target-repo-name>",
  "path": "<absolute-or-repo-relative-path>",
  "owner": "<org-or-maintainer>",
  "integration_program": "CCGE Unified Model",
  "target_status": "phase_1_candidate"
}

So the current report is telling the truth about what exists: a coherent MICA package around a still-baseline archive.

A README might have let that gap stay invisible. The session report surfaced it immediately. That is what honest governance looks like before an archive is fully populated.

7. What This Means for Anyone Building Agent Workflows

Three lessons from running this against two different projects.

Human-readable startup is not enough. If the only valid path lives in a README, the protocol is vulnerable to partial reading and host variance. STEM-BIO-AI is the clean example here: the memory layer was already mature, but correct startup still depended too much on the operator remembering to load it.

Session-start state must be machine-usable. If a host agent cannot consume the startup declaration as a structured object, it cannot reliably preflight the session. That is why guided mode matters more than another explanatory document: it gives the host an object to act on, not just instructions to interpret.

A gate needs an entrypoint. A session report can be a conceptual hard gate, but until a launcher or host uses it as an entry condition, it remains a convention. CCGE is the stronger proof of that point because the environment is already dense with governance logic; without an explicit entry surface, MICA would have been easy to blur into the surrounding framework instead of remaining its own startup layer.

8. What This Does Not Claim

MICA does not self-invoke automatically in all environments. There is still no natural law that forces an LLM session to load the governed archive first.

The real claim is narrower:

MICA can now be read naturally
MICA can now be requested deliberately
MICA can now be enforced mechanically

Not total automation. A realistic path to enforceable startup.

9. What Part 8 Will Address

The startup path is now much stronger.

But one question remains:

How much of the session-start contract should be owned by the archive itself, and how much should remain a runtime default?

The current line can emit session-report, compile guided packets, and block in forced mode. The next step is stricter archive ownership — richer session_report_format, explicit per-archive session_gate_policy, better drift contracts.

Part 8 is not about whether MICA should govern startup. It already does. It is about how much of that behavior should be declared by the archive rather than inferred by the runtime.

The series continues only where there is something concrete to specify, test, or correct.

Named decision from this post: A protocol is not yet an entrypoint. MICA becomes operational only when invocation is structured as natural, guided, or forced — and the session begins from a declared activation packet, not from hope.

MICA is part of the Flamehaven governance-first AI systems practice. Schema, technical report, and production instance: flamehaven.space. Open-source tooling: AI-SLOP-Detector. All schema references follow the v0.1.8.1 Universal standard unless a specific earlier version is named.

From Repo Scanner to Audit Architecture: What Changed in STEM BIO-AI Through v1.7.8

Kwansub Yun — Tue, 19 May 2026 14:38:53 +0000

Three technical changes that made the scanner less Python-shaped, the warning model more stable, and the reports more inspectable.

The last time I wrote about STEM BIO-AI, the focus was AIRI:

how a local repository scanner could expand its risk vocabulary without pretending to become a universal AI safety judge.

That was the right story for 1.7.0 and 1.7.1.

But the project changed meaningfully after that.

For readers who have not followed the earlier posts: Beyond Repo Scanning: How AIRI Expanded the Risk Vocabulary in STEM BIO-AI 1.7.x

By 1.7.8, the interesting question was no longer just:

Can this scanner attach a broader risk language to local findings?

It became:

Can this scanner make those findings more inspectable, less misleading, and more robust across real repository shapes?

That shift matters.

Because in audit tooling, correctness is only the first battle. The second battle is whether a reviewer can see why the tool landed where it did, and whether the output still makes sense when it leaves the terminal and becomes a report, a PDF packet, a Hugging Face demo, or a governance memo.

From 1.7.6 through 1.7.8, three changes mattered most.

They changed:

what counts as evidence,
how warning lanes are separated,
and how the final artifact stays legible across surfaces.

This is the more technical story behind those releases.

Basic AIRI(the AI Risk Repository) Context: Expanding the Language of Risk

Before getting into the release details, it helps to define what AIRI means in this series.

AIRI refers here to the MIT AI Risk Repository: a public AI risk resource from the MIT AI Risk Initiative that organizes fragmented AI risk language across research, policy, and industry sources.

The repository includes an AI Risk Database, a Causal Taxonomy of AI Risks, and a Domain Taxonomy of AI Risks. According to the MIT AI Risk Repository site, the database collects 1,700+ risks from 74 existing AI risk frameworks and classifications, while the public domain taxonomy organizes risks into 7 domains and 24 subdomains.

That makes AIRI useful as a vocabulary source.

But vocabulary is not truth.

A local scanner should not say:

this repository caused this risk.

It should say something more careful:

this local finding belongs to a broader class of AI risk language.

That distinction is the design boundary.

1. Problem: The scanner was still too Python-shaped

One of the more useful failures in this line came from an uncomfortable result: a repository could obviously have dependency and lockfile evidence, and STEM BIO-AI could still miss it.

That is not a philosophical problem.
That is an implementation problem.

In practice, the project was still too biased toward Python-native signals.

That showed up most clearly in JavaScript or mixed-stack repositories:

package.json
package-lock.json
pnpm-lock.yaml
yarn.lock
npm-shrinkwrap.json

were not being treated as first-class provenance and replication evidence in the same way that requirements.txt or pyproject.toml were.

The result was a false negative pattern:

Stage 3 provenance (B1) could be undercounted
Stage 4 replication evidence could be undercounted
and the report could quietly imply "no dependency evidence" when the repository clearly had dependency structure

That kind of miss is more dangerous than it sounds.

Not because it makes the score a little wrong.

But because it damages trust in the scanner's worldview.

If developers see a tool miss an obvious pnpm-lock.yaml, they stop believing the harder claims too.

What changed in `1.7.6`

The fix was straightforward but important:

JavaScript manifests and lockfiles were promoted into the same evidence families as the existing Python manifests where appropriate.

Concretely, that meant:

B1_data_provenance_controls started recognizing JS manifest/lock surfaces
S4_environment_lock_evidence started recognizing them
S4_exact_dependency_pins_or_hashes started recognizing them

This was not a scoring philosophy change.

It was a scope correction.

The rule engine learned that a dependency ecosystem is a dependency ecosystem even when it is not Python.

One boundary matters here.

B1_data_provenance_controls does not suddenly mean "dataset lineage was proven by a lockfile."

In this lane, B1 is using dependency manifests as repository provenance surfaces:

what environment the repository expects,
what dependency custody the repository exposes,
and whether the repo surfaces any adjacent data-source, IRB, or dataset-citation language around that environment.

That is weaker than dataset lineage evidence.

But it is also much stronger than pretending a mixed-stack repository has no provenance surface at all.

A small before/after that makes the point

The yorkeccak/bio case is a good example because the score movement was not philosophical. It was mechanical.

Before the JS manifest fix, the same repository could produce:

version: 1.7.5
final_score: 40
stage_3_code_bio: 6
B1_data_provenance_controls: 0 / 15
replication_score: 10
AIRI covered_count: 0 / 31

After the manifest and lockfile correction, the same repository shape produced:

version: 1.7.8
final_score: 48
stage_3_code_bio: 25
B1_data_provenance_controls: 15 / 15
replication_score: 30
AIRI covered_count: 7 / 32

The important part is not the score delta by itself.

One small boundary is worth making explicit here.

The AIRI change is doing two things at once:

the denominator moved from 31 to 32 because the governed AIRI detector-scope expanded by one mapping row across this release line,
and the numerator moved from 0 to 7 because the current release can now carry more bounded AIRI links around the findings it actually surfaced.

That explains the AIRI coverage delta.

The scoring delta came from a more mechanical correction:

package.json, package-lock.json, and pnpm-lock.yaml stopped being invisible,
Stage 3 stopped saying "no dependency/provenance manifest detected,"
and Stage 4 stopped undercounting replication structure that was obviously there.

That is what I mean by "blind spot removal" rather than score drift.

Why that matters

This is the kind of change that sounds small in a changelog but large in practice.

Because it changes the relationship between the tool and the developer reading it.

A scanner earns the right to say "this repo is weak on provenance" only after it can correctly see the basic surfaces that exist in the target stack.

That correction also made later report outputs more believable.

When B1 moved from 0 to 15 in affected repositories, that was not "score drift." It was the removal of a blind spot.

And that distinction is exactly why audit tools need explicit versioned rationale.

Without it, every score movement looks arbitrary.

2. Problem: The warning lanes were doing too many jobs at once

Before the split, it helps to read C1–C6 as code-integrity lanes.

They are not general AI risk categories. They are reviewer-facing signals that tell you what kind of repository weakness the scanner found, and where to inspect next.

Lane	What it means in STEM BIO-AI	What a reviewer should inspect
`C1`	Hardcoded credential signals	exposed API keys, cloud keys, tokens, or credential-like patterns
`C2`	Dependency pinning and external-service fragility	loose dependency ranges, missing exact pins, fragile external service assumptions
`C3`	Deprecated patient-adjacent paths	legacy, archive, or deprecated folders that still contain patient or clinical-adjacent patterns
`C4`	Fail-open exception handling	`except: pass`, `except Exception: pass`, silent fallbacks, or code paths where errors can disappear
`C5`	Compliance and clinical-boundary integrity	unsupported HIPAA, compliance, clinical-safe, self-hosted, or regulatory-adjacent claims
`C6`	Mock-auth or no-auth local/self-host trust boundaries	auto-login, mock authentication, no-auth flows, or weak local trust-boundary assumptions

That table matters because C4, C5, and C6 are not interchangeable.

A fail-open exception is not the same problem as an unsupported compliance claim.

And an unsupported compliance claim is not the same problem as a mock-auth self-host boundary.

That distinction became important once the report started surfacing more nuanced governance signals.

The old C4 lane had started life as a code-oriented fail-open/exception surface.

But as the scanner got better at spotting unsupported compliance language and boundary failures, more and more signals were being interpreted near that same lane.

That made the result harder to read.

If a reviewer sees:

C4_exception_handling_clinical_adjacent_paths: WARN

they should be able to infer the remediation class immediately.

They should know to inspect executable control flow.

They should not have to wonder whether the warning is actually about a README compliance claim, a missing clinical boundary, or a mock-auth local path.

Once one lane starts carrying all of those meanings, the ID stops doing its job.

This is a common failure mode in rule systems.

At first it feels efficient:

one warning lane,
one bucket,
multiple related issues.

Then a few releases later the bucket becomes a junk drawer.

That is exactly what had to be prevented here.

What changed in `1.7.7` and `1.7.8`

The solution was to split the lane cleanly:

C4 stayed reserved for executable fail-open exception behavior
C5 was introduced for unsupported compliance or boundary-integrity claims
C6 was introduced for mock-auth, auto-login, or no-auth self-host/local trust-boundary signals

This was more than renaming.

It made the model of the problem cleaner:

C4 is code-path failure semantics
C5 is governance/claim integrity
C6 is trust-boundary collapse in local or self-host flows

That distinction matters to developers because those are different remediation classes.

If a repository triggers C4, you inspect executable control flow.
If it triggers C5, you inspect public claim surfaces and supporting governance evidence.
If it triggers C6, you inspect local auth and trust-boundary design.

One warning label should not try to be all three.

The more interesting case is when two of those lanes fire together.

A repository can claim something like "HIPAA-ready self-hosting" at the README layer and also expose a mock-auth or auto-login local path.

That is not one problem.

It is two related problems:

C5 says the claim surface is overstating governance integrity
C6 says the local trust boundary is weaker than the claim suggests

That is exactly why the split matters.

If those two findings collapse into one bucket, the reviewer loses both remediation clarity and causal ordering.

If they stay separate, the report can say:

the public claim is weak,
the local boundary is weak,
and both together make the repository easier to over-trust.

The code insight

This is one of those places where good audit tooling starts looking more like good static analysis design.

A useful warning family is not just one that catches things.

It is one that stays semantically stable across releases.

That is why this split mattered:

it was not just about improving recall.

It was about preserving interpretability under growth.

Once a detector ID becomes ambiguous, your historical comparisons become weaker.

And once historical comparisons become weaker, your audit system starts losing its memory.

That is a bigger problem than one missed warning.

3. Problem: The report could still be correct and yet hard to trust

A repository scanner does not end its life in JSON.

It ends up in:

Markdown
HTML
PDF
demos
governance reviews
screenshots
and social arguments

That means the output architecture matters almost as much as the scoring logic.

And there were two places where this became obvious.

First: AIRI numbers needed explanation, not just display

Earlier versions could show AIRI coverage as a count, but not always make it obvious why a covered risk appeared.

That is a problem.

Because a number like 7 / 32 looks precise.

But precision without causal explanation is fragile.

Developers do not just want to know that a risk mapped.
They want to know:

which detector triggered it,
why that detector maps to that AIRI risk,
and what boundary still remains around that mapping.

So the AIRI layer had to become more explicit.

That is where mapping_details mattered.

Covered AIRI rows now carry bounded reasoning objects that can say, in effect:

detector ID
mapping justification
trigger reason

That is a much stronger artifact than a bare coverage count.

It turns AIRI from a visual add-on into an inspectable vocabulary layer.

In practice the object now looks more like this:

{
  "id": "24.01.03",
  "title": "Safe exploration problem with widely deployed AI assistants",
  "covered_by": ["C5_compliance_boundary_integrity"],
  "mapping_details": [
    {
      "detector_id": "C5_compliance_boundary_integrity",
      "mapping_justification": "Weak compliance and clinical-boundary integrity can cause users to over-trust unsafe exploration in clinical-adjacent contexts.",
      "trigger_reason": "Unsupported legal/compliance claim surfaced in boundary-integrity lane."
    }
  ]
}

That matters because the AIRI layer no longer asks the reviewer to trust a number alone.

It now gives the reviewer a bounded reasoning object to inspect.

Second: The packets themselves needed re-architecture

The PDF tiers had also drifted into an awkward shape.

The old packet boundaries were no longer matching the actual content density:

Stage 4 could disappear or feel collapsed
the closeout pages could become overcrowded
and "5-page detailed packet" could stop meaning what users expected

That led to a cleaner packet model:

level 1 = brief 1p
level 2 = standard 5p
level 3 = full 7p

And just as importantly:

the default CLI path moved to level 3

It is a statement about what the project now considers the normal artifact.

The normal artifact is no longer the brief scan.
It is the full evidence packet.

Why that matters

This is where the project moved from "scanner" toward "audit architecture."

A scanner can stop at a result.

An audit architecture has to preserve meaning across surfaces.

That means:

JSON must be canonical
HTML must be navigable
PDFs must honor real packet boundaries
and the same warning semantics must survive in all of them

That is why these changes matter to developers.

They are part of the correctness story.

If the why disappears when the result becomes a report, the audit object was never complete to begin with.

The hidden pattern behind all three changes

These releases can look like a mixed bag:

JS manifest support
legal/compliance claim surfacing
external dependency risk
C4/C5/C6 split
AIRI reasoning
packet restructuring
demo/output alignment

But there is a single pattern underneath them:

the system became less willing to let ambiguity hide inside a convenient surface.

That showed up in three ways:

a manifest should count if it exists
a warning lane should mean one thing
a risk mapping should explain itself

That may sound almost obvious.

But a lot of tools never make it that far.

They accumulate clever features faster than they reduce ambiguity.

This line of work did the opposite.

It made *the system stricter about what its outputs are allowed to imply.
*
That is a more durable path.

The more interesting lesson

The most useful thing about 1.7.6 through 1.7.8 is not that STEM BIO-AI became "smarter."

It is that it became harder to misread.

That is a better goal for audit tooling.

Especially now.

Because in a world increasingly full of fluent agent outputs, the differentiator is not whether a tool can generate a plausible narrative.

It is whether the narrative stays tethered to inspectable structure when the repository is messy, cross-stack, overclaimed, or partially misleading.

That is where this release line got better.

Not by pretending to know more than it does.

But by making its own boundaries clearer.

What I would tell developers evaluating this line

If you only look at the release notes, you might think:

better AIRI
more warnings
nicer reports

That is true, but too shallow.

The real changes are:

the scanner is less Python-centric than it was
the warning taxonomy is more semantically stable than it was
the artifacts are more inspectable than they were

That combination matters more than any one score change.

It means the tool is becoming less of a clever repo grader and more of a reliable evidence instrument.

That is the direction I care about.

Because once the repository is politically messy, clinically adjacent, or governance-sensitive, "good-enough automation" is not enough.

The system has to show its work.

These versions got noticeably better at doing that.

Try It

pip install stem-ai
stem /path/to/repo

If you want the full packet explicitly:

stem scan /path/to/repo --level 3 --format all --explain

The default path now lands on the full evidence packet, and that is the point.

In audit tooling, the serious path should not require an extra flag.

See the Artifact

If you want to inspect the actual artifact shape behind this release line, these two public outputs are the best reference:

Interactive HTML report: Open interactive HTML report
Full 7p PDF packet: Open full 7p PDF packet

The point of 1.7.8 is not just that the scanner scores the repository differently.

It is that the same result now survives translation into JSON, Markdown, HTML, and a full review packet without losing too much meaning along the way.

Beyond Repo Scanning: How AIRI Expanded the Risk Vocabulary in STEM BIO-AI 1.7.x

Kwansub Yun — Thu, 14 May 2026 13:41:43 +0000

This is the second half of the same 1.7.x transition.

In the previous post, I wrote about calibration governance: how STEM BIO-AI keeps score authority from drifting when users simulate policy posture.

That was about how the system decides.

This post is about a different layer:

how the system speaks about risk.

A local repository scanner can become trapped inside its own vocabulary.

It can detect dependency issues, weak provenance language, shallow validation, reproducibility gaps, and risky exception handling.

But if every finding stays only inside the scanner's internal language, the report may remain too narrow.

That is the problem AIRI helped address in STEM BIO-AI 1.7.x.

In this context, AIRI is used as a local risk-vocabulary layer built from the MIT AI Risk Repository ecosystem.

The point is not to replace deterministic repository scanning with an external risk database.

The point is to give local findings a broader risk vocabulary without turning that vocabulary into a truth claim.

Basic AIRI Context

The MIT AI Risk Repository is a public AI risk resource from the MIT AI Risk Initiative.

It helps organize fragmented AI risk language across research, policy, and industry sources.

The repository includes three main parts:

an AI Risk Database
a Causal Taxonomy of AI Risks
a Domain Taxonomy of AI Risks

According to the MIT AI Risk Repository site, the database collects 1,700+ risks from 74 existing AI risk frameworks and classifications. The public domain taxonomy organizes risks into 7 domains and 24 subdomains.

Some of those domain taxonomy nodes include:

2. Privacy & Security
2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information
2.2 AI system security vulnerabilities and attacks
6.5 Governance failure
7. AI System Safety, Failures, & Limitations
7.3 Lack of capability or robustness
7.4 Lack of transparency or interpretability

That makes AIRI useful as a vocabulary source.

But vocabulary is not truth.

A local scanner should not say:

this repository caused this risk.

It should say something more careful:

this local finding belongs to a broader class of AI risk language.

That distinction is the design boundary.

What Problem AIRI Was Meant to Solve

STEM BIO-AI began as a deterministic evidence-surface scanner for bio and medical AI repositories.

That core remains.

The scanner looks at observable repository surfaces:

README and docs
code structure
CI configuration
dependency manifests
changelogs
reproducibility signals
clinical-adjacent boundary language

But once STEM BIO-AI started producing richer audit outputs, a new question appeared:

How should the system talk about the broader risk territory around a detected finding?

For example:

a fail-open exception path may have implications beyond code quality
weak provenance language may connect to reproducibility and trust concerns
shallow validation around sensitive inputs may point toward a wider harm surface than the repository alone makes obvious

Without a broader vocabulary, those findings remain local and narrow.

AIRI helps widen the vocabulary without making the scanner less deterministic.

A Short Note on Detector Families

In this article, a detector family means a bounded local analysis surface inside STEM BIO-AI.

It does not mean an AI model judging the repository.

Examples include:

code integrity detectors such as hardcoded credential or fail-open exception checks
AST contract detectors such as shallow validator checks
bio diagnostics such as SMILES parser-guard or silent mock fallback checks
provenance and reproducibility evidence surfaces

A detector family produces a local finding.

The AIRI layer does not replace that finding.

It gives the finding a broader vocabulary anchor.

AIRI Does Not Replace the Scan

This boundary matters.

The AIRI layer does not:

validate that a real-world incident happened
prove that a repository causes a given harm
turn a detector hit into a clinical danger claim
replace due diligence or domain review
override the deterministic score

Instead, it gives the system a structured way to say:

what broader risk territory a finding may relate to
which risk vocabulary exists around that class of concern
where known coverage gaps remain

That is why AIRI is a risk-vocabulary layer, not a truth layer.

If a report says something like:

covered risks: 12 / 31

that should not be read as:

the repository is 38% safe

or:

the scanner covers 38% of all AI risk

A better interpretation is:

within the detector scope currently mapped into the curated AIRI runtime layer, this scan triggered findings that connect to these AIRI risk entries.

That is narrower.

It is also more useful.

From External Repository to Local Governance Layer

The AIRI story in STEM BIO-AI changed during 1.7.x.

The initial direction was simple: use AIRI to provide broader risk labels around local findings.

That was useful, but not enough.

If an audit system relies on an external risk source, it needs governance around that source.

So STEM BIO-AI separates AIRI into three local layers:

Local layer	Purpose
`airi_registry_full.v1.json`	normalized full local registry derived from the upstream AIRI snapshot
`airi_runtime_bundle.v1.json`	curated runtime subset used by deterministic scans
`airi_detector_mapping.v1.json`	detector-to-risk mapping registry plus known-gap records

This separation prevents a common mistake:

confusing the full upstream AIRI universe with the smaller curated runtime bundle used by the scanner.

The scanner uses the curated runtime bundle, not the entire upstream AIRI universe.

That keeps runtime outputs deterministic, reviewable, and tied to a known local snapshot.

What “Governed” Means Here

In the current 1.7.5 state of the 1.7.x line, governed does not mean that every mapping has gone through an external review board.

It means something narrower and more concrete:

AIRI data is stored as versioned local artifacts
runtime scan output uses a curated bundle, not the entire upstream universe
detector mappings are separated from the full registry
known gaps are recorded as part of the mapping layer
artifact metadata surfaces AIRI registry, bundle, mapping, snapshot, and license information
changes to registry, runtime bundle, or mapping versions require explicit version bumps

That is the current governance level.

It is not final.

But it is stronger than attaching a risk dataset as an unversioned appendix.

The Curation Logic

This is the part that matters most.

AIRI is broad. STEM BIO-AI is narrow.

STEM BIO-AI does not need every AIRI entry active at runtime. It needs the subset that can be responsibly connected to deterministic repository evidence.

So the runtime bundle is curated by exclusion as much as inclusion.

A risk vocabulary node should stay outside the runtime bundle when:

No local evidence surface exists
The scanner has no repository-level signal that can responsibly connect to that risk.
The mapping would require causal inference
The scanner would have to imply that harm occurred, that users were affected, or that the repository caused a risk.
The risk is too broad for repository-local evidence
Broad societal, geopolitical, or macroeconomic risks may be important in AIRI, but they should not become runtime scan outputs unless a local detector surface can support the mapping.
The mapping would confuse vocabulary with score authority
If a risk label might be read as changing the formal score or certifying danger, it should remain outside the runtime layer until the reporting semantics are clear.

So the runtime bundle is not a summary of all AI risk.

It is the subset of risk vocabulary that the scanner can use responsibly.

Example: Detector Hit to AIRI Domain Vocabulary

A concrete example helps.

Suppose STEM BIO-AI detects a shallow validator around sensitive or clinical-adjacent inputs.

The local finding might be:

CC3_shallow_validator:
validate_* or check_* function uses only length checks without structural validation.

At the repository level, this is a code-contract finding.

It says:

the function appears to validate input
the validation is shallow
the implementation may not enforce the boundary implied by its name

The AIRI layer should not turn that into:

this repository caused privacy harm.

That would be too strong.

A safer mapping uses AIRI as vocabulary:

Local detector surface	Local meaning	AIRI vocabulary anchor
`CC3_shallow_validator`	validation function appears shallower than its name implies	`7.3 Lack of capability or robustness`; possibly `2.1 Compromise of privacy...` if sensitive information handling is in scope
fail-open exception path	code path may silently continue after failure	`7.3 Lack of capability or robustness`
hardcoded credential signal	repository surface suggests exposed secret-like pattern	`2.2 AI system security vulnerabilities and attacks`
weak provenance surface	repository gives weak evidence about data/source traceability	`7.4 Lack of transparency or interpretability`; possibly `6.5 Governance failure`
silent mock fallback	production-like path may fall back to simulated behavior	`7.3 Lack of capability or robustness`; `7.4 Lack of transparency or interpretability`

The mapping does not prove harm.

It tells the reviewer which broader AIRI vocabulary may be relevant to the local finding.

That is the difference between:

this detector proves a risk occurred

and:

this detector finding belongs near this risk-language area.

The second claim is weaker.

It is also the correct claim.

Why Local Provenance Matters

AIRI is external.

That means STEM BIO-AI needs to answer governance questions explicitly:

which upstream snapshot is being used?
which subset is active at runtime?
which risks are included in the curated bundle?
which risks are known gaps?
which detector maps to which AIRI entry?
what version of the mapping is active?

This is why the AIRI work matters.

It is not just adding labels.

It is turning risk vocabulary into a governed local data layer.

In the current governance note, the upstream source is recorded as:

upstream source: https://airisk.mit.edu/
upstream artifact: The AI Risk Repository V4_03
upstream license: MIT
local snapshot date: 2026-04-23

That provenance is not cosmetic.

It allows an audit artifact to say which risk vocabulary it was using when the scan was produced.

What Is Implemented in the Current 1.7.5 State of 1.7.x

The current AIRI layer is implemented, but bounded.

Implemented surfaces include:

AIRI-backed coverage surfaces in scan outputs
local curated runtime bundle
local registry and mapping schemas
detector-to-AIRI mapping layer
known-gap reporting
provenance and bundle/source labeling

In current scan results, airi_risk_coverage is the main artifact surface for this layer.

The public result contract includes AIRI fields such as:

airi_registry_version
airi_bundle_version
airi_mapping_version
airi_bundle_scope
airi_upstream_snapshot_date
airi_upstream_license
total_risks_in_registry
total_risks_in_bundle
total_risks_in_detector_scope
detectors_triggered
covered_risks
covered_count
coverage_rate
known_gaps_in_bundle
known_gaps_outside_bundle

These fields matter because they let a reviewer distinguish three things that are easy to confuse: the upstream AIRI source, the local runtime bundle, and the detector mapping actually used by the scan.

The important part is not only that these fields exist.

The important part is that AIRI usage becomes auditable from the artifact itself.

If two scans use different AIRI snapshots or mappings, that difference should not be hidden.

Coverage Is Not a Safety Percentage

AIRI coverage in STEM BIO-AI is an audit-surface concept, not a safety percentage.

It does not mean:

the repository is safe
the repository is unsafe
the scanner covers all AI risk
the covered percentage is a compliance score

It means:

a local deterministic finding has been mapped to a known risk-vocabulary entry inside the curated AIRI runtime layer.

That is useful because it gives reviewers a wider frame.

But it does not turn local evidence into a global safety claim.

This is the same discipline used elsewhere in STEM BIO-AI:

scoring is not clinical validation
advisory interpretation is not scoring authority
reproducibility evidence is not automatic score authority
AIRI coverage is not a safety percentage

Each layer has a role.

Each layer has a boundary.

What Changed in 1.7.x

The 1.7.x AIRI story is not simply “we added AIRI.”

The actual change was a move from loose risk labeling toward governed local vocabulary.

1.7.0

AIRI V4 integration appeared in scan outputs.

The scanner began producing an airi_risk_coverage section that maps triggered detector findings to AIRI risk IDs, coverage rate, and known gaps.

The same release also introduced Layer 2 AST contract detectors such as CC1, CC2, and CC3, which expanded the local detector surface available for risk-vocabulary mapping.

1.7.1

AIRI became a governed local data layer.

The architecture separated:

full local registry
curated runtime bundle
detector mapping registry

This release also replaced hardcoded AIRI detector mappings and known-gap lists with packaged local registry files.

Runtime outputs began surfacing registry version, bundle version, mapping version, upstream snapshot date, license, attribution note, and split known gaps into known_gaps_in_bundle and known_gaps_outside_bundle.

1.7.2

No major AIRI architecture change.

The important governance point was regression stability: same-target self-scan comparison verified no drift in airi_risk_coverage alongside score, tier, code contract, detector summary, and evidence ledger count.

1.7.3

No major AIRI architecture change.

The release focused on runtime cleanup, stale demo wording, layout stabilization, and output routing.

1.7.4

AIRI presentation became clearer across demo and report outputs.

The release surfaced AIRI summary material more clearly across the Hugging Face overview card and markdown/explain report sections.

1.7.5

No new AIRI data architecture change.

But artifact-level governance improved more broadly through additive evidence-ledger quality fields and audit-freshness metadata.

That matters because AIRI is most useful when it lives inside a report surface that already carries freshness, evidence quality, and provenance signals.

The important change across the line is this:

AIRI moved from attached dataset toward versioned local risk-vocabulary layer.

What This Still Does Not Do

The AIRI layer still does not:

verify real incidents
prove causality
certify repository safety
replace domain review
turn AIRI categories into deterministic truth claims
collapse the full upstream AIRI universe into the runtime scanner

Those are not missing features.

They are the boundaries that keep the layer useful.

Where This Could Go

The next useful direction is not to overload the scanner with external systems.

It is to improve:

registry provenance
bundle governance
mapping confidence
known-gap clarity
artifact-visible mapping metadata
disciplined links to incident-oriented resources

The broader MIT AIRI ecosystem also includes related incident-oriented resources such as the AI Incident Tracker.

That ecosystem is relevant context, but it is not the same thing as current runtime integration in STEM BIO-AI.

A future version may choose to reference incident-oriented resources more explicitly, but deterministic scans should not ingest them casually or blur them with repository-local findings.

A future version should be able to say not only:

this detector maps to this AIRI risk vocabulary area.

But also:

this mapping has this confidence level, this review status, this local evidence family, and this known limitation.

That is the next governance step.

Final Thought

That is the role of AIRI in this release line.

Not truth replacement.

Not safety certification.

Not incident proof.

A governed vocabulary bridge.

Local evidence first.

External vocabulary second.

Explicit provenance always.

References and Acknowledgment

MIT AI Risk Repository: https://airisk.mit.edu/
MIT AI Incident Tracker: https://airisk.mit.edu/ai-incident-tracker
STEM BIO-AI repository: https://github.com/flamehaven01/STEM-BIO-AI

This AIRI-related direction in STEM BIO-AI was informed by broader public AI risk work, including the MIT AI Risk Repository ecosystem.

The framing around AIRI as a broader risk-vocabulary layer, rather than a repository-local truth layer, was also strengthened by public commentary and ecosystem work from people in this space, including Peter Slattery, PhD.

These references informed the vocabulary and governance direction described here. They do not imply endorsement of STEM BIO-AI or responsibility for its implementation choices.

When Control Becomes Authority: Calibration Governance in STEM BIO-AI 1.7.x

Kwansub Yun — Thu, 14 May 2026 05:41:41 +0000

Control slowly becomes authority when nobody marks the boundary.

That is the calibration problem I kept running into while building STEM BIO-AI.

At first, STEM BIO-AI was centered on the score. It scanned a local bio or medical AI repository, inspected observable repository surfaces, and mapped the repository to a structured review tier.

That was useful.

But it was not enough.

The harder problem was not producing a number. The harder problem was preventing every useful adjacent signal from becoming part of that number.

In a bio/medical AI repository review system, several lanes can look similar if the tool is not careful:

deterministic scoring
diagnostic findings
replication evidence
advisory interpretation
domain-specific review posture

They all matter.

But they should not all have the same authority.

That is the core reason calibration became a governance problem in the 1.7.x line.

The principle is simple:

easy experimentation, hard drift
STEM BIO-AI should let researchers express review posture. It should let operators simulate policy changes. It should make policy metadata visible in artifacts.

But it should not let those inputs silently mutate the official score.

A Short Context for New Readers

STEM BIO-AI is a deterministic evidence-surface scanner for bio and medical AI repositories.

It does not validate biomedical efficacy. It does not certify clinical safety. It does not prove that a model is correct.

It scans observable repository surfaces such as:

README and docs
code structure
CI configuration
dependency manifests
changelogs
evidence and boundary language

The formal score is currently built from three weighted score-bearing stages, plus an explicit credential penalty and clinical cap or hard-floor logic:

Stage	Role
Stage 1	README / stated evidence boundary
Stage 2R	repo-local consistency
Stage 3	code and bio-responsibility surface

The active formula still also applies:

C1_penalty when hardcoded credentials are detected
score_cap or t0_hard_floor when clinical-adjacent boundary rules require it

Stage 4 exists, but it is a separate replication lane. It reports reproducibility and replication posture without automatically changing the formal score.

That separation is intentional.

What Is Actually Implemented in the Current 1.7.5 State of 1.7.x

Before discussing calibration philosophy, the implementation boundary has to be clear.

In the current 1.7.5 state of the 1.7.x line, STEM BIO-AI has implemented a real calibration architecture, but it is still mostly a mirror-only and preview-oriented architecture.

This post describes the current released state of the 1.7.x line as of v1.7.5, not a future authoritative-read-through design.

Implemented surfaces include:

packaged calibration profiles
schema and runtime validation
profile identity surfaced in result metadata
stem policy list
stem policy explain
stem policy derive
stem policy simulate
simulation-only local profile files
profile hashes and read-mode metadata in artifacts

The current named recommendation surface is intentionally narrow:

default
strict_clinical_adjacency

reproducibility_first is still a draft posture, not an active release-grade named recommendation.

The important limitation is this:

the authoritative scan scoring path is still protected from arbitrary user-provided profile mutation.

In other words, scan --policy <name> can surface selected profile metadata. policy derive and policy simulate can show governed preview behavior. But user-provided profile files do not simply become the official scoring authority.

More specifically, local profile files are currently accepted only by stem policy simulate, and the CLI rejects them unless the file remains mirror_only.

That is not a missing convenience.

That is the boundary being tested before it is allowed to become authority.

The Pressure That Causes Drift

One question pushed this design forward:

*If advisory AI becomes more capable, will teams really keep the boundary between formal score and advisory interpretation?
*
I do not think the answer is automatically yes.

If an advisory layer becomes helpful, there will always be pressure to let it influence the formal score "just a little."

That is usually how audit systems drift.

The score stops being a stable artifact and starts becoming a moving interpretation layer.

The danger is not that users want control.

The danger is that control slowly becomes authority without anyone noticing.

So the design question is not:

How do we let people tune the system more freely?

The design question is:

How do we let people express domain judgment without making the formal score easy to mutate?

That is where calibration enters.

Calibration Is Not a Tuning Console

The wrong calibration UX looks like this:

{
  "stage_1_percent": 30,
  "stage_2r_percent": 25,
  "stage_3_percent": 45,
  "ca_no_disclaimer_cap": 61,
  "b2_partial_credit_mode": "looser"
}

This is editable.

But editable is not the same as governed.

Most researchers, operators, and domain reviewers do not think in raw score constants. They usually know something closer to this:

clinical-adjacent claims should be treated very strictly
reproducibility matters strongly in this environment
README polish should not outweigh code evidence
a casual mention of "limitations" should not count as meaningful transparency

That is why the current calibration design starts with posture questions, not raw constants.

The goal is not to ask a researcher to become a scoring-engine maintainer.

The goal is to let a researcher express domain posture while keeping the formal scoring boundary visible, versioned, and difficult to mutate accidentally.

The `1–5` Scale Is Input, Not Authority

In the current design, the user-facing intent layer uses a 1–5 scale:

1 = minimal emphasis
2 = light emphasis
3 = moderate emphasis
4 = strong emphasis
5 = very strong emphasis

The important line is this:

the 1–5 scale is a UX input surface, not part of the formal score engine.

That means the user can express posture in a natural way:

clinical strictness
code-integrity priority
reproducibility priority
structured limitations requirement

But those answers do not directly become score constants.

They are translated through explicit rules.

The current decision table is intentionally narrow:

Condition	Outcome
`clinical_strictness >= 4` and `reproducibility_priority <= 3`	recommend `strict_clinical_adjacency`
all four values are `2` or `3`	keep `default`
no named-profile rule matches	generate a `preview_only` profile delta from bounded deltas only

This table should not be mistaken for an empirically optimized model.

It is a conservative governance rule table.

The current threshold choices are design-steward decisions, not claims of statistical optimality. Their purpose is to keep the translation layer narrow, reviewable, and non-authoritative until a stronger benchmark-backed promotion process exists.

That matters because a calibration system can fail in two opposite ways:

it can be too rigid for domain experts to use
it can be so flexible that every local preference becomes a new score

The initial rule table chooses the safer failure mode.

If a posture is clearly within an existing release-grade profile, the system can recommend that profile. If the posture is ambiguous or combines competing priorities, the system falls back to preview_only.

For example:

clinical_strictness = 4
reproducibility_priority = 4

That does not automatically recommend strict_clinical_adjacency.

It falls back to preview_only, because two strong postures are competing and no release-grade named profile currently resolves that conflict.

A hidden similarity function might produce something that looks more flexible.

But it would also make the governance harder to audit.

A narrow rule table is less magical.

It is also safer.

What the CLI Is Allowed to Do

![Easy experimentation, hard drift — sandbox and vault

The preview workflow can look like this:

stem policy derive \
  --clinical-strictness 5 \
  --code-integrity-priority 4 \
  --reproducibility-priority 3 \
  --structured-limitations-requirement 4

or this:

stem policy simulate /path/to/repo --profile-file my_profile.json

But those flows are not the same as saying:

stem scan /path/to/repo --stage1-weight 0.35 --cap 72

The first two are governed preview surfaces.

The last one is an untracked tuning console.

The design intentionally supports the first and rejects the shape of the last.

This is the practical meaning of easy experimentation, hard drift.

What Actually Gets Verified

The central claim of this design is not:

the current calibration rules are perfect.

The claim is narrower:

calibration changes should not become score authority without a visible governance path.

That claim can be tested by checking whether the system exposes or blocks the relevant control surfaces.

Drift risk	Expected control	How to verify it
arbitrary score tuning	no free-form CLI weight / cap override	CLI help and accepted options do not expose direct score constants
hidden profile mutation	profile status and read mode are surfaced	result artifacts expose profile metadata
unclear profile identity	profile name, version, and hash are visible	scan output includes calibration profile identity
advisory influence leakage	advisory output cannot override score	advisory response validation cannot mutate `final_score`
reproducibility overcompensation	Stage 4 remains separate	`replication_score` does not change `formal_tier`
premature named-profile expansion	ambiguous postures fall back to preview	derive/simulate returns `preview_only` when no named rule matches
detector promotion drift	evidence-only detectors are not score-authoritative	detector policy is versioned in policy files and governance docs, even though per-detector score-integration status is not yet surfaced as first-class artifact metadata

This is still not the same as a full empirical benchmark.

But it is a real verification target.

The system can be checked for whether it allows the forbidden mutation path.

That is the level of proof appropriate for this release line: not "the final policy is optimal," but "the policy cannot quietly become authoritative without leaving a trace."

That trace is stronger for some surfaces than others. Profile identity, hash, and read mode are already artifact-visible in 1.7.5. Detector promotion semantics are already versioned and documented, but they are not yet surfaced as first-class per-detector policy metadata in the result object.

The B2 Tightening Example

The clearest scoring example is Stage 3 B2.

B2 is the bias and limitations measurement surface. Earlier scoring behavior allowed a weaker boundary: a simple vocabulary-level signal could still receive partial credit.

That became too permissive.

A repository that mentions "bias" or "limitations" once is not necessarily disclosing a meaningful boundary. It may only be surface signaling.

So the B2 rule became stricter.

The important change is not a marketing claim about benchmark improvement. The important change is a deterministic boundary change:

Case	Earlier posture	Tightened posture
no bias / limitations vocabulary	0	0
minimal single-term mention only	partial credit possible	0
structured limitations language	partial credit possible	partial credit possible
quantitative measurement evidence	full credit possible	full credit possible

This is the first place where calibration becomes visible as more than a principle.

The rule change creates a concrete score path difference:

a repository that previously depended only on a minimal single-term limitations mention no longer has a B2 partial-credit path after the tightening.

That is the current public claim.

I am not presenting a benchmark-wide before/after score delta here, because that would require a pinned fixture set and published comparison protocol.

Without that, a claimed "T3 became T2" example would be anecdotal at best and misleading at worst.

So the honest evidence level is rule-level impact:

the credit path changed
the changed path is deterministic
the changed path is inspectable
benchmark-level deltas should be published only when the fixture protocol is pinned

In clinical-adjacent repositories, limitation language is not decoration. It is part of the claim boundary.

A one-word mention does not carry the same weight as a structured limitations section, demographic coverage statement, known failure-mode description, or quantitative subgroup analysis.

This is why calibration cannot be only a UI problem.

If a user asks for a stricter limitations posture, the system should not silently subtract points through a hidden override. It should expose the rule that changed and the reason that rule exists.

That is the difference between a score tweak and a governed scoring rationale.

Why Stage 4 Stays Separate

Stage 4 is the place where the strongest counterargument appears.

The counterargument is fair:

If reproducibility is important, why does it not affect the formal score?

My answer is that importance and score authority are not the same thing.

Stage 4 measures replication posture: containers, reproducibility targets, dependency locks, artifact references, seeds, citation surfaces, and similar evidence.

Those signals matter.

But they do not mean the same thing as the formal claim boundary.

A repository can be highly reproducible and still make unsafe or unbounded clinical claims.

A repository can have clean containers and dependency locks while still lacking a clinical-use disclaimer.

A repository can be easy to rerun while still having weak data provenance or shallow limitation language.

If Stage 4 were allowed to lift the formal score too early, reproducibility could start compensating for claim-boundary weakness.

That would be a different scoring philosophy.

It may become valid in the future, but only if the rule is explicit.

For now, Stage 4 is reported as a separate lane because the system is saying:

reproducibility matters
reproducibility should be visible
reproducibility should affect review interpretation
reproducibility should not silently override the formal score boundary

That is why stronger reproducibility intent currently falls back to preview_only instead of becoming a release-grade named profile.

The system is not saying reproducibility is unimportant.

It is saying reproducibility has not yet been granted formal score authority.

Advisory AI Uses the Same Boundary

Advisory AI follows the same rule.

Helpful interpretation is not score authority.

STEM BIO-AI can export provider-neutral advisory packets and validate downstream advisory responses, but the deterministic scanner does not need an external model runtime to produce the formal score.

If an advisory system becomes useful, it may help interpret findings, prioritize review, or explain evidence patterns.

But unless a future release explicitly changes the policy, advisory output remains structurally subordinate to the deterministic score.

That is enough for this article.

The broader advisory boundary is a separate topic.

From Scoring Tool to Audit Workflow

The 1.7.x transition is best understood as a shift in the questions the tool is expected to answer.

Earlier scoring-tool question	Audit-workflow question
What score did the repository get?	Which policy profile was visible when the score was produced?
Which stage contributed most?	Was that stage score-authoritative, diagnostic, or separate-lane evidence?
What evidence triggered the tier?	Did the evidence change the formal score or only the review posture?
What should the user fix?	Would a proposed policy change be preview-only, experimental, benchmark-candidate, or release-authoritative?

This is why I describe 1.7.x as an audit-system transition.

The score still matters.

But the system is increasingly designed around the custody of the score: where it came from, what was allowed to influence it, and what was intentionally kept outside it.

What This Still Does Not Do

This boundary is just as important as the implementation.

STEM BIO-AI still does not:

validate biomedical efficacy
certify benchmark truth
determine clinical deployment safety
let advisory AI overwrite the formal score
open arbitrary numeric tuning in the official scan path
allow profile experimentation to become official policy without governance

Those are not missing conveniences.

They are boundaries.

A strong repository evidence tier is still an observable repository-surface signal. It is not clinical clearance, regulatory approval, or proof of biomedical validity.

The Next Version Direction

The next important step is not adding more knobs.

It is authoritative policy read-through in parity mode.

That means:

the default policy profile becomes the source read by the scoring path
existing fixtures should show no score or tier drift
policy hashes remain visible in artifacts
non-default and researcher-provided profiles remain governed preview surfaces until promoted
score-affecting policy changes become explicit release events

This is not a big-bang rewrite.

It is authority relocation.

The goal is to move score-affecting constants into versioned policy objects without changing the score by accident.

Only after that parity step does it become safe to discuss broader named profiles.

Final Position

The calibration problem is not really about giving users more control.

It is about deciding when control becomes authority.

If every useful signal can gradually influence the score, the score stops being an audit artifact.

It becomes a negotiation.

That is what STEM BIO-AI is trying to avoid.

Researchers should be able to express posture.

Operators should be able to simulate alternatives.

Policy stewards should be able to promote changes.

But the formal score should not move unless the governance path says it moved.

That is the difference between a tuning console and an audit system.

Building a Deterministic Governance Kernel: Separating Custody from Truth

Kwansub Yun — Tue, 12 May 2026 06:00:43 +0000

A governance engine should not pretend to know the truth of every domain.

That was the architectural lesson behind CGF.

At Flamehaven Labs, we build B2B governance engines for highly regulated environments. Over the past year, we developed specialized deterministic systems for different review contexts:

CareChainGovernanceEngine (CCGE): a fail-closed clinical-governance engine for enforcing safety-oriented review gates in bio-AI workflows.
The Analyst's Problem Framework (TAP): a “Proof Custody” engine designed to package and audit mathematical proof candidates.

Both worked inside their own domains. But both also exposed the same architectural problem: reusable custody mechanics were mixed with domain-specific decision semantics.

We needed to audit new targets — such as external open-source intake, RAG retrieval receipts, and AI evolution proposals. If we did not extract a common, domain-neutral kernel, we would be doomed to rewrite the entire scanning, hashing, and reporting pipeline for every new vertical.

The result was the Custody Governance Framework (CGF): a domain-neutral custody kernel for B2B technical review workflows.

This is an architecture extraction note on how we decoupled domain truth from custody mechanics, the API design that powers it, and why we specifically rejected the modern trend of “LLM-agentic” governance.

The Problem with “Agentic” Governance

Many emerging AI governance workflows are becoming document-shaped or agent-shaped: a YAML file, a Markdown policy, or an LLM prompt that says, “check whether this is safe.”

The problem is not that LLMs are useless. The problem is that they can produce compliance-shaped language without producing verifiable compliance artifacts.

In a strict B2B handoff — where auditability, legal review, and future regulatory mapping to frameworks such as the EU AI Act or NIST AI RMF may matter — you cannot rely on non-deterministic evaluations.

CGF takes the opposite approach: strict determinism.

The framework does not own domain-truth semantics. It owns the custody mechanics around findings, profiles, evidence, approvals, and artifacts.

The Architecture: A Deterministic Data Flow

To enforce this separation, we designed CGF as a deterministic pipeline with a narrow side-effect boundary.

The core engine takes a normalized review input and a GovernanceProfile, transforming them into immutable artifact dataclasses. The writer layer then materializes those objects as files, manifests, and release artifacts.

Here is what the end-to-end flow looks like:

The API Boundary

At the core boundary, the framework does not evaluate whether a finding is “bad” by its own logic.

It relies on a StatusDeriver driven by the injected profile.

A simplified sketch of the boundary looks like this:

# Simplified sketch, not the full implementation
@dataclass(slots=True)
class GovernancePipeline:
    profile: GovernanceProfile
    status_deriver: StatusDeriver

    def build_packet(self, result: ScanResult) -> GovernancePacket:
        # 1. Inject profile-specific requirements, such as mandatory surfaces
        governed_result = self._with_profile_requirements(result)

        # 2. Derive deterministic status via the profile
        status_result = self.status_deriver.derive(governed_result)

        # 3. Assemble the immutable packet
        return GovernancePacket(
            target=governed_result.target,
            profile_id=self.profile.profile_id,
            status=status_result.status,
            status_reason=status_result.reason,
            findings=list(governed_result.findings),
            evidence=list(governed_result.evidence),
            compliance_score=_compute_compliance_score(governed_result.findings),
        )

The exact implementation also handles timestamps, approval bridges, validation, artifact writing, and manifest verification.

The important point is architectural: the core does not decide domain truth. It records how a profile interpreted the evidence.

Architecture Highlight 1: Inspectable Artifacts Over Silent Mutation

A major risk in governance automation is the engine silently mutating the target repository, for example by automatically injecting compliance boilerplate.

Many tools solve this with a --dry-run flag that prints logs to stdout.

In a B2B audit, stdout logs are not enough. You need an auditable, verifiable artifact.

CGF implements a preview-first artifact flow. When the pipeline runs, it does not mutate the target repository by default. Instead, it consumes a normalized review input and emits a deterministic custody bundle:

$ cgf run --profile proof_custody.json --scan target_scan.json --out audit/

In real deployments, target_scan.json may be produced by a vertical adapter, repository scanner, RAG receipt processor, or customer-specific intake layer.

The output is an inspectable custody bundle:

audit/
├── governance_packet.json       # Machine-readable audit state
├── preview_report.md            # Human-readable summary
├── chain_ribbon.md              # Markdown tag for custody-chain review state
└── manifest.json                # Artifact manifest with file hashes

The important point is not that CGF edits the repository.

It does not.

The important point is that the review state, findings, proposed next actions, and artifact hashes become inspectable before any external system decides what to do next.

Architecture Highlight 2: The Reality of Audit Chains

In B2B handoffs, customers often ask for tamper-resistance.

CGF supports a GovernanceAuditChain, an append-only JSONL ledger where packet records can be linked through SHA-256 hashes.

But we need to be honest about the tradeoff:

Local hash chains are tamper-evident, not tamper-resistant.

If a bad actor has write access to the filesystem, they can delete the audit directory and regenerate the entire chain from scratch.

CGF does not use a blockchain. The cost and complexity of a distributed ledger would outweigh the benefits for local repository scanning.

Instead, CGF provides local tamper-evidence.

To achieve true tamper-resistance, the deployment environment still matters: CI/CD artifact signing, external timestamping, identity providers, or customer-controlled archival systems.

For example, an external identity token can be wired into an approval bridge:

# Wiring an external identity token into the local chain
bridge = ApprovalBridge(
    state="approved",
    approved_by="compliance_lead_JWT_subject",
    notes="Cryptographic signature from Identity Provider XYZ",
)

approved_packet = GovernancePipeline.apply_approval(packet, bridge)

The framework provides the cryptographic hooks and chronological integrity.

The deployment environment provides the immutability.

Current Limitations

This post is not a release announcement or a regulatory certification claim.

It is an architecture note about the extraction pattern: how we separated reusable governance mechanics from domain-specific truth semantics.

CGF is still early. It is not a compliance platform, not a hosted governance service, and not a regulatory certification product.

That distinction matters.

CGF does not prove that a medical system is safe. It does not prove that a mathematical argument is true. It does not certify legal compliance. It does not replace domain experts, auditors, clinicians, lawyers, or reviewers.

What it does is narrower, but more concrete:

It makes the custody surface inspectable.

There are also practical limitations.

As mentioned, local hash chains are tamper-evident, not tamper-resistant. True immutability still has to come from the deployment environment: CI/CD signing, external timestamping, identity providers, or customer-controlled archival systems.

CGF is also not yet a complete enterprise governance platform. Authentication, RBAC, multi-tenant profile registries, async approval workflows, and regulatory citation mapping are still roadmap items, not solved infrastructure.

Each domain still needs adapters, profiles, thresholds, and human review policies.

The kernel provides the custody mechanics. The domain owner still has to define what evidence matters.

That is intentional.

A generic governance kernel should not pretend to know the truth of every field.

Roadmap

The roadmap is not to turn CGF into a giant all-knowing compliance agent.

The roadmap is to keep the kernel small, deterministic, and inspectable while adding stronger boundaries around the places where real B2B workflows need them.

The next layers are:

Regulatory mapping: mapping finding codes to frameworks such as the EU AI Act, NIST AI RMF, and ISO/IEC 42001 without turning CGF itself into a legal authority.
Approval policy hardening: adding stronger policy checks around ApprovalBridge so approvals can be scoped, expired, and externally verified.
Async approval workflows: allowing human review, compliance sign-off, or customer approval to arrive after the initial custody packet is generated.
Profile registries: supporting versioned, tenant-scoped governance profiles so different customers can use different policies without changing the kernel.
Signed external receipts: allowing RAG systems, technology scanners, quality engines, and external tools to produce receipts that CGF can verify and attach.
Vertical adapters: binding existing domain systems back to the kernel without importing their domain-specific truth semantics into the core.

Every roadmap item has to preserve the same rule:

The kernel may govern custody, but it must not absorb domain truth.

The direction is deliberately conservative: more custody, more verification, more explicit boundaries — not more autonomous magic.

Why This Matters

A lot of AI governance today is still document-shaped.

A policy lives in Markdown. A checklist lives in YAML. A prompt says the system should be safe, transparent, aligned, compliant, or human-reviewed.

Those documents are not useless. They are often necessary.

But they are not governance by themselves.

Governance becomes real only when it has an execution surface:

a typed input boundary
normalized findings
profile-owned status derivation
explicit evidence references
generated review artifacts
manifest hashes
approval metadata
release bundles
verification commands
clear non-goals

That is the difference CGF is trying to make.

It is not another Markdown file describing how governance should work.

It is a deterministic custody pipeline that turns review inputs into inspectable artifacts.

The goal is not to make governance sound more sophisticated.

The goal is to make it harder to fake.

A governance system should leave behind more than confidence.

It should leave behind artifacts.

Conclusion

Extracting the Custody Governance Framework taught us that governance architecture has to separate process from truth.

Truth belongs to domains:

medicine
mathematics
law
security
finance
science

Process belongs to the governance kernel:

what was reviewed
which profile was applied
which findings fired
what evidence was attached
what status was derived
who approved it
whether the resulting artifacts can be verified later

That separation is the reason CGF exists.

It does not try to be an AI judge. It does not ask an LLM to guess whether a system is compliant. It does not hide governance inside a prompt, a policy document, or a YAML file.

It creates custody artifacts that can be inspected.

For us, that is the real boundary between governance as language and governance as infrastructure.

From Score to Workflow: Turning STEM BIO-AI Into a Local Audit System

Kwansub Yun — Fri, 08 May 2026 08:25:50 +0000

Earlier in this series, I wrote about why bio/medical AI repositories need more than benchmarks, what I learned after auditing 10 public repositories, and why an AI auditor itself needs a memory contract.

That work led to STEM-AI v1.1.2 and the MICA layer: a memory-contracted initialization step that forces the auditor to load bounded rules before scoring begins. If you have not read that part, the relevant post is here:

How Do You Trust the AI Auditor? STEM-AI v1.1.2 and Memory-Contracted Bio-AI Audits

For the broader arc, the full series is here:

STEM-AI / STEM BIO-AI series

But after that, a different engineering problem took over.

The audit logic was stricter.

The reports were richer.

The reasoning was more bounded.

But the developer workflow still felt too loose.

So the next question was no longer:

How do I score trust?

It became:

How does a bio-AI audit tool become something an engineer can actually run, gate, inspect, and integrate?

The answer turned out to be less about seeing more signals and more about refusing to confuse them.

That is the core argument of this post:

A detector becomes more trustworthy when it is strict about what it cannot conclude.

Once I took that seriously, STEM BIO-AI stopped looking like “one score plus some extra metadata” and started looking like a system with distinct lanes, distinct boundaries, and distinct operator workflows.

The problem was no longer scoring

By the time I reached the 1.6.x line, the rubric was no longer the main bottleneck.

The bottleneck was operational clarity.

A trust audit tool is not very useful if:

the normal path is one long command with too many flags
CI has to reverse-engineer the result from human-readable stdout
bio-specific diagnostics are mixed directly into the same surface as formal scoring
regulatory relevance shows up as vague implication instead of explicit traceability
advisory AI is present, but its relationship to the official score is unclear

At that point, the tool stops being hard to trust for conceptual reasons and starts being hard to trust for operational reasons.

That is a different class of problem.

The CLI had to reflect operator intent

The earlier CLI was functional, but too flat.

You could do things like this:

stem /path/to/repo --level 3 --format all --explain
stem /path/to/repo --tier-gate T3 --format json --quiet
stem /path/to/repo --advisory packet
stem /path/to/repo --advisory-response provider_advisory.json

All of that worked.

The issue was that it treated very different operator intents as one long option surface.

In practice, these are separate workflows:

scan a repository and generate artifacts
enforce a gate in CI/CD
export a bounded advisory packet
validate a downstream provider response
cross an explicit provider-call boundary

So I refactored the CLI around workflows instead of flag accumulation:

stem scan <folder>
stem gate <folder> --min-tier T2
stem advisory validate <folder>
stem advisory packet <folder>
stem advisory call <folder>
stem advisory check-response <folder> --response FILE

The older paths still exist for compatibility:

stem <folder>
stem audit <folder>
stem <folder> --tier-gate T2 --quiet
stem <folder> --advisory packet

But they are no longer the conceptual center.

That matters more than it sounds.

Once the command names match the operator’s intent, the system becomes easier to teach, easier to remember, and easier to wire into pipelines.

This is not just a DX cleanup. In a medical or bio-adjacent audit context, command ambiguity is part of the trust problem.

Repository trust needed four separate lanes

This was the biggest architectural shift.

I stopped treating repository trust as one object.

In practice, it needed four separate lanes:

deterministic structural scoring
deterministic diagnostics
regulatory traceability
optional AI advisory

If all of those collapse into one final confidence score, the tool becomes harder to reason about.

The more regulated the domain, the more dangerous it becomes to collapse every useful signal into one score.

Some evidence should change the score.
Some evidence should only raise review priority.
Some evidence should support traceability.
Some evidence should be handed to a human or advisory system.

The maturity of the tool is not that it sees all of them.

The maturity is that it does not confuse them.

This separation is not just conceptual. It exists in the code path.

One reasonable objection to any architecture write-up is: are these really separate lanes, or are they just different labels on the same output object?

In STEM BIO-AI, the answer is visible in the execution order.

The scanner computes the formal score first. In the result object, that means keys like:

Stage 1
Stage 2R
Stage 3
risk penalty
score cap
final_score
formal_tier

Only after that does it append the non-scoring layers, again as explicit result keys:

regulatory_basis
stage_traceability
regulatory_traceability
reasoning_model
optional ai_advisory

That ordering matters.

The score is not derived from the advisory lane.
The regulatory mapping does not mutate the formal tier.
The diagnostics lane can emit evidence without becoming a hidden score multiplier.

This is also why the JSON shape ended up more layered than earlier versions. The output had to preserve the distinction the code was already enforcing.

That execution order is the architectural reason the next four sections exist.

Once I had the lanes separated in code, each lane needed its own claim boundary, its own output semantics, and its own reason for not being collapsed into the others.

Put differently, the next four sections answer four different questions:

what is allowed to change the formal tier
what is useful enough to emit, but not yet mature enough to score
what can support regulatory review without pretending to be compliance
what can involve AI without letting AI become the scoring authority

1. Deterministic structural scoring

This remains the official score and tier.

It measures the main repository-visible signals:

README evidence
repo-local consistency
code and bio responsibility
dependency hygiene
changelog and provenance surfaces
code-integrity patterns

This lane is local, deterministic, and machine-checkable.

That is the part that can legitimately drive a formal triage tier.

I am not claiming this is the only possible architecture. A different system could have folded diagnostics or replication more aggressively into one unified score.

I chose not to, because the narrower score proved easier to defend. A smaller claim with cleaner boundaries was more valuable here than a broader score with ambiguous semantics.

2. Deterministic diagnostics

This is where the deterministic diagnostics spec became important.

I needed a place for findings that are real, useful, and inspectable, but should not silently perturb the main score until they are calibrated.

That is what docs/DETERMINISTIC_DIAGNOSTICS.md defines.

It separates the diagnostic problem into two lanes:

Lane A: deterministic local diagnostics
Lane B: optional AI-assisted semantic review

That separation is central.

The deterministic lane is authoritative for hard findings.
The AI lane is advisory only.

The local diagnostic lane currently focuses on evidence-bearing bio-specific signals such as:

malformed or suspicious SMILES-like outputs
missing parser guards
silent mock or simulated-data fallbacks
risky subprocess construction around bio tools
traceability manifest surfaces

The point was not to create a “bio slop detector” with a catchy label.

The point was to create a local evidence lane that could say:

here is the file
here is the line
here is the snippet
here is the bounded interpretation

That is much more useful than a vague semantic warning.

Why diagnostics stayed evidence-only

This was one of the harder engineering decisions.

It would have been easy to push every new bio-specific detector directly into the final score.

I did not do that.

The deterministic diagnostics spec is explicit that many of these findings begin as evidence-only. In practice, they are emitted as line-level records in the result object's evidence_ledger:

findings are emitted into the result object’s evidence_ledger
findings appear in Markdown and --explain
findings do not change final_score or formal_tier

That is the right default.

For example, the SMILES lane can be very useful for detecting:

malformed surface strings
low-entropy placeholders
repeated trivial outputs
missing parser guards

But it does not prove:

medicinal usefulness
synthetic feasibility
binding plausibility
biological efficacy
full chemical validity in every edge case

That boundary is important.

A detector becomes more trustworthy when it is strict about what it cannot conclude.

Just as importantly, this is not meant to be a permanent holding area for every detector. The diagnostics spec is explicit that score impact should only happen after commit-pinned benchmark evidence, explicit false-positive review, and reproducible calibration. In other words, evidence-only is the temporary safe default until a detector has earned score authority.

3. Regulatory traceability

The second document that became central was docs/REGULATORY_MAPPING.md.

This solved a different problem.

Once you audit clinical-adjacent repositories, people naturally ask:

does this align with EU AI Act themes?
does this help with FDA-oriented review?
is there anything relevant to IMDRF or SaMD evidence families?

The wrong answer would be to turn those questions into a fake compliance score.

So I did the opposite.

The regulatory layer is explicitly framed as:

a traceability aid, not a compliance verdict

That document maps observed evidence classes to requirement families with bounded confidence labels like:

strong
moderate
weak-moderate
weak
not assessed

And it makes an important distinction:

the confidence applies to the mapping relationship, not to legal acceptability.

Those confidence labels are not model outputs and they are not inferred at runtime. They are fixed, rule-level mapping judgments attached to evidence classes in the mapping document itself. For example, changelog / checksum / config-manifest style evidence is treated as a moderate traceability signal for Article 12-style review, while human-oversight interface signals stay weak because interface presence is not the same thing as oversight procedure.

That means the tool can say things like:

versioned manifests and changelogs may support record-keeping / traceability review
intended-use and disclaimer sections may support transparency scaffolding review
override interfaces may support human-oversight interface review
subgroup measurement language may support weak evidence of data-governance intent

without claiming:

legal compliance
regulatory clearance
clinical certification
deployer conformance

In a regulated domain, traceability is useful only when it does not pretend to be permission.

A concrete example: why Article 12 is traceability, not compliance

The best example here is EU AI Act Article 12 style traceability.

The regulatory mapping layer treats signals like:

changelogs
checksum manifests
versioned config surfaces
audit-log schema fragments
decision-event or override-event schema tokens

as evidence that a repository may have traceability scaffolding.

That is useful.

It is also bounded.

The mapping document is explicit that changelog presence is not the same thing as deploy-time event logging, and that current scope does not establish runtime log completeness.

So the output can legitimately say:

there is structural evidence relevant to traceability review

while refusing to say:

this system satisfies traceability obligations

That is exactly the kind of distinction I wanted this lane to enforce.

What this buys in practice is not a compliance shortcut, but a faster review question. If a repository exposes none of the scaffolding signals in this lane — no change history, no artifact hashes, no versioned manifests, no event-schema surfaces — then there is very little reason to treat it as traceability-ready for deeper institutional review. If those signals do exist, the next step is still expert inspection, but the scanner has at least opened the right folder and pointed at the right files.

Why regulatory mapping stayed subordinate to evidence

This was non-negotiable.

Regulatory relevance had to remain downstream from evidence, not a score multiplier pretending to be law.

That is why the output shape separates things like:

regulatory_basis
stage_traceability
regulatory_traceability

from the actual score computation.

And it is not just decorative structure.

The regulatory basis object is registry-driven. It can mark review_required when the basis registry is stale or required source families are missing. That is a traceability control on the mapping layer itself, not an input into the scoring formula.

This is also why the regulatory note belongs in a muted traceability panel, not next to the main score.

If a repo has traceability-relevant scaffolding, that is useful.

If a repo has traceability-relevant scaffolding, that is still not compliance.

The distinction has to remain visible in both the code and the artifacts.

4. Optional AI advisory

The fourth lane is the advisory layer.

This one exists for bounded model-assisted review, but it does not get to rewrite the official outcome.

That means workflows like:

stem advisory packet /path/to/repo
stem advisory check-response /path/to/repo --response provider_advisory.json

can exist without creating ambiguity about who owns the formal result.

The advisory layer can:

export a provider-neutral packet
validate downstream response structure
enforce finding-ID citation rules
reject prohibited claims
surface runtime and secret boundaries

What it cannot do is silently override:

score.final_score
score.formal_tier

How that rule is actually enforced

This is not just policy language in the README.

The advisory validator explicitly checks for score-override attempts. If a response includes fields like:

final_score
formal_tier
replication_score
replication_tier

or sets final_score_override, the response is marked invalid with final_score_override_requested.

The packet contract also exports the rule in plain language:

Do not modify or override final_score, formal_tier, replication_score, or replication_tier.

And provider responses must cite exact values from allowed_finding_ids; citation strings are not repaired or loosely matched later.

So the advisory lane is bounded in two ways:

it has no authority to change the deterministic result
it cannot cite evidence outside the bounded packet

That is the kind of mechanism I mean when I say “better boundaries.” If the rule cannot be checked, it is not really part of the architecture yet.

What operational use looks like now

Once these lanes were separated, the CLI became much easier to reason about.

Local engineering review:

stem scan /path/to/repo --level 3 --format all --explain

CI/CD gate:

stem gate /path/to/repo --min-tier T2 --summary off --output results

Offline advisory packet generation:

stem advisory packet /path/to/repo --output advisory_out

Downstream provider response validation:

stem advisory check-response /path/to/repo --response provider_advisory.json

The important point is not just that these commands exist.

It is that each one represents a distinct trust boundary.

That made the project feel more like engineering infrastructure and less like a scoring demo.

A real v1.6.2 packet

To make that less abstract, I re-ran STEM BIO-AI v1.6.2 against a local clone of ClawBio, which describes itself as a local-first, privacy-focused, reproducible bioinformatics-native AI skill library.

The command was:

python -m stem_ai.cli scan /path/to/ClawBio --level 3 --format all --explain

On my machine, that run took about 9.4 seconds and emitted the usual CLI output set: a machine-readable JSON result, a Markdown report, a 5-page PDF packet, and a line-level explain trace.

Before the numbers, the important context is that STEM BIO-AI uses a published triage scale:

T0 = 0-39
T1 = 40-54
T2 = 55-69
T3 = 70-84
T4 = 85-100

Stage 4 replication is reported separately as its own lane, where R2 means some reproducibility scaffolding is present, but not yet enough to call the repository replication-strong.

Governance note:
This is not a “bad repository” scoreboard, a clinical safety verdict, or a moral ranking. It is a deterministic evidence-surface pre-screen intended to support review, not replace it.

With that in mind, the result was:

67 / 100
T2 Caution
Replication lane: 55 / 100 (R2)
Clinical adjacency: CA-DIRECT (the repository surface makes direct healthcare-facing claims, even though it also carries an explicit non-clinical boundary)
Code integrity warnings: C2 dependency pinning, C4 exception handling

This is exactly the workflow shift I wanted the tool to support.

The same deterministic scan is rendered into multiple operator surfaces:

JSON for automation
Markdown for review
PDF for human-facing packet inspection
--explain for file / line / snippet proof tracing

That output shape is only possible because the result object already separates:

formal score and tier
replication lane
diagnostics lane
regulatory traceability
advisory boundary state

In other words, the PDF is not a separate product. It is a view over the same bounded audit object.

Two details from this run are worth calling out.

First, the scanner did not manufacture chemistry findings just because ClawBio is bio-adjacent. The deterministic diagnostics lane reported:

SMILES Surface Integrity: not_detected
SMILES RDKit Validation: not_applicable
SMILES Parser Guard: not_detected

That is the behavior I want. If a detector has no evidence, it should stay silent instead of inflating the report with domain-flavored noise. This is what the earlier thesis looks like when it hits real output: a detector becomes more trustworthy when it is strict about what it cannot conclude.

Second, the score is strict about observable repository conventions. ClawBio uses ClawBio_README_Repo.md rather than a root README.md, so the scan records S1_missing_readme: -20. A human reviewer might decide that this is acceptable contextually. The scanner does not make that leap for them. It only records what the repository exposes through the surfaces it knows how to measure.

That distinction matters. A T2 Caution result here does not mean “ClawBio is unsafe.” It means the current repository surface still raises review-relevant signals under the published deterministic rules, including dependency-pinning warnings, exception-handling warnings in a clinical-adjacent surface, and a stricter-than-human README convention check.

And that is exactly why the next section matters: once the workflow is concrete, the remaining question is not whether the tool can produce an answer, but where its current boundaries still need to stay visible.

What still has to stay bounded

The system is better than it was, but there are still obvious next steps.

1. The public surface is broad

There is now:

scoring
diagnostics
replication
advisory packeting
regulatory traceability
JSON / Markdown / PDF / explain outputs

That is useful, but it increases onboarding cost.

The CLI is clearer now, but the broader public surface has to stay disciplined.

2. The deterministic diagnostics lane is still missing a published calibration threshold

The diagnostics lane is evidence-first by design, but one practical gap remains: the public release does not yet ship a benchmark-backed threshold document saying exactly when a detector is mature enough to graduate from evidence-only into score-bearing territory.

Right now the rule is conceptually clear:

commit-pinned fixtures
reproducible detector output
explicit false-positive review

But the public decision boundary is still partly narrative. Until that calibration surface is published in a more operational form, keeping diagnostics evidence-only is the safer choice.

3. The regulatory confidence labels are rule-authored, not empirically validated

The mapping labels like strong, moderate, and weak-moderate are currently fixed rule-level judgments in the mapping document. They are not runtime model outputs, but they are also not yet backed by inter-rater reliability studies or a published reviewer-agreement benchmark.

That means they are useful as bounded structural mapping language, but they should not be treated as empirical proof that multiple auditors would converge on exactly the same label distribution.

Earlier context

Try it yourself

STEM BIO-AI is Apache 2.0 and fully open source.

If you want to know whether a bio/medical AI repository is actually exposing reviewable evidence, or whether your own repository is weaker than you think, run it yourself.

That is the real test.

GitHub: https://github.com/flamehaven01/STEM-BIO-AI
License: Apache 2.0

Final thought

The earlier STEM-AI posts were about why repository trust deserves its own audit layer.

This phase was about something more practical:

what does that audit layer have to look like if an engineer is actually going to run it, inspect it, and put it in a pipeline?

For me, the answer was simple:

Separate the workflows.
Separate the lanes.
Keep diagnostics evidence-first.
Keep regulatory mapping subordinate to evidence.
Keep advisory AI bounded.

Optimize for inspectability, not just score production.

That is what changed the project.

Not bigger claims.

Better boundaries.