Pressure-testing Ota on lead-quorum: native Python truth, repo-local fulfillment, and runtime bind projection

Overview

lead-quorum was a strong pilot repo because it was small enough to reason about and real enough to fail honestly.

It has:

• repo-local Python environment ownership
• pinned dependency installation
• env bootstrap from example truth
• a deterministic local test surface
• live external verification
• a local web runtime
• a distributed demo path
• a Docker build lane

That is exactly the kind of repo where a contract can look clean while still hiding real setup and execution drift.

Why this repo mattered

The useful pressure here was not “can Ota run one Python command.”

The useful pressure was whether Ota could stay truthful when the repo itself owns:

• the .venv
• the dependency install lane
• the local executable path
• the runtime listener truth

If Ota probes or fulfills those in the wrong order, the contract is not trustworthy even if the repo itself is valid.

That is what made lead-quorum valuable.

What the contract now models

The final contract is explicit about the repo’s real setup split.

Setup is not one opaque shell step. It is three different ownership surfaces:

• copy .env from .env.example only if missing
• create the repo-local virtual environment
• hydrate dependencies through typed uv requirements-file installation

That looks like this in the contract:

setup:
  aggregate:
    tasks:
      - setup:env
      - setup:venv
      - setup:deps

setup:env:
  action:
    kind: copy_if_missing
    from: .env.example
    to: .env

setup:venv:
  action:
    kind: ensure_virtualenv
    path: .venv
    python: "3.12"

setup:deps:
  prepare:
    kind: dependency_hydration
    medium: package_dependencies
    source:
      kind: uv
      cwd: .
      mode: pip_requirements
      requirements_file: requirements.txt

The contract also keeps verification and external-runtime claims separate:

• verify for deterministic local validation
• live for Gemini-backed end-to-end testing
• app for the local web service
• distributed for the A2A demo path

That matters because a working local scoring test and a live distributed runtime are not the same readiness claim.

What lead-quorum exposed in Ota

This repo exposed three real Ota gaps.

1. native repo-local fulfillment and probing were ordered incorrectly

Older Ota could still probe repo-local Python executables too early.

That is the wrong trust order for a repo that creates its own .venv as part of setup. If the repo-local interpreter path is declared as:

exe: .venv/bin/python

then Ota has to materialize the dependency/setup closure before treating that path as a fulfilled runtime command.

Otherwise a valid contract can fail just because Ota asked the question too early.

That is exactly what this repo exposed.

2. native Python candidate selection was too weak

lead-quorum also exposed a narrower but important gap in typed Python hydration.

When Ota selected a local Python candidate for setup and hydration, it could still choose the wrong host interpreter path instead of the repo’s intended environment. In practice that meant a dependency like cryptography could start building against the wrong target environment instead of the repo-owned Python lane.

That is not a repo bug. That is a readiness engine selecting the wrong execution truth.

3. runtime bind truth was duplicated between launch args and runtime listeners

The local web service made a third weakness obvious.

Before widening, the contract still had to repeat bind truth in two places:

• command-line runtime args such as --host and --port
• the declared runtime listener surface

That duplication is fragile.

The stronger product shape is for the runtime listener to stay canonical and for Ota to project the supported bind flags for known servers. lead-quorum became the first real repo to pressure that widening cleanly.

What changed in Ota

lead-quorum drove platform fixes, not repo-local workarounds.

native repo-local fulfillment now respects setup materialization order

Ota now runs the selected setup closure before probing repo-local backend/runtime commands that depend on that materialized state.

That closes the gap where a repo could declare a truthful .venv/bin/python lane and still fail because Ota evaluated it before the repo-local environment existed.

Python candidate selection is now stricter and more truthful

Ota also now prefers version-matching Python candidates before falling back to weaker generic host candidates.

That makes typed Python hydration much less likely to drift onto the wrong interpreter family when the contract has already declared the intended Python lane.

runtime listener truth can now project bind args

This repo also helped widen Ota’s runtime-to-launch projection.

The local web service can now declare listener truth once and let Ota project supported bind args for a known adapter:

launch:
  kind: command
  exe: .venv/bin/uvicorn
  args:
    - web.app:app
  runtime_projection:
    listener: web
    adapter: uvicorn

That is a cleaner long-term shape than duplicating --host and --port in every Python service contract.

What the matrix now proves

The released v1.6.24 pressure matrix on Ota's fork is green across Ubuntu, macOS, Windows, and
a Dockerfile-owned Ubuntu container lane. Ubuntu and macOS execute the native deterministic and
runtime-proof lanes. Windows is intentionally audit-only in this matrix: it validates the contract
and discovers the task surface, but does not execute native tasks or runtime proof.

That matters because this repo’s truth is not only “tests pass on one machine.” The matrix proves:

• ota validate
• ota doctor
• ota tasks --use
• ota tasks --safe --use
• task dry-run coverage for setup, deterministic test, live test, distributed demo, and Docker build lanes
• workflow dry-run coverage for verify, app, live, and distributed
• real setup
• real deterministic test
• real verify workflow proof
• real local app workflow proof
• real docker:build
• real verify:container workflow against the repository Dockerfile image
• contract-modeled and dry-run-covered live external and distributed workflows; their real matrix execution is conditional on GOOGLE_API_KEY and was skipped in this released-version run

That is a much stronger outcome than “the contract parses.”

Why this repo was a good pilot

lead-quorum did exactly what a first pilot should do.

It did not mainly expose repo noise. It exposed trust gaps in Ota itself:

• setup before probe
• correct interpreter selection
• one canonical runtime bind truth

Those are real product boundaries.

That is why this repo was worth pressure-testing.

It also reinforced an important standard for future pilots:

a receipt or contract is only useful if it stays aligned with the actual decision and execution path, not just with what the repo intended in prose.

lead-quorum is created and maintained by Vinicius Pereira. The
Ota integration is proposed upstream; the forked matrix below is pre-merge pressure evidence, not
evidence from the canonical repository.

Links

• Upstream repository: vinimabreu/lead-quorum
• Integration PR: #3
• Pre-merge pressure contract: fork branch ota.yaml
• Pre-merge released-version matrix: #29572074145

---

Originally post here: https://ota.run/blog/pressure-testing-ota-on-lead-quorum-3x2p

Comments

[Vinicius Pereira]

Repo author here. The ordering gap is the one I would expect to bite first: probing repo-local executables before the .venv is materialized asserts on a state that does not exist yet, the exact failure mode I build lead-quorum to refuse at runtime. Observe the environment in the order the runtime builds it, not the order that is convenient to probe. And projecting the bind from the canonical listener declaration is the right call, host and port duplicated across two places drift the moment someone edits one.

[Bobai Kato Ota]

Exactly. The contract was truthful; Ota was observing it in the wrong order.

We fixed the selected native path, so setup materializes the repo-owned environment before Ota probes .venv/bin/python. We also made the declared runtime listener canonical, with Uvicorn bind arguments projected from that truth instead of duplicated.

Thanks for designing lead-quorum with those failure boundaries clearly exposed. It made both Ota gaps impossible to dismiss as repo noise.

Curious whether you’ve seen the same ordering or duplicated-runtime-truth problem in other repos. What setup or runtime assumption still causes the most avoidable failures for contributors or CI?

[Vinicius Pereira]

The one that bites most, in my experience, is ambient state: setup and tests that pass because the author's machine already has something a fresh CI runner or a new contributor does not. A warmed venv, a model already downloaded, an env var exported three weeks ago, a service already running, a migration applied by hand once and never again. The ordering gap Ota hit is a special case of it, asserting on a .venv that exists on your box because you built it last Tuesday, not because setup built it this run.

The discipline that kills the whole class is the same one you just applied: build every dependency explicitly, in the order the runtime needs it, and assert only against state this run created, never state that happens to be lying around. The cheapest way to prove you did it is to run setup and the suite in a clean container or a fresh runner from nothing, because a cold start is the only honest test of a setup contract. The related one, and the reason the bind fix matters, is duplicated truth: a host, a port, a path repeated in two places is a bug with a delay on it. It passes until someone edits one copy. Projecting from a single canonical declaration is how you stop paying that later.

[Bobai Kato Ota]

Ambient state is the dangerous setup state: it passes because it already existed, not because the contract created it.

Ota is adding explicit execution freshness: cold_start_verified, persistent_state_reused, or unknown. A green run will only claim cold-start verification when Ota can prove an isolated boundary created the needed prerequisites.

The same principle applies to runtime binds: declare the listener once and project command flags from it.

What single receipt field would make you trust a green setup was genuinely cold: created artifacts, boundary identity, or reused-state disclosure?

[Vinicius Pereira]

Created artifacts, but only in one specific form: a per-prerequisite creation record, the digest minted by the setup step at the moment it built the thing, re-verified at assertion time. If every artifact the assertions touch matches a digest this run minted, the run was cold with respect to everything it tested, and that is the only cold that matters.

Why I pick it over the other two. Boundary identity tells me where the run happened, but a fresh boundary is itself a claim: cache mounts, volumes, and fetched warmed state all survive a pristine image digest, so it is a strong witness and a weak proof. Reused-state disclosure is honest but self-declared, which puts it in the same family as the flags we already agreed not to trust: it reports intent, not fact.

The creation-record form also has a second payoff for your tri-state. It turns the run-level flag into a per-artifact verdict: instead of one cold_start_verified over the whole run, the receipt can state exactly which prerequisites were minted by this run and which were inherited, computed from digest comparison rather than declared. Cold stops being a property of the machine and becomes a property of each dependency's provenance, and that version you can enforce.

[Bobai Kato Ota]

That distinction is useful. We’re shaping this as runner-authored provenance per prerequisite, not a global cold flag.

Ota will separate target freshness from derivation posture: a clean .venv or node_modules rebuilt this run can be cold_start_verified even when a package cache assisted reconstruction. Each consumer assertion must bind to the producer identity it actually relied on; anything Ota cannot verify remains unknown.

The harder next boundary is services, databases, and volumes. What identity would you trust there: initialized-state digest, migration lineage, or adapter-authored attestation?

[Vinicius Pereira]

Migration lineage, because it is the only one you can enforce rather than declare.

An initialized-state digest hashes the wrong thing: a database's bytes are non-deterministic (row order, page layout, timestamps, counters), so it flags functionally identical states as different. That is the byte-level version of the shape check we already threw out.

Adapter attestation is the self-declared family: a claim about what the adapter did reports intent, not fact, unless it binds to something checkable, and then it is just a signature wrapped around the lineage anyway.

Lineage works because it is the derivation, not the artifact. For a file, identity is its bytes. For state, the bytes are the wrong identity, so you move up a level: the identity of state is the ordered, content-addressed transformations that produced it over a pinned base. Deterministic and replayable, so verified instead of declared. Add a digest of the seed data computed over the logical state, not the physical file, since lineage covers schema but not what init seeds.

The honest boundary: lineage only determines the state while each migration is a pure function of the base. The moment one reaches for a clock, a random seed, or a network call, that step is your ambient-state problem one level up. Pin it or it stays unknown.

[Bobai Kato Ota]

Migration lineage as schema-derivation evidence makes sense, but database identity must be composite rather than lineage-only.

Ota would need a verified engine/base identity, ordered content-addressed migrations, and runner-witnessed application inside the selected boundary. Seed or data-state identity should be required when the selected proof actually depends on it.

A pre-existing database can be verified_reused only when Ota verifies the reused boundary; matching lineage alone does not verify its current schema or data state. It can never be cold_start_verified.

Agreed that unpinned time, randomness, network access, or external state leaves the affected derivation path unknown.

For seed-dependent proofs, would you trust a canonical logical dump or scoped digests over selected tables and queries?

[Vinicius Pereira]

Scoped digests, but the scope has to be witnessed, not chosen. A canonical logical dump is over-specified on scope the same way the initialized-state digest was over-specified on shape: it hashes rows the proof never reads, so a functionally irrelevant seed change (a backfilled column, a reordered tenant insert, an extra row) flips a proof that was valid. Canonicalizing the dump fixes the byte-shape half and leaves the scope half untouched.

The trap is that a hand-declared scope ("this proof depends on tables X and Y") is adapter attestation again: a claim about the read-set rather than the read-set. So the scope cannot come from the author. It has to be the set the runner actually observed the proof touch during application inside the selected boundary, then digested over the logical state of exactly those rows and queries (multiset, ordered, typed), not the physical dump.

That also dissolves the "required when the proof depends on it" question into something you derive instead of toggle. Empty witnessed read-set, the proof did not lean on seed data and there is nothing to digest. Non-empty, seed identity is required and its scope is already pinned, because the runner just watched the proof read it. verified_reused is exactly the case where that scoped logical digest over the reused boundary matches, while matching lineage alone only agrees on the schema derivation, not on the data the proof stood on.

[Bobai Kato Ota]

Scoped logical digests make sense, but I would keep the proof scope contract-declared rather than letting one observed run define it.

The runner or database adapter should compute the identity and record actual reads as corroborating evidence. Ota can then detect undeclared access or missing observations. Caches, views, triggers, stored procedures, conditional paths, and incomplete tracing mean an empty read-set cannot prove there was no data dependency.

So the model becomes declared obligations plus witnessed evidence. If they disagree, or instrumentation is incomplete, the affected obligation remains unknown. A reused database also requires verified boundary, engine/base, migration lineage, and relevant scoped data identity; it can never become cold_start_verified.

Where would you trust the witnessed access evidence to come from in practice: database audit/query logs, a proxy, application tracing, or proof-specific instrumentation?

[Vinicius Pereira]

You are right, and it corrects what I overstated. An empty witnessed read-set proves nothing, since triggers, stored procedures, views, and cache hits run below the trace. So witnessed is not a replacement for declared, it is a non-subtractive auditor on top: it can add undeclared access or challenge a claimed read, never shrink the declared floor. Either declared and witnessed agree with complete instrumentation, or the obligation stays unknown. That is your model, and it holds.

On the source, rank it by how far it sits from where those hidden reads actually execute. Application tracing and proof-specific instrumentation are the weakest, they live above the engine and never see a trigger or a stored procedure's reads. A proxy sees the wire but not server-side reads. The engine's own audit or execution log is the only witness at the layer where triggers, procedures, and view scans run, so trust the lowest enforceable layer. Where you cannot show it captured a given path, the obligation stays unknown rather than verified.

[Bobai Kato Ota]

Agreed. Declared obligations remain the floor; witnessed access can expand or challenge them, never shrink them.

Engine-level audit or execution logs are the strongest source because they observe server-side behaviour, but only when Ota can verify that instrumentation was enabled, complete for the selected path, and bound to the current proof transaction. Otherwise, the affected obligation remains unknown. That is the boundary we will preserve for a future database adapter.

[Vinicius Pereira]

Agreed, and one thing I would pin down for that adapter: those three conditions have to be witnessed too, not taken on the log's word. Enabled and complete-for-the-path should fall out of the log's own structure (sequence continuity, no gaps in the segment), and bound-to-this-proof should be a token or digest tying the log slice to the transaction id, not a flag the log asserts about itself. Otherwise the log is grading its own coverage, which is the self-report problem one layer down. Same rule the whole way: derived, not declared. Good thread.