DEV Community: Bobai Kato

Pressure-testing Ota on Dagger: generated SDK lineage and bounded engine truth

Bobai Kato — Fri, 04 Sep 2026 11:18:15 +0000

Overview

Dagger is a useful pressure repository because it already has a strong execution model. Its SDKs are generated from a shared API surface, its CLI starts a Docker-backed engine, and a generated-source change has consequences beyond the command that wrote it.

The useful Ota question is narrow: can Ota govern one generated TypeScript SDK closure from declared source inputs, through the generator, to a committed and replayable output without pretending to govern every Dagger SDK, engine operation, or release artifact?

This note records the Ota contract and pressure boundary. It is intentionally a slice proof, not a claim that Dagger is globally ready or that Ota controls Dagger's managed engine.

Why this repo mattered

Generated source is easy to under-model. A CI step can run a generator and exit green while the repository still carries stale output, or while a later consumer has never actually seen the generated files.

The selected Dagger shape is small enough to prove honestly:

one source-managed Dagger CLI;
one generator for the TypeScript SDK client;
one declared generated-source artifact;
one consumer that detects an uncommitted generated diff; and
one promoted replay consumer that checks the reviewed baseline without regenerating it.

That is enough to test lineage and replay authority without converting a selected SDK path into a claim about Dagger's complete code-generation or release system.

What the contract models

The contract pins released Ota and declares the minimum compatible version. It models the generated output as a first-class artifact:

artifacts:
  typescript-sdk-client:
    kind: generated_source
    producer: sdk:generate
    paths:
      - sdk/typescript/src/api
    inputs:
      - dagger.json
      - toolchains/typescript-sdk-dev/ts-sdk.dang
    replay:
      authority_manifest: replay/typescript-sdk-client.ota.json
      consumption: verify_unchanged

tasks:
  sdk:check-generated:
    depends_on:
      - sdk:generate
    requires_artifacts:
      - typescript-sdk-client

The producer invokes the declared Dagger release asset. The ordinary consumer depends on the producer and rejects a generated diff. The promoted consumer has no generation dependency; it must consume only the explicitly promoted authority and then verify the same output directory.

The contract assigns Ota ownership of selected tool fulfillment, contract closure, output lineage, replay admission, and receipt evidence when those stages are reached. Dagger owns the meaning of the generator, its engine lifecycle, and the module orchestration inside that engine.

What Ota pressure exposed

The first pressure path exposed a real Ota runner defect: the declared Dagger release asset was fulfilled and version-checked, but native shell execution did not preserve the managed PATH when the generator started. The contract had named the correct tool; the runner failed to carry that truth into execution.

The fix belongs in Ota, not in Dagger-specific shell glue. Native shell execution now receives the resolved managed PATH, with regression coverage for the source-managed command path.

The same pressure also sharpened the engine boundary. Dagger starts an engine through Docker and performs package and module resolution inside that managed runtime. Ota can declare and verify the Dagger CLI and Docker requirement, but it does not independently attest the engine's internal package index, module image, or all external state used by the generator. Those facts remain bounded rather than silently promoted to repository proof.

Pressure design

The pressure workflow is deliberately staged. Ota must establish contract truth and admission before the repository's generator is allowed to start; generation, receipt capture, and replay are separate intended stages rather than one undifferentiated green check. This design is not itself matrix evidence.

Intended stage	Intended evidence
Contract and discovery	`ota validate`, `ota tasks --use`, and `ota tasks --safe --use`
Admission	Generated SDK task and workflow dry-runs before execution
Generation	`sdk:generate` plus `sdk:check-generated` over the selected TypeScript API directory
Fulfillment	Dagger `v0.21.7` release-asset resolution and version probe
Receipt	Archived selected-workflow execution evidence
Replay	Explicit record, explicit promotion, promoted consumer, and no-regeneration diff check
Diagnosis	`ota doctor` on the fulfilled and promoted paths

The workflow installs Ota from the contract's released source, not from a branch or implementation revision. The generated SDK lane and promoted replay lane also remain separate: recording creates evidence, promotion selects it, and replay consumes that selection without falling back to the latest local receipt.

What the run established

The released-source pressure run
contains one Ubuntu job and concluded with failure. It is not Linux/macOS/Windows matrix evidence. The run is bound to exact Dagger revision 3ffcd68bad1a52011c3f08f390c0093a1e0b9787. Its retained GitHub job logs establish only the successful stages reached before that failure:

the repository installed Ota from its declared released source;
the contract validated without warnings;
runnable and agent-safe surfaces were inspectable;
the generated SDK task and workflow could be previewed; and
workflow preparation completed before the Dagger-managed generator started.

Execution then stopped inside Dagger's managed engine. Its package resolver could not satisfy the engine-internal protobuf-dev~32 constraint from the configured repositories. This is useful pressure evidence: the contract made the boundary visible and Ota stopped at the Dagger-managed runtime failure instead of converting a partial run into generated-source or replay proof.

The generator, receipt, promotion, and replay stages were not reached. They are therefore not claimed as exercised by this run.

What this does not prove

The selected lane does not prove:

every Dagger SDK or generated language client;
Dagger documentation generation or release artifacts;
the health, package index, module image, or internal state of the Dagger-managed engine;
production Docker policy, registry credentials, or deployment behavior;
cross-platform or repeated matrix behavior; or
repository-global correctness beyond the selected generated TypeScript SDK closure.

When exercised, promoted replay would not prove application correctness. It would prove only that the explicitly selected authority and output identity were consumed without the producer closure. This run did not reach that stage, and a future green replay lane would not remove the engine,release, or broader-repository boundaries above.

Uncovered material behavior

Contract-owned and exercised

released Ota bootstrap;
Dagger release-asset fulfillment and version checking;
contract validation and task/workflow admission; and
workflow preparation before the Dagger-managed generator started.

Contract-owned but not reached

the selected generated TypeScript SDK producer and consumer closure;
generated-output lineage and committed-diff detection;
explicit replay recording and promotion; and
promoted replay consumption without regeneration.

Repository- or runtime-owned

Dagger engine lifecycle and internal package/module resolution;
Docker daemon policy and image availability;
other SDKs, documentation, release, and deployment workflows;
GitHub triggers, permissions, credentials, runners, and artifact retention; and
Linux, macOS, Windows, and repeated-run matrix behavior.

Ota platform gaps

The earlier pressure path exposed and led to the repair of Ota's managed-PATH runner defect. The final released-source run exposed no additional Ota platform gap; it stopped at the bounded Dagger-managed engine dependency described above. If a later run shows that Ota cannot model a material Dagger boundary cleanly, that result belongs in a named widening opportunity rather than a repository-local helper or an over-broad proof claim.

A Protected Runner Is Not Approval: Pressure-Testing Ota's First Signed Authority Carrier

Bobai Kato — Wed, 02 Sep 2026 18:00:37 +0000

A runner label chooses a machine. It does not authorize a repository action.

GitHub Actions can route a job to a self-hosted runner. A label answers one useful question:

Which machine is eligible to receive this job?

It does not answer the question that matters for a heavier repository action:

Is this exact publish, migration, deployment, or other non-routine task authorized to run now?

Those are different boundaries. A workflow must not be able to create its own approval merely by asking for a more privileged runner.

Ota's first signed crossing-authority carrier is designed around that separation. The repository declares the governed lane. The workflow asks GitHub to schedule a protected runner. A separately managed authority source decides whether the exact lane may cross its boundary. Ota verifies that decision immediately before effects start and emits a fresh transaction record after it does.

We pressure-tested that model on a pre-provisioned Linux/x64 VPS runner. One live authorization executed and produced a retained archive. Three invalid-authority cases refused before the task ran. The result is not a general approval system. It is concrete evidence for a deliberately bounded carrier.

The flow under pressure

Repository contract selects governed lane
        |
Workflow requests a protected runner label
        |
Protected runner invokes `ota run ... --grant <id>`
        |
Ota verifies fixed authority state and exact semantic scope
        |
One fresh crossing transaction begins
        |
Selected task runs, then receipt and archive bind admission to outcome

Each layer retains its own job:

Layer	Owns	Does not own
Repository contract	Which lane requires a crossing	Trust keys, grants, bundle paths, or revocations
GitHub workflow	Scheduling, checkout, credentials, and provider policy	Authority issuance
Authority provisioner	Signed bundle, revocation, sequence state, and signing key	Repository task selection
Ota on the runner	Admission, scope verification, transaction, receipt, and archive	Issuing its own authority

That separation is the product value. GitHub still owns scheduling and platform controls. Ota adds repository-specific governance: a way to bind independently issued authority to the complete action the repository is about to execute.

What we tested

The pressure workflow verified the exact administrator-installed Ota binary against a root-owned full-commit and SHA-256 manifest before it checked authority. It then ran Ota's read-only hardening diagnostic as the unprivileged job user. Required observations had to pass: Linux/x64, non-root execution, no declared Docker host or common Docker socket, and valid fixed trust, bundle, and sequence records.

The four hosted scenarios ran against the same merged workflow revision:

Scenario	Result	What the evidence proves
Live grant	Passed	Exact-scope admission created a completed crossing transaction; the retained artifact contains the verified receipt archive and valid receipt history.
Expired grant	Passed	Dry-run and real execution refused with `crossing_grant_expired`; before/after checkout manifests matched.
Revoked grant	Passed	Dry-run and real execution refused with `crossing_grant_revoked`; before/after checkout manifests matched.
Out-of-scope grant	Passed	Dry-run and real execution refused with `crossing_grant_out_of_scope`; before/after checkout manifests matched.

The refusal cases are important. A green validate result or a rejected command alone would not establish the boundary. The pressure workflow retained typed refusal JSON, the human refusal output, and complete checkout manifests before and after both dry-run and real refusal. The selected scaffold
task never started.

What Ota verified before the live task ran

The repository contract names only an authority identifier:

governance:
  crossing_authority:
    authority_id: platform-release-authority

It does not contain the signing key, bundle location, trust-store path, or revocation state. Those live outside the checkout at fixed protected system paths:

/etc/ota/crossing-authorities.json
/var/lib/ota/crossing-authority.json
/var/lib/ota/crossing-authority-sequence.json

Before Ota admits the selected lane, the first carrier verifies the fixed authority binding and the signed bundle, then checks the grant's exact contract identity, complete selected execution scope, crossing family, classification, actor posture, expiry, revocation, and sequence/high-water state.

The scope is not a friendly task name. Changing dependencies, hooks, task effects, target platform, or execution selection changes the semantic scope identity. A standing grant for yesterday's publish lane cannot silently authorize a changed closure that happens to keep the same label.

Why the archive matters

A successful run needs more than an approval-looking log line.

For the live scenario, Ota created a fresh crossing transaction before the governed scaffold action, recorded its terminal success, and retained the receipt archive in the hosted artifact. Receipt history revalidated that archive rather than merely reporting that a file existed.

That gives a reviewer a linked chain:

declared governed lane
        -> verified signed authority and semantic scope
        -> fresh crossing transaction
        -> selected task outcome
        -> archive that re-derives the recorded scope

This is the gap Ota is intended to close. A provider can tell you that a workflow ran. Ota can preserve which repository action crossed a declared boundary, what authority admitted it, and what happened afterward.

What this does not prove

The correct claim is intentionally narrow:

The authority files were protected from the current Ota process under the observed filesystem
boundary, and Ota verified the selected crossing against them before work started.

The carrier reports this as current_process_filesystem_guarded. It does not prove:

provider-attested isolation between the job and an administrator;
absence of every possible escalation path, mount alias, metadata credential, or host-control endpoint;
a verified human, CI, or platform identity;
one-use authority or atomic broker consumption;
repository-global safety, application correctness, or prevention of raw-shell bypasses.

The runner's local inspection is diagnostic evidence. It cannot independently attest the whole provider boundary. A signed image or a protected label alone would not change that fact.

What came next

The VPS matrix closed the pressure gate for Ota's first signed-file carrier. V11.7 later added the separate broker-backed carrier, available in Ota v1.6.26 and later, and completed its bounded OSS
slice.

The completed broker path binds launcher attestation to the job, consumes a short-lived one-use work-unit lease atomically, retains terminal cleanup and archive evidence, and proves recovery on an independently administered hardened launcher. Provider attestation remains optional stronger hardening rather than a V11.7 completion requirement.

The useful and honest statement remains narrow: Ota can verify separately issued, exact-scope authority on a protected runner and produce transaction-bound evidence of execution or refusal. It does not call that provider isolation, verified human identity, raw-shell prevention, or a complete enterprise control plane.

For the operator layout and verification procedure, see the canonical Prebound Crossing Authority reference.

Originally posted here: https://ota.run/blog/a-protected-runner-is-not-an-approval-system-4q8m

Ota v1.6.27 Now Available: Effect-Bound Refusal Assurance

Bobai Kato — Tue, 01 Sep 2026 13:57:16 +0000

Idea

Ota v1.6.27 closes V12: effect-bound refusal assurance. It adds a deliberately narrow foundation for declaring a database-schema mutation, deriving its exact selected realization, and refusing before selected work begins. Provider execution remains disabled in this release.

The important boundary is exactness. Ota derives the selected contract attachment, resource, migration bytes, working directory, invocation origin, and, when policy is available, one command-scoped decision before its selected execution path begins. It can then report a refusal with execution_started: false rather than starting setup or rendering a workflow preview that describes the lane as runnable.

This is not a database executor, provider integration, or a claim that Ota has protected a database. It is a reviewed, machine-readable refusal boundary for one selected contract closure.

Feature

Typed effect declarations

Contracts can declare database_schema_mutation through a canonical resource binding, a typed effect definition, and an exact task attachment. Resource, consequence, attachment, evidence, and realization identities are separately content-addressed so the policy decision cannot flatten distinct sources or invocation origins into a broad task label.

Declared-only commands remain review truth, not execution authority. A command without an eligible typed realization cannot become runnable merely because it names the same conceptual effect.

Refusal before selected work

ota run, ota up, ota up --dry-run, runtime and lifecycle proof, CI checkout re-evaluation, and sandbox capability reporting share the typed-admission boundary. A selected typed denial is evaluated before Ota's selected setup, workflow environment rendering, proof artifacts, durable logs, services, dependencies, shell dispatch, or provider contact.

Blocked ota up --dry-run --json output retains the admitted non-secret application plan and available decision evidence. It contains only the refusal action, reports BLOCKED, and does not say that Ota would execute the selected command.

Explicit policy and canary evidence

Policy packs can evaluate exact typed-effect realizations with deterministic deny > warn > allow precedence. Typed lanes stay provider-disabled even when policy evaluates to allow or warn. An effect-refusal canary passes only for one eligible realization with an applicable explicit typed deny before execution starts; fallback and unrelated denials cannot turn into a passing control.

Private archive reconciliation

Operators can archive a workflow typed refusal explicitly. Ota re-derives the archived contract, selected closure, migration plan, policy snapshot, decision, typed-deny basis, and pre-execution posture before accepting it. Doctor promotes only the exact current workflow claim backed by valid private evidence; stale, task-only, ambiguous, invalid, equivalent-path, and opaque-path claims remain unknown.

ota contract effect-refusal-candidate is the first review-only bridge from that evidence back to contract authoring. Its schema-v5 candidate proposes one canary declaration as unknown, has no application projection, and remains non-writable through apply-candidate.

Detector and evidence improvements

GitHub Actions detection now retains canonical job and step working directories in source-bound task closure truth. Dynamic, noncanonical, missing, or escaping directories do not become runnable root-task claims. Named multiline verification steps remain complete ordered bodies and unresolved rather than being truncated into a promoted command.

The Core-owned pressure-evidence registry records each retained case's exact revisions, matrices, exercised surfaces, proven facts, and not_proved boundaries. The Site pressure index is generated from that registry; it is not a repository certification, endorsement, or badge wall.

Pressure and evidence

V12 retained immutable Linux and macOS controls for the exact selected boundaries, including typed run and up refusal, blocked preview construction, policy and CI re-evaluation, sandbox admission, canaries, archive history, Doctor fallback after tampering, and review-only candidate reconciliation. Final bounded real-repository matrices ran against forked Plausible and Outline revisions.

The evidence is tied to exact revisions and its limits are retained alongside the claims.

What this does not claim

v1.6.27 does not prove provider contact, authorization, or mutation behavior. It does not prove database migration success, rollback, correctness, or data integrity. It does not prove absence of arbitrary child processes, complete repository immutability, independently administered policy authority, positive execution receipts, positive assurance, public archive-export safety, or repository-wide readiness beyond the selected contract closure.

Those are not release-note caveats. They are boundaries that keep a refusal Ota witnessed distinct from an outcome it did not observe.

Docs

Use the live contract reference, policy-pack reference, andcommand reference for the current specification and operator flows. The pressure-testing index is the canonical registry for retained evidence and its limits.

Release

v1.6.27 is live here: https://ota.run/releases/v1.6.27

If you already use Ota, upgrade and verify:

ota upgrade
ota --version --json
ota validate
ota doctor --json
ota tasks --use
ota run <task> --dry-run --json
ota up --workflow <workflow> --dry-run --json

The GitHub release contains the shipped binaries and checksums. Author only behavior your repository can review; a typed declaration is precise contract truth, not a reason to infer provider authority.

If you are evaluating Ota for the first time, v1.6.27 shows the standard the product is setting: a consequential repository operation can be declared precisely, evaluated against exact policy truth, and refused before selected work begins without claiming an outcome Ota did not observe.

Ota is not turning an effect declaration into execution authority. It is making the difference between declared intent, reviewed refusal, and proved outcome explicit enough for humans, CI, and agents to rely on.

Original posted here: https://ota.run/blog/ota-v1-6-27-release-essay

Pressure-testing Ota on EventCatalog: generated artifact lineage across sibling consumers

Bobai Kato — Fri, 28 Aug 2026 12:22:22 +0000

The finding

EventCatalog exposes a common monorepo failure mode: generated code may exist, its producer may be green, and the real downstream consumer can
still fail. Its Langium language server generates AST, grammar, module, and syntax files; a sibling VS Code extension consumes that output alongside the workspace SDK and visualiser.

The useful question is therefore not "did generation finish?" It is whether the repository can execute the complete consumer closure from declared dependency hydration through the package that needs the generated result.

The contract boundary

Ota models the generated output separately from the tasks that establish and consume it:

artifacts:
  language-server-ast:
    kind: generated_source
    producer: language-server:generate
    paths:
      - packages/language-server/src/generated/ast.ts
      - packages/language-server/src/generated/grammar.ts
      - packages/language-server/src/generated/module.ts
      - packages/language-server/syntaxes/ec.tmLanguage.json
      - packages/vscode-extension/syntaxes/ec.tmLanguage.json
    inputs:
      - packages/language-server/src/ec.langium
      - packages/language-server/langium-config.json

tasks:
  vscode-extension:build:
    depends_on:
      - language-server:generate
      - language-server:build
      - sdk:build
      - visualiser:build
    requires_artifacts:
      - language-server-ast

The setup task owns typed, frozen-lockfile pnpm hydration with the language-server package filter. That removes bespoke install shell glue without pretending the dependency path is harmless: it reaches the package registry, so the selected closure is intentionally not routine agent-safe execution. Humans and CI can run the declared verification workflow; unattended agents cannot
silently acquire that networked setup authority.

What Ota had to learn

This pressure case made two platform requirements concrete. Generated-source lineage had to remain visible at consumer admission and in execution evidence, rather than surfacing only after a build failure. And pnpm dependency hydration needed a package filter that could select a sibling package without collapsing the workspace into one opaque install-and-build command.

The resulting boundary is deliberately narrow. Ota knows which task produced the declared paths, which source inputs define that artifact, and which consumer requires it. A path merely existing is not a freshness claim, and this note does not treat it as one.

The released pressure run

Run 33167453142 passed against released Ota v1.6.26 at EventCatalog commit b593861fc84147884dcfcaf03898e490596927a0.

The native matrix ran on Ubuntu, macOS, and Windows. Each lane validated the contract, ran Doctor, listed runnable and agent-safe task surfaces, dry-ran both the consumer task and named workflow, executed the real VS Code-extension consumer closure, archived its receipt, and uploaded the resulting evidence. A separate Linux job ran ota proof runtime --workflow verify, archived the workflow receipt, and retained its proof artifact.

That is stronger than proving the generator alone: the final build consumes the generated source after Ota has resolved its producer, language-server compilation, SDK build, and visualiser build. The matrix pins the released Ota source through the contract and pins every GitHub Action revision to an immutable SHA.

Boundaries that remain

This is contract-owned and proved for one finite native language-server-to-extension build path. It does not prove the full EventCatalog workspace, every generated artifact, the semantic correctness of the generated language model, VS Code packaging or distribution, package publishing, releases, or deployment behavior.

The contract does not advertise container execution for this slice, so the three operating-system legs are native-only evidence. Package-registry availability remains an external dependency of the declared hydration lane. Those are explicit limits, not green-run implications.

Pressure-testing Ota on cloud.devenv.sh with Nix and devenv

Bobai Kato — Wed, 26 Aug 2026 16:32:15 +0000

Overview

cloud.devenv.sh is a useful Ota pressure case because its execution path crosses three distinct
ownership boundaries:

the host provides nix
Nix launches a pinned devenv revision
devenv owns the repository's test and up subcommands

A weak contract would flatten that into an opaque shell command. The stronger model preserves who
owns each layer and lets Ota select the repository operation without pretending to own devenv's
internal process semantics.

The immutable hosted matrix at repository revision
29c44c235795d3af3b1b7eafdaacd63a416d1f06
completed successfully on Ubuntu and macOS with released Ota v1.6.26. The result proved the
modeled boundary, but it also exposed an important limit: a green orchestrator exit is not proof
that every nested process succeeded.

Evidence snapshot

The hosted workflow run
used:

cloud.devenv.sh revision 29c44c235795d3af3b1b7eafdaacd63a416d1f06
Ota v1.6.26 source revision 08855fd1b7269abe9342dc9dd45acf4ed1885063
contract-pinned devenv launcher revision 65a7c40d185350f0e96783b3dd8d4afab3bb7034
Ubuntu binary SHA-256 3d545be8eb9701542284d28bb89fba83c77ae30c1d1157d9dd87fb83d961e05e
macOS binary SHA-256 a2448590b0fedfeaffd29535103d5209856e3b0fcb42a93d0f47256920b58a79

Both jobs validated the contract with zero errors and warnings, inspected Doctor's deterministic
fix preview, listed the public task surface, exported execution topology, and dry-ran test,
verify, dev, and the verify workflow. The contract intentionally declares no agent-safe task,
and the retained output confirmed that ota tasks --safe --json returned an empty list.

Doctor did not call the repository ready. Its retained fix preview reported verdict: risky and
agent_verdict: not_ready because .ota/state/, .ota/contracts/, .ota/receipts/, and
.ota/proof/ were not ignored by Git. The workflow intentionally accepted that advisory posture;
its green status must not be read as an agent-readiness verdict.

The Ubuntu job additionally executed the finite test task through Ota. The macOS job did not run
that real task; it proved only validation, discovery, topology, Doctor, and dry-run admission.

The contract boundary

The contract keeps launcher ownership explicit:

orchestrators:
  devenv:
    kind: devenv
    required: true
    config_files:
      - devenv.nix
    launcher:
      exe: nix
      args:
        - run
        - github:cachix/devenv/65a7c40d185350f0e96783b3dd8d4afab3bb7034#devenv
        - --

The repository's verification command remains a direct devenv subcommand:

tasks:
  test:
    command:
      exe: test
    execution:
      orchestrator:
        ref: devenv
        mode: subcommand

The long-running development path uses the same owner without being mislabeled as finite work:

tasks:
  dev:
    launch:
      kind: command
      exe: up
    execution:
      orchestrator:
        ref: devenv
        mode: subcommand

This separation gives Ota enough structure to explain and select the path while leaving devenv
responsible for the process graph it launches.

What the hosted matrix proved

The retained artifacts and logs prove that:

released Ota v1.6.26 loaded and validated the immutable contract on Ubuntu and macOS
Ota resolved nix as the host requirement and the pinned devenv invocation as orchestrator truth
task discovery preserved finite test, aggregate verify, and long-running dev as distinct lanes
Doctor preserved the repository-hygiene warning and not_ready agent verdict
all advertised native lanes admitted dry-run on both operating systems
the Ubuntu job executed ota run test --native --stream and Ota reported success after devenv returned exit code zero
the Ubuntu run stopped the devenv-managed processes before Ota emitted its successful run summary

This is execution evidence for one selected Ubuntu task. It is not runtime proof, lifecycle proof,
or evidence that every repository behavior was governed.

The green run exposed a real boundary

During the Ubuntu task, devenv logged that backend-migrate failed because
cloud.devenv.toml was missing. Other work continued, the frontend build completed, and devenv
eventually printed Tests passed :) and returned zero. Ota therefore reported the selected task as
successful.

That is faithful to the current contract: Ota invokes the selected orchestrator operation and
uses its terminal result. It does not independently reinterpret every nested devenv process log.
The matrix consequently proves that Ota executed the declared boundary, not that every internal
devenv process completed successfully.

The task also modified four tracked files:

devenv.lock
frontend/elm-srcs.nix
frontend/generated-api/.openapi-generator/VERSION
frontend/generated-api/README.md

Those mutations are part of the observed repository behavior. The run did not assert that the
worktree remained clean, so the note must not turn its green status into a non-mutation claim.

Uncovered material behavior

The pressure result classifies the remaining behavior explicitly.

Contract-owned and proved

native contract validation on Ubuntu and macOS
task, workflow, and execution-topology discovery
Doctor and Doctor fix-preview output, including the retained not_ready agent verdict
native dry-run admission for test, verify, and dev
real Ubuntu execution of the selected test task through the pinned Nix/devenv launcher

Explicitly bounded as not proved

real task execution on macOS
successful completion of every nested devenv process
lifecycle readiness and teardown assertions for dev
container and Windows execution
deployment, production readiness, and repository-global governance
a clean worktree after verification

Repo-owned outside this pressure slice

the internal devenv process graph and its terminal acceptance semantics
whether generated files should be committed, ignored, or checked for drift
the missing cloud.devenv.toml configuration expected by backend-migrate

Ota platform gap

Ota does not currently receive structured nested-process outcomes from the devenv adapter. It can
govern the selected orchestrator operation and report its exit status, but it cannot prove that all
material child processes succeeded when the orchestrator itself returns zero. A stronger future
carrier would need typed nested outcome evidence rather than log scraping.

Why this pressure case matters

The result is useful precisely because it is not a staged perfect success.

Ota correctly preserved the Nix/devenv ownership boundary and executed the selected finite lane.
The same run then showed where that evidence stops. That distinction is the product: a green outer
command should never silently expand into a claim about nested work Ota did not independently
witness.

Links

Originally Posted: https://ota.run/blog/pressure-testing-ota-on-cloud-devenv-sh-4q1n

Ota v1.6.26 is here. We’re looking for design partners.

Bobai Kato — Tue, 18 Aug 2026 19:05:38 +0000

Adamma

Aug 18

Ota v1.6.26 is here. We’re looking for design partners.

#productivity #opensource #programming #github

1 min read

Ota v1.6.26 Now Available: Protected Execution Foundations and Receipt History

Bobai Kato — Mon, 17 Aug 2026 13:29:24 +0000

Idea

v1.6.26 closes the bounded V11.7 OSS audited-crossing slice.

Prior releases built execution governance: what should run, what closure to consume, when replay is allowed, whether cleanup was witnessed. The next boundary was authority: who decides whether work executes, and how do downstream consumers verify that decision independently?

v1.6.26 addresses this through the Linux systemd protected launcher. The release does not attempt provider attestation or cross-organization trust separation. Instead, it establishes a bounded first-class path where:

the contract specifies whether execution is allowed or explicitly disabled
the launcher independently attests each boundary before Ota processes an authorization request
the crossing evidence chain becomes explicit and re-verifiable through receipt history
receipt history immutably records each decision and persists independent archives
operator-driven recovery preserves exact evidence through failures and reboots

The evidence is locally-bounded: independently administered Linux/x64 systemd production pressure proves the consumer-only positive path, while separately verified crash/reboot recovery preserves evidence through failures. Provider attestation remains open.

Feature

1. Closed the bounded V11.7 OSS audited-crossing slice

v1.6.26 completes the production-client and independently administered hardened-launcher acceptance branches.

Immutable Linux/x64 PID 1 runs 31939777636 and 31953535665 prove:

Production client with protected receipt history, one-use selected execution, and terminal cleanup
Exact archive reconciliation and administrator-driven reboot/fault recovery
Repository state unchanged, complete terminal cleanup, zero invalid protected archives

Provider attestation remains optional stronger hardening rather than an implied V11.7 requirement. Contract-authored governance.crossing_requirements is explicitly deferred to follow-on work.

2. Production protected receipt-history Core surface

ota receipt --history --source systemd_protected_launcher adds immutable receipt history anchored to the Linux systemd launcher:

ota receipt --history --source systemd_protected_launcher --json
ota receipt --history --source systemd_protected_launcher --archive-identity sha256:<hash>

The bounded manifest binds:

Admitted non-agent operator profile and live peer
Repository and catalog identities
Three content-addressed objects per entry: receipt archive, immutable contract snapshot, launcher-finalization sidecar

Core reconstructs exact bytes and applies existing semantic archive verifier. Optional --archive-identity selects one exact archive without exposing protected paths.

Local history remains default and now publishes explicit source and completeness posture in text and JSON.

Immutable Linux/x64 PID 1 run 31823037642 proves the installed production invocation client and protected-history source against exact Protocol, Core, and Launcher revisions with one valid protected archive, zero invalid archives, unchanged refusal worktrees, and no private signing material.

3. Execution-disabled systemd-launcher posture gate

The release adds bounded execution-disabled foundations through the protected launcher.

Core verifies signed V3 systemd launcher and job-principal attestation, but the launcher deliberately withholds authorization before exact scope, cgroup, child, and active-slot cleanup. The result is:

posture_admitted_before_authorization_boundary_removed typed refusal
Zero selected work, lease consumption, receipt, or archive
Exact terminal cleanup preserved

Immutable Linux/x64 PID 1 runs 31561247605 and 31389237232/31389713244 prove execution-disabled allowed, denied, stale, wrong-scope, protected-installation/runtime drift, missing-credential, and crash-recovery paths with independently re-verifiable public signed decision/relay identities.

4. Protected systemd V3 one-use lease consumption

After exact admission, the launcher atomically consumes a signed lease from the broker before spawning work:

One-use leases are consumed only once; reuse is cryptographically refused
Core binds pending transaction posture to private persistence
Launcher persists consume intent before relay and records signed response

Immutable Linux/x64 PID 1 systemd run 31631358796 proves one-use consumption, identical-lease already_consumed refusal, crash recovery, byte-identical repository state, and zero terminal slots/scopes.

5. Selected execution and receipt archives

Once the lease is consumed, Core may execute only the frozen work unit:

ota run <task> --grant <crossing_authority_id>

The launcher persists completion evidence before Core exits, verifies child reaped and scope removed, and persists launcher-owned finalization. Core then independently verifies both launcher signatures and every identity relationship.

Immutable Linux/x64 PID 1 pressure run 31664495937 proves completed, failed, interrupted, replay-refused, and crash-recovered selected execution with exact child, scope, cgroup, and active-slot cleanup.

V3 receipt archives accept only launcher-active-slot transactions. Receipt history preserves the carrier distinction and refuses to reinterpret legacy broker or signed-file carriers as V3.

6. Portable launcher finalization

v1.6.26 adds the portable protected-systemd launcher-finalization path.

New launcher-owned crossing transaction schema v3 is bound into the signed consume exchange and requires broker-archive schema v2 plus portable finalization verification. Historical transaction v2 and broker-archive v1 evidence retain original compatibility.

The launcher retains protected post-cleanup recovery state until the client acknowledges a producer-signed sidecar binding exact cleanup evidence, receipt-archive identity, and crossing transaction. Core durably publishes the exact execution receipt archive before emitting launcher completion. The root launcher reopens that private archive through the execution-principal repository descriptor, requires exact owner, content, and transaction identity, and atomically publishes the root-owned sidecar; the job principal only acknowledges it.

Signed profile bindings include CAP_DAC_OVERRIDE in the exact bounding set, solely so the root launcher can traverse private hierarchies inside its protected mount namespace.

Immutable PID 1 crash pressure and the production operator client are included in the released
bounded carrier; stronger provider-attested separation remains a follow-on boundary.

7. Honest launcher crash-recovery evidence

Live systemd finalization keeps directly observed exit and child-reaped posture; restart recovery uses signed finalization schema v2 with recovered_absent_completion_bound, verified child absence, and no claimed observed exit or reaping. Receipt history re-verifies either exact version without upgrading legacy evidence.

Immutable Linux/x64 PID 1 run 31758094819 proves the corrected portable-finalization path, one-use consumption, exact crash recovery, and valid archive history.

8. Refined active-execution admission

v1.6.26 refines native service listener conflicts around actual resource ownership:

Fixed or Ota-managed dynamic listeners: Long-running tasks bind fixed, Ota-managed dynamic, isolated, or unresolved runtime listeners across their complete execution closure, including hooks.
Native and container coexistence: Services can coexist when their projected host endpoints and effective write namespaces are disjoint.
Ancestor/descendant path conflicts: Shared write and env-materialization paths now conflict on ancestor/descendant overlap, not exact text only.
ota run <task> --host-port <port>: Directs native service execution. Container and Compose lanes remap only the host publication; direct native lanes apply the port to both canonical bind and projected host endpoint. Ota reprojects typed launch arguments and runtime environment values.
Clear conflict reporting: "Host port already in use" now names requested and active execution modes plus endpoint owner, suggests truthful --host-port <free port> rerun, and preserves existing run-summary layout with additive Reason and Host port rows.

9. Fixed interactive native task closures

The release fixes interactive native task closures so typed hydration and bootstrap phases retain Ota's canonical 🦦 loader instead of inheriting terminal ownership from a later interactive command.

On Unix, Ctrl-C now terminates Ota's complete native child process group and waits for it to settle before emitting the interrupted summary, preventing late npm output after Ota reports completion.

Native task progress loaders now show selected execution class explicitly, for example Running setup:dev (native), matching existing container and remote labels.

10. Schema embedment and version pinning

v1.6.26 removes the source checkout's absolute CARGO_MANIFEST_DIR from production schema discovery. Published JSON schemas are now embedded into the Ota binary, so installed or relocated source builds validate machine output against their exact version-matched schema set without a source checkout or compiler-path dependency.

The release also emits the full 40-character source commit from ota --version --json so protected deployment evidence can bind the installed Core binary to one exact source revision.

11. Wildcard tool-version handling

Treat wildcard (*) task tool and runtime requirements as executable-availability requirements when a present command does not expose parseable version output. Pinned requirements still fail closed when Ota cannot establish the installed version.

This keeps POSIX sh tasks portable to Debian and Ubuntu systems where dash --version exits nonzero without weakening version pins.

12. Compatibility-preserving profile support

Add compatibility-preserving support for the separated-producer ota.authority-launcher.systemd/v2 profile. V3 broker bindings and the published receipt schema require an exact registered profile-ID/identity pair. Legacy V1 evidence remains verifiable but cannot be relabelled as V2. Complete ota.authority-launcher.systemd/v3 and ota.authority-job-principal.systemd/v2 verification branches re-derive ordered launcher and job-principal observations, nested identities, protected socket source, and limited-primary-group posture from signed evidence.

Boundaries

The most important v1.6.26 behaviors are what it qualifies and refuses to imply:

execution-disabled posture proves no work runs, not that the launcher is trustworthy
protected receipt history proves exact archive binding, not system-level secret protection
launcher crash recovery preserves evidence through failures, not immunity from crashes
locally-bounded independently administered pressure proves consumer-only operation, not provider attestation or cross-organization trust
V3 receipt archives accept only launcher-active-slot transactions; legacy carriers remain in their original branches
one-use lease consumption prevents duplicate execution within a transaction; it does not prevent replay across different invocations
a successful protected execution does not prove downstream deployment or production health

Those are not disclaimers around the product. They are part of the product contract.

Docs

Use the live references for protected execution, receipt history, and crossing-evidence semantics:

Release

Install or upgrade to the released version, then inspect the contract and protected-execution surface:

ota upgrade
ota --version --json
ota validate
ota doctor --json
ota tasks --safe --use

For protected execution and receipt history:

ota receipt --history --source systemd_protected_launcher --json
ota run <task> --dry-run --json
ota run <task> --grant <crossing_authority_id> --dry-run --json

The canonical release and complete patch-level changelog are available at
ota.run/releases/v1.6.26.

v1.6.26 closes the bounded V11.7 OSS audited-crossing slice through locally-bounded protected-launcher foundations: not only what should run and what proof exists, but which boundaries admitted it, how launcher isolation prevented unauthorized work, and what immutable receipt archives capture each decision through recovery scenarios. Provider attestation and cross-organization trust separation remain open for follow-on work.

Pressure and Evidence

The entire protected-execution stack was shaped by production pressure on isolated Linux/x64 systems:

Installation and runtime drift: Invalid credentials and launcher-startup failures preserve zero selected work.
One-use lease consumption: Identical lease-reuse is cryptographically refused; fresh invocations require fresh authorization.
Crash recovery and restart scenarios: Launcher crash pressure and administrator-driven reboot recovery preserve exact evidence through multiple transitions.
Execution-disabled posture: Core admits the contract, but the launcher withholds authorization and emits a typed refusal with exact scope, cgroup, and child cleanup.
Receipt history and archive verification: Protected receipt history verifies exact archive, contract snapshot, and finalization sidecar identities without exposing private paths.

All evidence is immutable Linux/x64 hosted proof bound to exact Core, Launcher, and Protocol revisions. Independently administered Linux/x64 PID 1 production pressure (runs 31939777636, 31823037642, 31953535665) proves the consumer-only positive path and administrator-driven recovery branches.

Provider attestation and cross-organization trust remain open as stronger follow-on boundaries. The
independently administered launcher, portable finalization, and operator attachment surfaces are
part of the released bounded carrier.

The complete bounded V11.7 OSS slice is live. Follow-on work includes contract-authored governance.crossing_requirements, provider-attested separation, and broader multi-system authority delegation.

Originally posted here: https://ota.run/blog/ota-v1-6-26-release-essay

Pressure-testing Ota on Flagr: native and container Go verification with honest boundaries

Bobai Kato — Fri, 14 Aug 2026 23:06:06 +0000

Overview

Flagr is a useful test of whether Ota can keep a real service repository legible. It has a portable Go verification path, a local SQLite-backed API, a six-instance Docker Compose integration topology, four database services, and a locally tagged integration image. Those are different execution truths. One green test command should not flatten them into one readiness or deployment claim.

The pressure contract pins released Ota v1.6.25, separates deterministic verification from runtime and integration work, and gives the Compose topology a runner-owned lifecycle proof.
It also removes Windows from the selected verification surface rather than carrying an audit-only lane that could be mistaken for execution support.

Why this repo mattered

Flagr combines several patterns that show up in mature service repositories:

a finite Go verification lane;
a long-running local service with a declared HTTP surface;
an integration topology that needs an image build, Compose startup, readiness, test execution, and teardown; and
CI that needs to distinguish a safe verification lane from deliberately external-state-mutating integration work.

The pressure question was not merely whether go test could pass. It was whether the contract could own hydration, execution mode, runtime readiness, image construction, service dependencies, failure cleanup, and agent refusal without hiding the important transitions in CI shell.

What the contract models

The deterministic lane uses typed Go module hydration in native and container contexts. Linux and macOS own the native verification path; Linux also owns the container path through golang:1.26-bookworm. The selected Go closure is intentionally unavailable on Windows.

The integration image is not a raw docker build string:

integration:image:
  description: Build the locally tagged image consumed by the six-node Compose integration topology
  category: build
  action:
    kind: build_container_image
    file: integration_tests/Dockerfile-Integration-Test
    context: .
    tag: flagr_integration_tests

The workflow split stays explicit:

verify is deterministic Go package verification;
local owns the SQLite-backed API runtime;
integration owns the image, Compose topology, readiness, integration suite, benchmark, and explicit teardown;
integration:lifecycle gives Ota ownership of service start, assertion, teardown, and archive evidence;
integration:lifecycle:failure-control intentionally fails the real assertion and verifies that teardown still completes; and
agent refusal canaries exercise the real boundary around the unsafe image task and integration workflow.

The lifecycle model includes the Flagr services and their MySQL and PostgreSQL dependencies. Each managed service declares manager_inactive as its teardown assertion. Readiness failure is not accepted as proof that a service stopped; the Compose manager must report the terminal state.

What Flagr exposed in Ota

Flagr exposes several real Ota boundaries without needing repository-specific workarounds.

Aggregate verification needed to remain visible as executable closure support in task discovery.
Doctor also had to stop probing repo-owned commands before their producer task had created them.
Those are ordering and discovery problems in the runner, not defects in a Go repository.

The CI pressure exposed another distinction: a finite workflow cannot stop after a readiness-oriented ota up. Ota's provider-neutral CI projection distinguishes a finite task closure from a service-runtime path. The finite lane executes its selected task; runtime proof owns
its own execution path.

That is why the image build matters. Ota owns the action, Dockerfile, context, local tag, and evidence surface. Flagr still owns what the image means and how its integration system behaves.

The lifecycle lane closes the remaining shell-shaped gap. Starting Compose, probing six Flagr endpoints, running an assertion, and calling down in a final CI step showed that readiness was proved but transaction-owned cleanup was not. Ota's managed lifecycle surface now acquires cleanup authority before start, runs the declared assertion, finalizes each managed service, and archives the bounded result. The failure control matters because successful cleanup alone would not prove that the same finalization path survives an assertion failure.

Matrix design

The released matrix is deliberately split by evidence type:

Lane	Platform and mode	Evidence
Managed `verify` projection	Linux native	Contract validation, Doctor, safe discovery, dry-run admission, real verification, receipt archive, and two refusal canaries
Managed `verify` projection	macOS native	The same finite verification and refusal boundary on the second supported host OS
Managed `verify` projection	Linux container	The same selected verification closure inside the contract-owned Go image
Native runtime proof	Ubuntu and macOS	Bounded execution of the SQLite-backed local API and its declared health surface
Container runtime and integration	Ubuntu	Container runtime proof plus the six-instance Compose suite, benchmark, receipt, and explicit teardown
Managed lifecycle pressure	Ubuntu	Successful lifecycle archive plus an assertion-failure control that must still finalize every transaction-owned service
Contract-to-CI drift	Ubuntu	The first-party Action checks that CI consumes contract-owned bootstrap and verification truth

The contract and generated workflows bind bootstrap to released v1.6.25. All three supported managed projections are generated from the final contract, and their human-owned caller bindings are checked against the same projection identities before hosted execution.

The complete hosted governance matrix is green in
run 31834610141 at commit 58f5263a. It executed Linux and macOS native verification, Linux container verification, both refusal canaries in every generated lane, native runtime proof on Ubuntu and macOS, container runtime proof, the Compose integration suite and benchmark, explicit teardown, and the contract-to-CI drift gate.

The managed lifecycle lane is independently green in
run 31834610099. The successful transaction acquired ownership for and finalized ten services, passed the real integration assertion, and observed every declared teardown state. Its separate failure control sent the same assertion to an unreachable endpoint, recorded the expected non-zero outcome, and still completed finalization for all ten services.

That lifecycle result is deliberately passed_with_unproven_boundaries, not a bare pass. The four
database services do not declare readiness or another manager-backed start-state observer, so service_started_state_not_proved remains attached to each database obligation. Application output and broader repository completion also remain explicitly unproved.

The distinction between the lanes is part of the result. A refusal canary is not a successful integration run; it proves that the real runner still rejects that path in agent mode. Runtime readiness is not lifecycle finalization. Lifecycle finalization is not application correctness or production deployment evidence.

Doctor stayed honest

The released contract validates with no errors, but Doctor intentionally retains two classes of warning:

the agent-safe verification closure performs declared Go dependency hydration over the network; and
the lifecycle closure mutates Docker, MySQL, and PostgreSQL external state.

Those warnings are not noise to remove. They tell an operator that repository write boundaries do not govern registries, the Docker engine, or database state. The managed verification lanes remain
agent-admitted because the network effect is typed as dependency hydration. The integration and lifecycle lanes remain outside agent-safe admission.

Uncovered material behavior

The pressure result is bounded as follows:

Contract-owned and exercised

typed Go module hydration for the selected native and container verification modes;
deterministic Go package verification on Linux, macOS, and Linux container execution;
SQLite-backed local API runtime and HTTP readiness;
the locally tagged integration-image build;
the selected six-instance Compose integration suite and benchmark;
successful and assertion-failure lifecycle finalization; and
agent refusal canaries plus contract-to-CI drift detection.

Explicitly outside this proof

the UI build, browser E2E lane, actionlint job, dependency-freshness policy, and broader upstream benchmark surface remain repo-owned CI behavior outside this selected Ota pressure contract;
GitHub triggers, permissions, credentials, runner policy, and artifact retention remain provider-owned;
release publication to GHCR, multi-architecture release images, and production deployment remain outside Ota's selected execution scope; and
external production services, production data, and deployment credentials were neither selected nor proved.

Ota platform gaps

This pressure lane exercises shipped Ota surfaces without adding a repo-local workaround. Any hosted failure must be classified before editing: contract defect, upstream Flagr behavior, provider failure, or a new Ota platform gap. A green matrix does not erase the explicit boundaries above.

What this did not prove

The matrix does not prove Windows execution. The contract prevents that over-read by excluding the selected verification tasks from Windows rather than publishing a non-executing audit lane.

The selected integration topology is real local proof, but it is not evidence that every deployment environment has the same topology or that production operations are governed by this contract. The lifecycle archive proves only the selected Compose transaction and its declared teardown assertions. It does not prove Flagr's application outputs, repository-wide correctness, or external
provider policy. It also does not promote the four database services to observed started state; their command execution and terminal manager state are evidenced, while their start-state obligations remain bounded as not_proved.

Hosted result

The hosted governance and lifecycle matrices are green on the same pressure commit and install Ota from the contract's released v1.6.25 source. Local validation also passed Doctor, discovery,
native and container dry-run admission, real native and container verification, refusal canaries, managed projection checks, and workflow linting. The hosted runs, not that local result, are the
publication evidence.

Test What Your AI Agents Must Not Do

Bobai Kato — Wed, 12 Aug 2026 14:19:58 +0000

A Guardrail Without A Negative Test Is Still An Assumption

Most AI agent governance starts with the positive path.

A repository tells an agent which task it may run. The agent invokes that task. The test suite
passes. CI goes green.

That proves one useful thing: the allowed path still works.

It does not prove that the forbidden path is still forbidden.

A stale allowlist, an incomplete dependency closure, a runner regression, or a hand-maintained CI
condition can quietly widen what an agent is able to execute. Every positive check may remain green
while the safety boundary has stopped doing its job.

This is why agent guardrails need negative tests.

Ota calls those tests refusal canaries.

They are available in Ota v1.6.25, and they reflect a deliberate product position:

Agent safety should be executable contract truth, not a promise that the model will remember to
behave.

Ota uses the same ota.yaml contract to define the safe task surface, evaluate the complete
selected closure, refuse unsafe execution, emit machine-readable evidence, and project the canary
into CI. There is no separate shell rule for the negative test to drift away from.

That is the difference between documenting a guardrail and operating one.

What A Refusal Canary Tests

A refusal canary names a task or workflow that must remain outside the repository's agent-safe
execution boundary.

For example, a repository may allow agents to run verification while reserving publishing for a
human-controlled path:

agent:
  safe_tasks:
    - verify
  refusal_canaries:
    - task: publish
    - workflow: release

tasks:
  verify:
    command:
      exe: pnpm
      args: [test]
    safe_for_agent: true

  publish:
    command:
      exe: pnpm
      args: [publish]
    safe_for_agent: false

workflows:
  release:
    intent: release
    run:
      task: publish

The positive check proves that the allowed lane remains callable:

ota run verify --agent

The refusal canary tests the opposite boundary:

ota run publish --agent --expect-refusal
ota up --workflow release --agent --expect-refusal

These commands do not run the unsafe task and then inspect its exit code. Ota exercises the same
agent admission boundary used by ordinary execution and requires refusal before the selected task,
its dependencies, or the workflow closure starts.

That distinction matters. The canary tests the execution boundary, not the behavior of a dangerous
command after it has already begun.

Expected Refusal Becomes A Passing Test

An ordinary agent refusal is a blocked execution result. That is the correct behavior for a direct
attempt to run an unsafe task, but it is inconvenient as a CI assertion because the shell sees a
non-zero exit.

--expect-refusal gives that result explicit negative-control semantics:

refused_as_expected means the declared agent-safety boundary refused the selected target before execution and the canary passes;
refusal_not_observed means the target was admitted, so the canary fails; and
wrong_refusal_boundary means execution was blocked for another reason, so the canary still fails.

The third state prevents an important false positive.

Suppose publish accidentally becomes agent-safe, but an unrelated organization policy still
denies the command. A weak negative test would see "something failed" and call the guardrail
healthy. Ota does not. The canary passes only when the safety boundary it was created to test is
the boundary that refused the lane.

For machine consumers, the same result is available as structured JSON:

ota run publish --agent --expect-refusal --json

The output carries the canary target, the runner-authored refusal record, whether execution
started, and the associated blocked receipt when refusal was observed. The agent does not
self-report that it behaved safely.

This is where Ota is intentionally more opinionated than a task runner. A task runner answers,
"Can I invoke this command?" Ota also asks, "Should this actor be admitted to this complete
execution closure, and what evidence should exist when the answer is no?"

Why This Must Run Through The Real Runner

It is easy to imitate a refusal canary with shell logic:

if [ "$AGENT_MODE" = "true" ]; then
  exit 1
fi

That only proves the shell condition still exists.

It does not prove that Ota's selected task closure, safety declaration, dependency expansion, and
execution admission agree. It can also drift independently from local execution, turning CI into a
second policy system.

A useful refusal canary has to pass through the same chokepoint as the real action. Otherwise the
test and the boundary can fail independently while both appear green.

Ota therefore derives the refusal from current contract truth at execution time. Authors declare
which task or workflow is the canary; they do not author the reason that should make it pass.

Put Refusal Canaries In CI

Local refusal gives agents and developers fast feedback. CI turns the same boundary into a
repository-level control.

A mature governance lane should test both directions:

At least one meaningful safe verification path is admitted and completes.
Each material unsafe path selected as a canary is refused by the agent boundary.

Ota's provider-neutral CI projection includes each declared refusal canary. The GitHub adapter
emits each one as its own provider check with a stable merge-check identity. That lets a repository
make the negative control merge-required instead of hiding several boundaries inside one opaque
script step.

This is stronger than one broad "agent safety" job. If the publish canary fails while verification
still passes, the provider shows exactly which execution boundary widened.

Branch protection remains provider-owned. Ota can generate and identify the GitHub check, but the
repository's GitHub policy must make it required if the organization wants a non-optional merge
gate. Future provider adapters must preserve the same ownership boundary rather than pretending
Ota controls provider policy.

What Refusal Canaries Prove

A passing refusal canary proves a narrow and valuable claim:

At this contract snapshot and runner boundary, the selected task or workflow was refused for the
expected agent-safety reason before execution started.

It does not prove:

that every dangerous repository action has been declared;
that the maintainer classified every task correctly;
that raw shell execution outside Ota is impossible;
that CI branch protection requires the canary check;
that runtime filesystem, network, credential, or provider controls were enforced; or
that the repository is globally safe for autonomous operation.

Those are different obligations.

Contract-claim assurance can evaluate whether observable repository evidence supports a declared
safety claim. Sandbox and capability providers can enforce runtime controls. CI policy can make
checks mandatory. Refusal canaries test whether one declared runner boundary still refuses what it
is supposed to refuse.

Keeping those claims separate is what makes the result trustworthy.

Turn Incidents Into Permanent Negative Tests

A refusal canary protects a boundary the repository has already identified. It cannot protect an
omitted path merely because that path reaches a similar outcome.

If publish is refused, Ota has proved that the selected publish closure was refused. It has not
proved that another task, migration, helper script, or raw shell command cannot reach the same
external effect. Presenting task-level refusal as effect-level protection would turn a precise
negative test into a broader claim the evidence does not support.

The practical operating model is a ratchet: when an incident or review exposes another material
execution path, add that real task or workflow as a permanent canary. The incident should improve
the executable governance suite, not remain only in a retrospective document.

Ota's current refusal canaries are task- and workflow-bound. Effect-level assurance across
equivalent execution paths is a separate, planned governance boundary; it is not something this
feature claims to provide today. Until that boundary is implemented and pressure-tested, canary
coverage should remain explicit about which declared lanes it exercises and which equivalent paths
remain unproved.

Choosing Useful Canaries

Do not turn every non-agent-safe helper into a canary. Choose actions whose accidental admission
would materially weaken the repository's governance story.

Good candidates include:

package publishing and release workflows;
production deployment;
destructive database operations;
credentialed external mutations;
infrastructure apply or teardown tasks; and
expensive or metered execution that should require a different authority path.

The canary should name the real contract task or workflow. Do not create a fake unsafe command
whose only purpose is to be refused; that proves the fixture, not the material repository boundary.

Also keep the positive lane. A system that refuses everything is not safely governed execution. It
is unavailable software. The useful result is selective admission: routine work remains easy, and
materially different work remains outside the agent-safe closure.

Try It In Your Repository

The fastest way to understand refusal canaries is to make one fail deliberately on a temporary
branch.

First, install Ota, then inspect the repository before changing its
contract:

ota doctor
ota tasks --use
ota tasks --safe --use

Add one real unsafe task or workflow to agent.refusal_canaries. Choose publishing, deployment,
infrastructure mutation, or another material lane that already exists in the repository. Then
validate both directions:

ota validate
ota run verify --agent
ota run publish --agent --expect-refusal --json

Now temporarily make publish agent-safe. The positive verification lane can remain green, but the
refusal canary changes to refusal_not_observed and exits non-zero. That is the failure shape the
feature exists to expose: the repository still works, but its agent execution boundary has widened.

When the local contract is correct, inspect the provider-neutral CI projection:

ota ci projection \
  --workflow verify \
  --mode native \
  --target-os linux \
  --json

For GitHub Actions, Ota can render the governed reusable workflow from that projection:

ota ci github render \
  --workflow verify \
  --mode native \
  --target-os linux

The repository keeps ownership of triggers, permissions, secrets, runners, environments, and
branch protection. Ota owns the contract-derived execution and refusal checks. That boundary lets
teams adopt Ota without replacing their CI provider or creating another hand-maintained workflow
language.

The Larger Point

AI governance cannot stop at telling agents what they may do.

It also needs tests for what they must not be allowed to execute.

Positive tests prove that an allowed path still works. Refusal canaries prove that a selected
forbidden path still stops at the runner boundary. Together they turn an agent allowlist from
static metadata into executable governance.

That is the standard Ota is building toward: boundaries that are declared once, exercised through
the real chokepoint, and retained as evidence rather than trusted as prose.

If your repository already tells agents what they may run, the next useful question is simple:

Which one unsafe action would you want CI to prove they still cannot run?

Go Deeper

Originally posted here: https://ota.run/blog/test-what-your-ai-agents-must-not-do

Pressure-testing Ota on nopCommerce: managed .NET verification across native and container CI

Bobai Kato — Tue, 11 Aug 2026 11:34:53 +0000

The question

nopCommerce is a useful test of where Ota should
stop owning CI. Its .NET repository has substantial provider-owned workflow and deployment policy;
Ota should not turn that into a second CI language. The pressure question was narrower: can Ota own
a restore-and-build verification lane while GitHub retains triggers, runners, permissions,
environments, and release policy?

The setup task expresses restore as typed dependency hydration, then the workflow runs a finite
aggregate verification task:

setup:
  prepare:
    kind: dependency_hydration
    medium: package_dependencies
    source:
      kind: dotnet_restore
      cwd: src/Tests/Nop.Tests

verify:
  aggregate:
    tasks:
      - build

workflows:
  verify:
    setup:
      task: setup
    run:
      task: verify

The pressure result

The contract declares native and container execution for the same workflow. Ota projects each into
an Ota-owned reusable workflow, binds it to a target OS and projection identity, and asks the
human-owned GitHub caller to supply that identity. The caller is an adapter, not a second authority.

Moving the contract from the old implementation branch to released v1.6.25 changed both
projection identities. The old caller refused immediately. That was the useful failure: it made
stale governance YAML visible before any verification task ran. We regenerated the native and
container projections with the released Ota binary, updated the caller bindings, and re-ran them.

The result is two linked release-backed proofs at commit
409bcc073d:
the contract matrix installed
Ota v1.6.25, executed restore and build on native and container Linux, and dry-ran the selected
surface on macOS and Windows. The managed governance run
verified both regenerated identities before executing the finite task and archiving its receipt.

This is the ownership split Ota needs: the contract owns restore, verification, admission, mode,
and projection identity; GitHub retains its provider policy. Neither side has to duplicate the
other's truth.

Why the refusal matters

Bootstrap truth is execution truth. Changing an Ota source from an implementation branch to a
release changes the engine that interprets the contract, so it must change the projected lane's
identity too. Treating that as a cosmetic edit would let a caller keep asserting governance that it
was no longer able to reproduce.

The initial refusal made the mismatch actionable: no task ran, the expected identity was reported,
and ota ci github sync could regenerate the Ota-owned files from the same release that CI would
install. That is a better failure mode than a hand-maintained workflow silently drifting until a
toolchain or execution change reveals it later.

Why the split is practical

This is not an argument that all GitHub workflow YAML should disappear. nopCommerce still needs
provider-owned decisions that are specific to the repository: when to run, which runner to use,
what credentials are available, and how releases are protected. Those are not contract defaults.

What Ota can own without overreaching is the governed lane inside that caller: the selected
workflow, its toolchain fulfillment, native or container mode, agent admission, and the stable
check identity branch protection can require. That leaves a small human-owned adapter instead of
two independently authored execution plans.

Boundary

This is a restore-and-build baseline, not full nopCommerce proof. The modeled test task was
admitted and dry-run but not executed. Database-backed behavior, frontend workflows, runtime
readiness, Windows execution, and deployment remain outside the claim. The container image uses a
tag rather than a digest, so the container result is run-specific execution evidence, not immutable
image provenance.

Pressure-testing Ota on Azure SDK for .NET: typed NuGet hydration across ephemeral containers

Bobai Kato — Mon, 10 Aug 2026 14:29:16 +0000

Overview

Azure SDK for .NET is a large multi-service repository. Its contributor, test, API-compatibility,
code-generation, live-resource, and release paths are not one runnable surface.

The Ota contract therefore owns one useful contributor slice and names it plainly:

restore the dependencies for sdk/core/Azure.Core through the repository's NuGet.Config;
build that library with --no-restore;
execute the same sequence in native mode and in a fresh .NET SDK container.

That is enough to pressure a real boundary without pretending that a green build means the Azure
SDK repository, its services, or its releases are ready.

The Contract

The repository owns package-source policy. Ota owns the typed restore operation that consumes it:

tasks:
  setup:
    prepare:
      kind: dependency_hydration
      medium: package_dependencies
      source:
        kind: dotnet_restore
        cwd: sdk/core/Azure.Core/src
        config_file: ../../../../NuGet.Config

The later build is deliberately no-restore:

tasks:
  build:
    depends_on: [setup]
    command:
      exe: dotnet
      args: [build, --no-restore, --framework, net10.0]
      cwd: sdk/core/Azure.Core/src

This makes the boundary testable. If restore state does not reach the build, --no-restore fails
instead of silently going back to the network.

Ephemeral Container Cache Ownership

The selected container context is ephemeral, so restore and build do not share a long-lived
container. The contract declares the NuGet cache as Ota-managed attachment state:

execution:
  contexts:
    dotnet:container:
      backend: container
      lifecycle: ephemeral
      container:
        image: mcr.microsoft.com/dotnet/sdk:10.0.103
      attachments:
        isolated_paths:
          - .nuget/packages

Ota derives NUGET_PACKAGES for the selected context and carries that cache across the restore and
later no-restore build containers. The repo does not need to duplicate a host cache path or a
Docker volume name in a helper script.

Azure SDK exposed two Ota corrections while this was pressure-tested:

command tasks must mount the contract root, then apply command.cwd inside it, because restore writes shared intermediates under the repository artifacts/ tree;
typed dotnet_restore must carry the resolved NuGet package cache into later ephemeral tasks, not merely report that restore exited successfully.

Those are runner ownership fixes. They are not Azure-specific workflow glue.

.NET Version Truth

The contract follows global.json, which requests 10.0.103 with rollForward: feature:

toolchains:
  dotnet:
    version: ">=10.0.103, <10.1.0"

That admits valid servicing releases in the same feature band. An exact patch equality would turn
the repository's declared roll-forward policy into a false readiness failure.

Released Matrix

The requalification matrix installs Ota from the contract as released v1.6.25, verifies the
binary version before any Ota command runs, and pins all GitHub Actions by immutable revision.

The released four-lane matrix
is green. Every lane recorded ota v1.6.25 (source, commit a3e9f98f3) before running Ota.

It covers:

validate, Doctor, task/workflow discovery, and safe/unsafe task-surface discovery;
task and workflow dry-runs on Ubuntu, macOS, Windows, and Ubuntu container mode;
real restore, no-restore build, and declared workflow execution in each advertised mode;
retained version, diagnosis, contract snapshot, and archived workflow receipt artifacts on every result.

The receipts agree on one contract snapshot:
sha256:f7a96703141dc4e6cbb9626e0f893f4692d781db48113ac639c18770d972134b.
The Ubuntu container receipt records dotnet:container with ephemeral lifecycle; the other three
receipts record the native context. A historical matrix that installed an implementation branch is
not evidence for this release claim.

What It Does Not Prove

This is selected-lane evidence only. It does not prove:

every Azure SDK client library, test suite, API-compatibility check, generator, or release build;
Azure credentials, live-resource tests, external package-feed availability, or service behavior;
application correctness, package publication, or repository-wide readiness;
container execution on macOS or Windows, where those lanes exercise native contract execution;
hermetic replay or a contract-pinned expected identity for the SDK image tag.

The last point is an intentional limitation to keep visible: this contract records the configured
SDK image tag, not an immutable image digest. A mutable tag is not a maintainer-declared immutable
input, so this is not a claim of a pinned container base.

Why This Matters

The useful result is not that Ota can invoke dotnet restore.

It is that the contract declares dependency intent once, while the runner owns the workspace and
cache continuity needed to make a later no-restore build meaningful across fresh containers. The
result remains narrow, inspectable, and honest about the much larger Azure SDK surface it does not
govern.

Pressure-testing Ota on Grafana: bounded Go proof and a staged Compose slice

Bobai Kato — Wed, 05 Aug 2026 17:37:56 +0000

Overview

Grafana is a large mixed repository. Its upstream CI spans sharded backend and frontend tests,
multiple databases, end-to-end paths, release builds, and a Tilt-owned development environment.

That makes it a useful pressure target precisely because a plausible-looking single verify
command would be dishonest.

The Ota 1.6.25 contract owns two specific surfaces:

a Go package build path rooted at ./pkg/...;
a separately declared Compose dependency slice for PostgreSQL, Prometheus, Loki, and Tempo.

Everything else remains outside the claim.

The Contract

The backend proof is intentionally narrow. Ota hydrates Go modules, then runs the named package
build rather than guessing at Grafana's full sharded test matrix:

tasks:
  setup:
    prepare:
      kind: dependency_hydration
      medium: package_dependencies
      source:
        kind: go_modules

  build:backend:pkg:
    command:
      exe: go
      args: [build, ./pkg/...]
    depends_on: [setup]

workflows:
  proof:backend:pkg:
    setup:
      task: setup
    run:
      task: build:backend:pkg

The Go range is also honest about the claimed lane:

>=1.26.3 <1.27, not an artificial patch pin;
native and ephemeral-container modes for the package build;
no claim that this range proves every Grafana build or test path.

The named lane matters. build:backend:pkg says what it owns. A generic build or verify
would blur frontend, release, and backend truth in a repository where those are different paths.

The Compose Boundary

The development workflow is equally bounded. Grafana's frontend-service environment uses more than
the four observability dependencies in this contract, including application and proxy behavior
owned by Tilt. Ota therefore models only the selected Compose services:

tasks:
  devenv:observability:up:
    compose:
      kind: up
      services: [postgres, prometheus, loki, tempo]
    effects:
      network: true
      network_kind: container_image_hydration

Ota captures the Compose adapter inputs, starts only that selection, inspects it with a separate
safe status task, archives a receipt, and stops it again. This is not a claim that Grafana itself,
its frontend, its proxy, or its Tilt orchestration became ready.

Released Install Truth

The pressure matrix consumes the Ota version declared in the contract:

metadata:
  ota:
    minimum_version: "1.6.25"

agent:
  bootstrap:
    ota:
      source:
        kind: version
        version: v1.6.25

The GitHub workflow installs via ota-run/setup with source: contract, checks that the installed
binary is exactly v1.6.25, and retains the version record, topology, receipts, and proof output
for seven days on every result. The workflow actions themselves are pinned to immutable revisions.

What The Matrix Covers

The v1.6.25 requalification matrix covers:

contract validation, Doctor, task discovery, safe-task discovery, workflow discovery, and execution-topology export on Ubuntu, macOS, and Windows;
native task and workflow dry-runs across that matrix;
container task and workflow dry-runs on Ubuntu;
real native and ephemeral-container execution of the selected Go build on Ubuntu;
real start, safe status inspection, archived receipt, receipt diff, and stop for the selected observability Compose dependencies on Ubuntu.

The released-v1.6.25 requalification is green: Grafana matrix run
30965322618. All three legs
recorded ota v1.6.25 at source commit a3e9f98f3. The Ubuntu evidence also retains the Compose
receipt and its unchanged baseline comparison, including the selected image digests. That comparison
is witness-only: it does not execute a replay or establish hermeticity.

What It Does Not Prove

This is selected-lane evidence, not a Grafana-wide readiness verdict. It does not prove:

Grafana's sharded backend unit, integration, frontend, E2E, or release workflows;
application readiness, user-visible dashboards, API correctness, or migration behavior;
the Tilt-managed frontend-service application, proxy, or lifecycle;
Docker Desktop behavior on macOS or Windows;
registry availability, external services, or host capacity beyond each selected execution.

The material uncovered behavior is therefore explicitly repo-owned or unproved, not silently
absorbed into a green label.