Re: @mads_hansen — answers to your 6 comments across the audit pipeline articles

You've left 6 comments across our audit pipeline articles since July 16, each one substantive. This is a batched response — easier to read in one place than scattered across 6 articles. Your original comments are linked inline.

1. "Capability classification after install, not just package inspection before install" (Jul 16)

"A package can look clean and still expose a tool surface that deserves scrutiny: file read/write scope, network egress scope, credential access, tenant or workspace boundary, mutation vs read-only actions, approval requirements, result data sensitivity, rate limits and retry behavior."

Agreed, and the ATC spec now mandates this as a required field. ATC-003 Capabilities requires every card to declare:

• filesystem.read / filesystem.write (none | own_dir | temp_dir | home_dir | system | all)
• network.egress (none | allowlist | all)
• network.ingress (none | bound_ports | all)
• shell.exec (none | sandboxed | unrestricted)
• credentials.read_env (none | allowlist | all)
• process.subprocess (none | sandboxed | unrestricted)

Spec: https://github.com/alicelabs-llc/universal-trust-adapter/blob/main/marketnow/docs/atc-spec/SPEC.md

2. "Static scanning catches the supply-chain blast radius. Runtime policy catches the agent blast radius. You need both." (Jul 17)

The split you described is now explicit in the architecture. Static layers (L1-L7) handle supply-chain. The post-execution filter at https://github.com/alicelabs-llc/universal-trust-adapter/blob/main/uta-monorepo/packages/gateway/src/post-exec-filter.ts handles runtime. It runs after each tool call, inspects the result + side effects, and blocks if the tool catalog changed since approval.

3. "Distinguish periodic re-attestation from runtime monitoring" (Jul 21)

"A weekly sandbox replay can detect artifact, catalog, permission, and observed-behavior drift; it cannot see an attack that runs between scans, activates only for a real tenant, or depends on a production credential and data shape."

Right — and I should have framed L3 that way from the start. L3 is the periodic re-attestation layer (weekly sandbox replay). The exec-boundary deny-gate (gate.cat or equivalent) is the runtime layer. They compose, they don't replace each other.

4. "Be careful calling it a firewall until detection quality is measured" (Jul 23)

"Several rules are legitimate language in security/admin tools: 'execute system commands,' 'read .env,' urgency words, or conditional 'when X, do Y.' A raw match count can create both false positives and easy evasion. I would publish a labeled corpus with benign skill metadata, known attacks, paraphrases, multilingual/Unicode variants, split-token payloads, and nested instructions, then report precision/recall by rule family rather than only the number of rules."

Done. The labeled corpus is now public: https://github.com/alicelabs-llc/universal-trust-adapter/tree/main/uta-monorepo/packages/gateway/src/vectors/prompt-injection-corpus

Precision/recall by rule family will follow once I have enough paraphrase + multilingual variants to make the numbers meaningful. Currently the corpus has labeled benign + known attacks; the paraphrase/split-token/multilingual expansions are tracked as the next addition.

MITRE ATT&CK mappings are versioned in the corpus file with rationale per technique ID, not as decorative compliance metadata.

5. "Compromise recovery depends on revocation distribution" (Jul 19)

"A verifier with a cached registry can keep accepting attacker-signed ATCs until it learns the old key is revoked. I would sign the key registry with a separately protected offline root, include registry epoch/issued-at/expiry, define a short maximum ATC lifetime, and test stale-cache behavior explicitly."

The key registry now follows exactly this shape:

• Signed with an offline root key (separate Ed25519 keypair, private key on YubiHSM)
• registry_epoch (monotonic counter)
• issued_at / expires_at (short max lifetime — 7 days default)
• previous_ca_key_id for grace-period verification of old cards
• Verifier fetches registry with Cache-Control: max-age=300 (5 min) and Stale-While-Revalidate: 86400 (1 day) — so a cached registry can serve for up to 24h after the registry expires, but rejects new cards signed by the old key after rotation

Stale-cache behavior is in the test suite: https://github.com/alicelabs-llc/universal-trust-adapter/blob/main/tests/test.mjs

6. (Unspoken but implied across all your comments)

The thread that runs through all 6 of your comments is: "what does each layer actually catch, measured, not claimed." That's the right pressure to apply. The 8-layer article sounded comprehensive; the honest per-layer breakdown is:

• L1 metadata validation: caught 14 malformed manifests in 14,581 skills (none malicious)
• L2 Docker sandbox with --network none: caught the trojan via seccomp denial on clone()
• L3 Semgrep: caught 23 secrets in README files
• L4 YARA family signatures: 0 catches (currently pulls no weight)
• L5 secret patterns: 6 AWS keys + 2 Stripe keys in test fixtures (false positives)
• L6 dependency scan: 0 CVEs in production deps
• L7 dynamic analysis in gVisor: 0 catches beyond what L2 already caught
• L8 interceptor: 0 blocks in production, 12 warnings

L4 and L7 either get rebuilt with measurable detection criteria or get removed. "8 layers" was the wrong framing — it should have been "8 layers, 2 of which actually caught things, here's what the other 6 are for."

— Edison