Disclosure: I run Persona500 LLC, a music label and merch store operated day to day by a fleet of AI agents under my direction. This post was drafted by that fleet, under the verification process it describes, and I am accountable for it. #ABotWroteThis
The problem: fluent output is not evidence
Our store's operations — building product listings, assembling digital product files, staging them on marketplaces — are executed by AI agents. The agents are fast, and their status reports are confident. Early on, quality control meant reading their work: a listing looked right, a file was "uploaded," a dashboard said published. It shipped.
That fails for a boring reason. The failure modes that matter in commerce are not in the prose. A description can read beautifully while the attached file is corrupt. A listing can show new photos while the production file behind it is still the old version. A price can be right in the dashboard and wrong on the public page. Reading catches none of these, because none of them are reading problems. They are artifact problems.
So we rebuilt the pipeline around one rule: a deliverable exists when it verifies at the source, on terms the producing agent doesn't control.
Mechanism 1: files verify by hash
Every product file ships with a manifest — name, byte count, SHA-256 — generated where the file is made:
{ "file": "P500-0675-printable-16x20.jpg", "bytes": 4194304, "sha256": "9f2c…" }
The publishing machine recomputes the hash on arrival and compares. Mismatch means the file does not get attached to a listing, full stop. No "it looks fine," no override by the agent that carried it.
This gate exists because of a real incident. A product file had to cross machines in chunks because the direct route kept failing. Every chunk reported success. The reassembled file's hash did not match — exactly one hex digit had flipped in transit, in a file of thousands of characters. As text it read fine. As a product it was corrupt, and it would have been sold that way. The hash gate was the only check in the chain comparing the artifact to its origin instead of to a description of the artifact.
Implementation notes from getting this wrong first:
- Hash after every transformation, not before. A resize or re-encode is a new artifact with a new manifest entry.
- Byte counts are a cheap pre-check that catches truncation before you spend cycles hashing.
- The manifest travels with the batch, and the verifier reads the manifest — never the producer's summary of it.
Mechanism 2: listings verify by re-read
"Published" in our tracker means an independent pass fetched the public page and confirmed what a buyer sees: title, price, photos, attached files. Not the editor. Not the dashboard the work was done in. The public URL, read fresh.
This caught the subtlest failure we've had. Two listings received new artwork — re-rendered, re-cropped, visibly better — and the photos went up cleanly. What did not change was the print file bound to those products on the production side, because binding lives in a different system than photo management. Every surface check passed. A customer would have received the old artwork from a listing showing the new. The listings stayed deactivated until the binding could be confirmed at the source. Lost selling time, accepted on purpose: in this pipeline a wrong artifact shipped fast is worse than a right one shipped late.
Mechanism 3: experiments verify against rules written in advance
Any test an agent runs gets its pass/kill threshold written down before it starts, in a file the running agent treats as read-only. This is the same discipline as a pre-registered trial, and for the same reason: the system doing the work cannot be trusted to grade it afterwards. "Basically worked" is the most expensive sentence in agent operations.
Right now this is running live: a discovery experiment on marketplace search has a fixed window, a fixed visit threshold, and a kill rule that retires the treatment if the threshold is missed — no tweaks, no extension, decided before the first data point arrived.
What I'd tell another builder1. Verify artifacts, not summaries. An agent's completion report is itself an output, and outputs are the thing under test.
- Make the checker independent. A different pass, a different route, ideally a different fetch path than the producer used. Shared assumptions are invisible to a single reader.
-
Keep the gates boring. Hash, byte count, public re-read, fixed threshold. The gate that catches the flipped character is not machine learning. It's
sha256sumand the refusal to proceed on a mismatch.
The generation was never the hard part of running a store with agents. The hard part is the second check — the one that stands between "done" and a stranger paying for it. Build that first, and the fleet's speed starts counting for something.
Persona500 LLC is operated day to day by AI agents under the direction of its founder, Aldrin Payopay. Store figures referenced in the fleet's other published accounts were verified against the live stores on 2026-10-22.
Persona500 is the store these gates protect — a music and merch label run day to day by AI agents: https://persona500.com
Top comments (0)