I run two sales agents in production now. InboxCopilot reads inbound email and qualifies leads. EnrichmentMessenger takes a CSV of prospects and writes first-touch messaging for the ones worth touching. Both of them write to the same attribution schema, a Postgres setup where every agent action logs as a touchpoint and gets a capped share of revenue credit when a deal closes.
Before either agent touched a real inbox or a real CSV, I ran them cold. Skill file verbatim as the system prompt, JSON Schema output contract, AJV validation on every response, zero retries, zero coaching. If the model can't follow its own instructions on the first try, the instructions are wrong, not the model.
InboxCopilot went 3 for 3. EnrichmentMessenger went 2 for 3.
Row 2 of the EnrichmentMessenger test was a prospect with no usable information. Not enough to enrich, not enough to message. The agent's job in that case is simple: skip it, log nothing, move on. Instead it returned a disposition of created, an empty enrichment payload, and a null message. That disposition is supposed to mean "this became a real sourced lead." Nothing about that row earned it. If that output had reached the database unexamined, it would have written a lead_created touchpoint for a lead that didn't exist in any meaningful sense, a phantom lead with a live attribution record.
Sourced pipeline is this product's cleanest number. It's the one metric that doesn't need a cohort comparison or a minimum sample size to mean something. A single phantom row poisons it quietly, because nothing about a bad created disposition looks wrong from the outside. The row exists. The lead exists. The number is just wrong.
I went in expecting a model failure. It wasn't. The skill file's disposition logic had a step-order conflict: a match-first read let the model assign created before the no-basis check ever ran. Given a row with almost nothing to work with, the instructions let the agent find something plausible to say instead of forcing it to say nothing. That's not a hallucination. That's a skill file that let ambiguity resolve toward action instead of toward silence.
The fix was to rewrite the whole disposition decision as one ordered tree with the no-basis check first, not as a set of conditions the model picks through in whatever order looks convenient. Ambiguity now resolves toward skipped_no_basis by default, with zero touchpoints attached, and only escalates toward a real disposition when the row actually earns it.
Re-run cold: EnrichmentMessenger 3 for 3, the same row now correctly skipped. InboxCopilot unchanged at 3 for 3. Six for six combined, no regression.
The schema carries a hard 50% cap on AI attribution credit regardless of how many touchpoints a deal accumulates, so a defect like this couldn't have inflated any single deal's number past that ceiling. But it could have created deals, and pipeline that was never real is worse than a capped number that's honest. The production assertion suite for the schema now runs 21 checks, all passing, and the ordered-tree pattern from this fix is the one I use for every disposition decision I write now, in this project and outside it.
If you want the receipts: schema and views at commit a231d16, the repo reaching shippable v1 shape at af7ec15, and the docs made fully accurate to the shipped code at 5d58f51.
No traction numbers here, because there aren't any yet. The kit launched with no case studies, and the plan was always to let the attribution data speak for itself once it exists. What I can show you today is that the test caught its own worst mistake before a real prospect ever touched it. If the "what can this actually claim" question interests you more than the pitch does, the methodology behind it is at stack.chadtdyar.com/deep-dive.
Top comments (0)