I run a travel site that tells visitors what the cannabis laws are wherever they're going. This week I re-checked every US state against its official sources. The pages needed it. One said Nevada allows an ounce; the limit has been two and a half ounces since January 2024. One said Colorado allows an ounce; it's two, though you can only buy one at a time. Massachusetts doubled its limit in April 2026 and the page hadn't noticed.
Fifty-one jurisdictions, each with a possession limit, a penalty, a rule for visitors and a source, is too much to check by hand in a week. So four AI checkers did it, and the interesting part is not that they found the old errors. It's that each of them made new ones, and none of those reached the site.
The four checkers
A coding agent (Claude Code, on a home server) drafts every change: it reads the old page, searches for what changed, writes the new text and sends the questions it can't settle.
A browser agent gets those questions. It's a separate AI agent with its own real browser, and I talk to it through a Slack channel. Each request is numbered, lists the claims to confirm, and asks for the exact wording from the official page plus the URL. "NOT FOUND" is an accepted answer; guessing isn't.
A critic watches that channel. When the browser agent posts an answer, a second instance of Claude Code reads it in read-only mode and replies in the same thread within seconds: which claims are sourced, which aren't, which contradict what's already on the site. It can't change anything. Nothing is applied until I say "apply it".
A classifier grades the final output. Jev is a decision model: you give it text and a question with fixed answer options, and it picks one with a confidence score. It can't look anything up, which is the point. It only judges whether a conclusion follows from the text in front of it.
The browser sees what the server can't
The first surprise was physical. The coding agent runs on a server outside the US. Of the 39 source URLs it collected for the first half of the states, 17 wouldn't load from there: nine returned 403 (bot protection) and eight never answered at all. Those included Florida's statutes, Idaho's, Kentucky's, Nebraska's, Utah's and Wisconsin's.
The browser agent opens all of them, because it's a real browser on a US connection. So the work splits by what each side can reach, not by preference: the server checks every page it can load, and the browser agent quotes the rest.
Everyone was wrong somewhere
The browser agent said Tennessee banned THCA products from July 1, 2026. The state regulator's FAQ says something else. THCA has counted toward the 0.3% THC limit since January 1. July 1 was the date by which stores must buy only from licensed wholesalers. A real date, attached to the wrong rule. The coding agent caught it by checking the regulator's FAQ before applying the answer.
The browser agent also said a 2025 law opened Iowa's medical program to non-residents. The critic flagged that there was no source. The rulemaking notice turned up: it's a 2026 law, and in August the rules weren't final. The page now says to check with the state first.
The critic doubted that Maryland fines public smoking $50 the first time and $150 after that. Its "understanding of the law" was a flat $50. The statute says $50, then $150. The browser agent was right, and the critic's doubt cost one lookup.
The critic also warned that Maine's concentrate cap was 5 grams, because the original law said so. The state cannabis office's FAQ says 10 grams. Again the browser agent was right.
The coding agent labelled Washington DC "with a medical card": you need a card from home to buy there. The browser agent read the DC regulator's page and corrected it. Visitors don't need a home card at all; anyone 21 or over can register online as a temporary patient. My label wasn't just imprecise. It undersold the one thing a visitor wants to know.
The classifier graded all 51 "can visitors buy?" labels against the text of each state's rules, and disagreed with one. I had North Carolina as a flat "no". Jev said yes, at 0.83 confidence, because the text says a tribal store on Cherokee land sells to any adult 21 or over. Under state law the answer is no. On the tribal land the answer is yes. The label now says both, because a classifier with no world knowledge noticed that my "no" contradicted my own paragraph.
The pivot
I expected the checkers to stack: each one a stricter filter, so that whatever survived all four was right. That isn't what happened. The critic was strictest, and it was the one that doubted two correct answers.
A second checker isn't a more careful copy of the first. It's blind in a different place.
The browser agent reads well and quotes accurately, but it can attach a real date to the wrong rule. The critic is good at spotting missing sources and bad at knowing the law, because it's arguing from memory. The coding agent has the whole context and makes judgement mistakes: it had read the right DC page and still filed DC by category. Medical stores, so medical card. The classifier knows nothing at all, so it can't be fooled by what it thinks it knows, and catches the conclusion that doesn't follow from its own evidence.
None of them is the safety net. The net is that their errors don't overlap.
What makes it work
A few rules turned out to matter more than which models were involved.
- Every claim needs a quote and a URL. "Confirmed" without wording isn't accepted. That's what made the Tennessee mix-up visible: the quoted text didn't say what the summary said.
- "NOT FOUND" is a valid answer. If an agent has to answer, it will produce a fact. When no official source existed for a local fine or a visitor fee, it stayed off the page.
- Disagreement goes to a source, never to a vote. When the critic and the browser agent disagreed, the statute decided, not two models outvoting one.
- Nobody applies their own work. The critic can't write. The browser agent can't write. A human says "apply it", after a summary of what changes.
Each state page now lists its sources and the date it was actually checked. That date used to come from the server's clock, which made every page look verified that morning. It's now set only when someone has checked the page, and it's the date this post is about.
Top comments (0)