The bet, pre-registered here: articles published with quality_contract: v2 will show a higher median click-through rate in Google Search Console than my legacy articles by November 14, 2026. I'm betting behavioral constraints outperform better prompts for AI-assisted content accuracy. I won't have the data to prove it until then.
The problem prompt engineering doesn't solve
When I write with AI assistance, the generation step is confident. Claude doesn't hedge unless I ask it to. If a draft says "HuggingFace embedding models account for over 60% of downloads in the top 50" and the number sounds plausible, it appears with the same tone as a verified fact. Prompts help — "verify claims before stating them" or "flag uncertain statistics" both shift behavior — but they don't create a verification event. They create a tendency.
The gap I kept running into: I'd generate a draft, edit for voice and structure, fix the obvious errors, and publish. Later I'd find a claim that was subtly wrong — not fabricated, but dated, or based on a pattern I'd observed months ago that I couldn't verify at publication. The generation step had no forcing function to check the primary source.
I explored this with the three-tier content quality ladder early in the project: programmatic ETL entries, template-generated content, and hand-edited editorial content have different reliability profiles. The hand-edited tier is more reliable not because the author is smarter, but because editing creates a verification event that ETL doesn't.
What verified_at actually enforces
The quality_contract: v2 frontmatter block requires four fields:
quality_contract: "v2"
primary_keyword: <the one search phrase this article targets>
search_intent: <who is searching and why, in one sentence>
verified_at: "YYYY-MM-DD"
original_evidence: <first-person evidence only this author can name>
The verified_at date is a commitment: I'm claiming that on this specific date I verified the factual claims in this article. Not "at some point during writing" — on this date. The audit-articles.mjs gate rejects any article missing this field. The publish pipeline is fail-closed on it.
What this does behaviorally: it creates a verification event with a date. When I write verified_at: "2026-08-19", I'm implicitly committing to having checked the primary sources today. Filling it in without actually checking means falsifying a date — a different act from skipping verification, which is easy to rationalize as "I'll fix it later."
The original_evidence field is harder. I have to name something only I can observe: repo code, API response logs, database query output. If I can't name it, the article doesn't have first-person standing and I shouldn't publish it. This was the design intent behind E-E-A-T transparency pages — making first-person experience legible, not just claimed. Google's creating helpful, reliable, people-first content guidance describes "experience" explicitly as first-hand interaction with the topic, which is exactly what original_evidence forces you to name.
Three behaviors that changed
I now check primary sources on publish day, not draft day. Previously I'd research during writing and then publish without re-checking. The verified_at date defaults to the publish date, which means I look at the live source on the day I ship. For a claim about HuggingFace download counts, I check the API. For a claim about pipeline behavior, I run the script and read the output.
I now write original_evidence before the outline. If I can't name specific first-person evidence before starting, I don't have a unique-perspective article — I have a synthesis piece. Synthesis is fine, but it doesn't carry E-E-A-T signal the way direct observation does. Starting with the evidence forces me to pick topics where I have real standing.
I now articulate search_intent before writing, which changes the article structure. Before this field, I'd pick a topic and write at it. Now I write one sentence about who is searching and why before the first word of the body. This is the same collection-before-interpretation separation I use in the market-listening pipeline: the two steps contaminate each other when run together.
The behavioral effect isn't dramatic per article. Over 40 articles under this constraint since July 14, my correction-commit rate — commits that fix a factual error post-publish — is lower than for earlier articles. I'm not claiming causation; there are too many confounds. But it's the metric I'm tracking.
How the workflow changed
| Step | Before quality_contract v2 | After quality_contract v2 |
|---|---|---|
| Topic selection | Recent commits or trending topics | Must name original_evidence before committing |
| Research timing | During draft, approximate | On verified_at date, specific sources checked |
| Drafting | Generate → edit → publish | Generate → fill four fields → verify → edit → publish |
| Pipeline gate | Audit checks formatting only | Audit rejects missing quality_contract fields |
| Review | Ad hoc | Codex review via codex-handoff.mjs on every staged article |
The Codex review step catches a different class of errors than verified_at does: internal inconsistencies, link rot, voice rule violations. The verified_at constraint catches factual staleness at the source — before the review even runs.
The falsifiable bet
At November 14, 2026: articles published under quality_contract: v2 (post July 14, 2026) will show higher median organic CTR in GSC than my legacy articles (pre-July 14), when comparing articles targeting similar search intent categories.
The confound I can't fully control is topic drift — I'm covering different subjects now than I was in May. I'll segment by intent category (how-to, comparison, meta) and compare within segments where both sets have articles.
Programmatic freshness alone doesn't produce quality — the automation-first approach is necessary for pace but insufficient for depth. The quality_contract fields are an attempt to add a behavioral quality layer without breaking the publication pace. I'm currently at 140+ articles without skipping a weekday.
The honest counterargument
Google's crawler doesn't read YAML frontmatter. verified_at is invisible to Googlebot. The fields don't appear in the rendered article body. The behavioral improvements I'm claiming are real, but they're not legible to search engines through the metadata itself.
The mechanism I'm betting on is indirect: better-verified articles have fewer factual errors → readers who click engage more deeply → lower bounce rate and longer dwell time → behavioral signals that influence ranking. Both steps need to hold, and step two requires enough traffic for behavioral signals to register.
I made a separate bet last May about AI directories surviving Google AI Overviews. That one is measured on the same timeline and the two bets are correlated. If AI Overviews intercept most comparison queries before the click, behavioral signals on clicked articles become irrelevant regardless of article quality.
Detecting template residue in AI-generated content is the ETL-layer version of this problem — content generated from a template without real input degrades reader engagement predictably. The quality_contract constraint is a forcing function at the article layer in the same way the template-residue detector is a gate at the directory-page layer.
What would change my mind
Three signals would make me revise the bet:
- GSC data at month 6 shows the top-performing articles (by CTR) disproportionately come from the pre-July legacy set
- My correction-commit rate for v2 articles catches up to the legacy rate as articles age
- Repeat-and-vote testing on the same prompts produces identical quality distributions with and without a manual verification pass — which would mean verification adds no quality signal beyond what generation already provides
The third condition is the cleanest test of my core claim. If it fires, I'll admit prompt engineering was enough and simplify the pipeline back.
Frequently asked questions
Can't you get the same result by adding "verify all claims" to the system prompt?
I tried this for about two months. A prompt produces a tendency, not an event. The model adds hedging language and skips unsupported claims — but it can't check live sources. The verified_at field requires a human verification event against primary data. Prompts asking for verification are better than no prompts, but they don't create the same behavioral friction as a date field you have to sign your name to.
Is verified_at visible to Google?
No. It's YAML frontmatter stripped before generating the HTML. The bet is that the behavioral effect on writing quality shows up through article content and reader engagement, not through the field itself.
What if you fill in verified_at without actually verifying?
You're falsifying a date, which is a distinct act from simply skipping verification. The psychological friction is part of the mechanism. Filling it in without checking requires a deliberate choice to lie to yourself — harder to rationalize than just not having the step.
How do you handle articles about forward-looking bets, like this one?
The original_evidence field covers present-day evidence, not future data. For this article, it's the 40+ v2 articles in the git history and the audit-articles.mjs gate that runs on every commit. The bet is what I expect to happen. The evidence is what exists now and what I'm basing the reasoning on.
Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.
Top comments (0)