Three months into this experiment I had a problem I couldn't measure: every article was passing the quality gate and none of them felt distinct. Google's E-E-A-T guidelines frame this as experience and first-hand knowledge — signals I couldn't produce with structural auditing alone. The cliché checker was clean, the tag pool was valid, the word counts were in range. But there was no difference between an article I'd written from direct code inspection and one the generation routine had assembled from nothing in particular.
That's the gap quality_contract v2 addresses. Where v1 checks what you didn't do, v2 checks what you did do. Four fields. Fail-closed for new articles. No retroactive rewrites of history.
Why v1 was catching the wrong things
The original quality gate I described in June tackled three specific failure modes: clichéd phrases, tags outside the allowed pool, and word counts outside the target range. That gate blocked maybe a dozen articles in its first month — mostly ones where the generation routine got lazy on voice.
What it didn't catch was whether an article had any reason to exist. Cliché-free, tag-valid, word-count-correct prose can still be generic prose. E-E-A-T — experience, expertise, authoritativeness, trustworthiness — requires affirmative claims. v1 had no model of what the article positively asserted.
I noticed the failure mode in the aggregate. Reading back ten articles from June, I couldn't easily say which ones were grounded in direct code observation vs. reconstructed from memory. The articles didn't lie — they just didn't signal anything about their own provenance.
The four v2 fields and what each does
Five frontmatter keys in total: one version marker, four with actual signal.
quality_contract: "v2" lets the audit script know which rule set to apply. This is useful when you need to introduce v3 later and want to distinguish between article generations. I've wasted time debugging audit scripts that applied the wrong checks to historical files; a version marker costs one line.
primary_keyword is the one search phrase this article targets. Writing it before the article forces a scoping decision at the cheapest moment — before section one exists. Articles with a specific primary_keyword tend to include fewer filler sections, because the keyword keeps pulling the focus back. An article with primary_keyword: "AI article quality gate frontmatter fields" is harder to let drift into generalities than one with no keyword named.
search_intent is one sentence: who is searching and why. I write it as "[persona], looking for [specific thing]." This constraint is genuinely useful for the generation step — an article written for "a developer whose AI pipeline already ships 40 articles a month and wants to raise the bar" covers different ground than the same title written for someone building their first pipeline.
verified_at is a date: the day I checked that the claims in this article are current. For code-based articles, it's the date I read the relevant files. For tool comparisons, it's the date I ran the tool. The field doesn't enforce verification — a date string isn't a verification record. But it surfaces a commitment: on that date, I claimed these facts were accurate. That friction is enough to make typing a wrong date feel dishonest, and that turns out to be the friction that matters.
original_evidence is the hardest field to fill in. It names one piece of first-person evidence that only the author of this particular project can offer: a measurement, an error message I hit, a function I wrote. If I can't name something specific, the article probably doesn't have a competitive reason to exist. There are thousands of articles explaining how GitHub Actions cron works. An article about the specific cron timing edge case that broke my daily Bluesky pipeline on a specific date is harder to replicate.
Here's the complete block from this article:
quality_contract: "v2"
primary_keyword: "AI article quality gate frontmatter fields"
search_intent: "Developers running AI-assisted article pipelines who want to enforce
E-E-A-T credibility standards at the content-generation step rather than at publish time."
verified_at: "2026-08-03"
original_evidence: "The audit-articles.mjs script in this repo, which implements the v2
contract checks with date-based cutoff logic and grandfathering for already-published articles."
How audit-articles.mjs enforces it
The check is date-gated: only articles whose filename date is on or after 2026-07-14 get the v2 rules. Everything older runs against v1 only.
const QUALITY_V2_KEYS = [
"quality_contract",
"primary_keyword",
"search_intent",
"verified_at",
"original_evidence",
];
const dated = basename(path).match(/^\d+-?(\d{4}-\d{2}-\d{2})-/)?.[1];
if (dated && dated >= "2026-07-14") {
// v2 checks run here
}
The date comparison is a string comparison. ISO dates sort lexicographically, so this works without a date parser. Small details like this matter at 11pm when you're deciding whether to finish the script tonight.
Missing v2 keys produce errors — errors.push(...), exit code 1, which blocks the dry-run step in the publish pipeline. That fail-closed behavior is the whole point. An article missing original_evidence doesn't leave the repository without it.
There's one nuance: the script distinguishes between articles with pending publish destinations and articles fully published everywhere.
const publishedTo = Array.isArray(meta.publish_to) ? meta.publish_to : [];
const publishedUrls = meta.published_urls ?? {};
// Bluesky excluded: older articles don't reliably record Bluesky URLs
const allPublished = publishedTo
.filter((t) => t !== "bluesky")
.every((t) => publishedUrls[t]);
if (allPublished) {
warnings.push(...missingKeys.map((k) => `(legacy) quality_contract v2: missing "${k}"`));
} else {
errors.push(...missingKeys.map((k) => `quality_contract v2: missing "${k}"`));
}
For a fully published article, missing v2 keys are warnings. There's no publish step to block. The Bluesky exclusion exists because early Bluesky posts didn't always record a URL in frontmatter — a behavioral gap documented in the Dev.to and Hashnode API comparison. Platform quirks accumulate into audit exceptions over three months of real publishing.
Grandfathering 130+ historical articles without backfilling
When I introduced v2 in mid-July, 115 articles were already published without the four new fields. My first attempt was backfilling: read each article, add the fields, commit. I got through about ten before stopping.
The problem: writing credible original_evidence for an article I published two months ago requires reconstructing what I was actually looking at when I wrote it. For most articles I can do that with reasonable accuracy. But "reasonable accuracy" is exactly the problem — original_evidence is supposed to prevent reconstructed-from-memory claims, not be composed of them.
The grandfathering rule — treat missing v2 keys on fully-published articles as warnings rather than errors — lets the legacy backlog exist honestly. Warnings show up in CI logs. When I write follow-up articles on the same topics, those get proper v2 fields. The older articles stand as they are.
This is an uncomfortable trade-off. The earliest articles, from when the project was building its initial presence, are the ones least likely to carry credibility markers. But the alternative — filling in original_evidence from memory — produces exactly the kind of low-confidence claims the field was designed to prevent. Honest gaps are better than filled-in noise.
What worked, what didn't, what I'd do differently
What worked: fail-closed at generation time. The five fields live in the article frontmatter template, filled in before the body starts. When I'm writing section three, verified_at and original_evidence are already there. I can't forget them; they're in the way until I fill them in correctly. This is the opposite of an end-of-pipeline check.
The three lint rules from last week address a parallel problem at the data layer — catching repeated boilerplate sentences across 1500 directory entries. The quality_contract v2 fields address it at the metadata layer. Between them, I can audit whether an article says something unique (lint) and whether its claims trace to first-person experience (contract). They're complementary and I run both before every publish. The broader bet — that quality gates protect AdSense approval better than content volume — is still unverified at month three, but the v2 contract is the mechanism I'm most confident about.
What didn't work: verified_at is still an honor system. The script checks that the field is a plausible ISO date string; it doesn't verify I actually looked at anything. I've noticed I occasionally type today's date reflexively while synthesizing from memory rather than reading live code. The field creates friction but not proof.
What didn't work (2): original_evidence for recap and curated articles is awkward. Those articles are observation-based by design, not code-based. "Observed from HN top-15 on 2026-08-01" is valid original_evidence, but the field label implies measurement or code. I'm considering splitting this in v3: empirical_evidence for things measured or built, editorial_provenance for aggregation-based articles.
What I'd do differently: add these fields to the generation routine's prompt template from day one. Rolling out a new frontmatter requirement three months into a project forced the grandfathering compromise. If primary_keyword and search_intent had been required from article 01, the quality signal would be richer and there'd be no two-tier system.
One related area I haven't automated yet: verified_at staleness. An article published in May with verified_at: "2026-05-15" is citing code state from 90 days ago. A sweep that flags articles where verified_at is more than 90 days behind the current date would help prioritize which articles need a refresh. That's probably the next thing I wire up.
FAQ
Q: Does primary_keyword actually change what gets written, or is it just metadata?
Both. As metadata, it's useful for audit logs and any downstream SEO pass. As a forcing function, writing the keyword before section one changes the article. An article with primary_keyword: "astro glob loader pnpm monorepo path resolution" stays on that specific topic. Without a named keyword, the same article tends to expand into generalities. The field narrows scope before I start, which is when narrowing is cheapest.
Q: Why run the v2 checks in a pre-publish dry-run rather than in the publish script itself?
Because the goal is to catch problems before I'm emotionally committed to shipping. An error at generation time is cheap — fix it while still writing. An error at publish time means I've finished something I'm proud of and now face either a rewrite or a deliberate quality skip. Friction placed at the wrong point doesn't change behavior; it gets bypassed.
Q: How do you prevent original_evidence from being filled with vague claims like "I built this"?
I can't, programmatically. The gate checks presence and type, not quality. A suspiciously short value (under 30 characters) gets a warning, not an error. What I rely on is visibility: the field is in raw markdown, appears in audit logs, and a vague claim reads as obviously thin when you see it next to a specific one. The social contract is: if the evidence claim is that thin, the article shouldn't ship. It's enforced by visibility, not validation.
Q: Can these fields transfer to programmatic directory pages?
verified_at transfers cleanly. A verified_at field on each SaaS entry in the OSS alternatives directory would make freshness audits straightforward: filter entries where verified_at is more than 90 days old, regenerate those, update the field. Same pattern, different content type. I haven't wired this yet — it's on the list.
Q: What's the difference between v1 and v2 in practice?
v1 checks absence of violations: no banned clichés, valid tags, word count in range. v2 checks presence of affirmative claims: named keyword, stated intent, verification date, first-person evidence. They catch different things. An article can pass v2 and fail v1 (right metadata, wrong voice) or pass v1 and fail v2 (clean prose, no provenance). I run both. See the original v1 post for the v1 implementation.
Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.
Top comments (0)