TL;DR: AI audit output is a lead, not a verdict. Give every recommendation one of five dispositions, verify the survivors against crawl evidence and primary docs, and prove the fix held with a re-crawl. The five-way triage and a seven-point checklist are below.
Before you action a single AI recommendation, sort it into one of these:
ACCEPT specific, aligned with the page, corroborated by the crawl -> task as written
REVISE right direction, wrong specifics -> fix the detail, then action
MERGE several prompts, one root cause -> collapse into one task
REJECT generic, unsupported, or contradicted by the crawl -> it dies here
BACKLOG real but low priority -> recorded with its evidence
That taxonomy is the whole method. Everything else is how you decide which bucket a lead lands in, and how you prove you were right after the fix ships.
The trap with AI audit output is that it reads the same whether it is correct or not. It names the URL, the issue, and the fix in clean prose, with the same confidence either way. The obviously wrong recommendation is cheap because you ignore it. The plausible, confident, wrong one is expensive because you build a ticket around it: developer time on markup that already validates, internal links pruned because a score flagged them, a redirect removed that was doing its job.
Why a model can only ever hand you a lead
A lead is a candidate finding. A verdict is a decision to act. A model produces the first and cannot produce the second, and the gap is structural. A model scoring your pages has no access to your live index, no ground truth about your stack, and only the part of the page it can parse. Strong pattern match, narrow view. Good at widening the search for problems, bad at closing the decision on any one. So the rule is absolute: nothing enters the plan on an AI suggestion alone.
How the AI gets into the crawl
Screaming Frog connects directly to OpenAI, Gemini, Anthropic, and Ollama, and runs custom prompts at crawl time against each page, returning a score and rationale on the URL row. You can configure up to 100 prompts.
In the cycle these examples come from, nine prompts ran, each bound to the segment it was written for so it only saw the pages it could judge. Four segments, sixteen pages in the article segment. Binding was also a cost control. Unbound against every URL, the prompts would have made about 312 model calls. Bound, the same analysis took 111, a 64 percent cut, with nothing scored out of scope.
Three ways the model is confidently wrong
It scores what it can see, not what is true. The operational-depth prompt in this cycle credited configuration values written in prose and scored them as zero when the same values lived only in a code block or a screenshot. An article with real rule expressions in a fenced block returned zero operational items. An article that stated the same thresholds in sentences scored well. Same values, different visibility. A low score can mean weak content, or content the model could not read.
It restates vendor behavior that has drifted or is plan-gated. Four claims in this cycle only survived a check against current vendor docs, each verified against the primary source:
- Cloudflare's Log action for custom rules is Enterprise-only, so "start in log mode" is wrong for most readers on lower plans.
- Cloud Armor backend buckets accept only edge security policies, not backend security policies, so buckets and backend services are not interchangeable targets.
- HSTS preload needs a max-age of at least 31536000 seconds with includeSubDomains present.
- Google Cloud auto-disables exposed service-account keys by default, on by default since June 16, 2024.
None of those are corrections a model makes for you, because it is repeating a confident generalization. Verify vendor claims against the vendor's live docs, by name.
It comments on things it cannot verify. A model reading a page has no view of Google's index, so any index-status claim is inference dressed as fact. Confirm it in URL Inspection. The same blindness produces false positives on legitimate business logic: deliberate redirects called chains, product or filter variants called duplicates. The model sees a pattern that looks like a problem and cannot tell it is intentional.
Verification, in order
A lead becomes a finding only when it clears three checks:
- Does the crawl row actually say what the model claims? Most rejects die here.
- If it rests on a vendor feature, plan tier, default, or policy, does the vendor's current documentation confirm it?
- Does the schema pass Google's Rich Results Test, not only the crawler's validator? The two must agree.
Prove it with a re-crawl
A crawl finds problems. A re-crawl proves they were solved. Diff a baseline crawl against a re-crawl in Screaming Frog's Compare mode with identical settings, segments, and prompts. Hold the instrument constant or a later difference could come from the measurement instead of the work. In this cycle, baseline June 11 against re-crawl June 22 showed invalid head HTML 39 to 0, rich result errors 34 to 0, internal and external redirects 9 and 20 to 0, and twelve of sixteen articles gaining 13,703 words of narrated detail.
Re-running the same prompts on the edited pages is how you check the edits, not a chance to improve the prompts. The prompt is the ruler. Hold it steady.
The checklist
Run every AI lead through this before it becomes a task:
1. Specific, or generic best practice it would say about any page?
2. Aligned with what this page is for?
3. Corroborated by the crawl row, rendered content, or response code?
4. If it rests on vendor behavior or a plan tier, do the current docs confirm it?
5. Can the model see what it scored, or is it locked in code, an image, or behind rendering?
6. If it claims indexation, was that checked in Search Console, not inferred?
7. After the fix shipped, was it re-measured against the same baseline?
Clears all seven, it is a finding. Fails one, it is a lead that is not done yet. Anyone can run prompts against a crawl. The skill is the triage, and the re-crawl that proves the survivor held.
Top comments (0)