Everything below uses a site I generated. These are test results, not results from anyone's real website.
The idea
A crawl of your own site already holds most of what an SEO metadata audit needs: titles, descriptions, canonicals, language, Markdown (so headings), word counts. Crawling again to check them is a second bill and a second chance to get blocked. So I wrote an Apify Actor that audits the dataset you already have, such as Website Content Crawler output, and never fetches anything.
The hard part of building an auditor is not writing the rules. It is knowing the rules fire when they should, and stay quiet when they should not. I had no real site I could publish results about, so I built a fake one.
The benchmark
A seeded generator produces a synthetic 400-page site in Website Content Crawler's shape: url, metadata.title, a description, a canonical, Markdown text and a status. Every generated page carries a hidden list of the exact issue codes the audit must report, planted on purpose: titles built right at the length limits, a canonical that points at another canonical, groups of pages that share a title, thin pages, broken status codes.
The test passes only if the audit reports exactly the planted issues on every page: nothing missing and nothing extra. It does: 400 of 400 pages match, across 54 distinct rule codes.
On that site the audit reports 370 issues (50 errors, 238 warnings, 82 info), a site score of 94 (grade A), 333 pages graded A and 67 graded B. The top issues:
| Issue | Pages |
|---|---|
| Very thin content | 11 |
| Canonical chain or loop | 7 |
| Canonical invalid | 6 |
| HTTP error status | 6 |
| No H1 | 5 |
Site-level analysis finds duplicates across pages: 6 groups of duplicate titles, 3 of duplicate descriptions, 3 of duplicate H1s, 2 of duplicate page text and 3 of URL variants. Every page's score lists the points it lost, for example -6 T005 duplicate title (group dt_1, 2 pages) or -15 D001 meta description missing.
What this does and does not prove
It proves the rules do what I wrote them to do, including at the boundaries, and that the scoring adds up. It does not prove the auditor is accurate on real sites. I wrote the generator, the rules and the tests, so this is a self-consistency check, not an independent one. Real crawls will have shapes I did not think of. If a field my adapter expects is missing from your dataset, for example the HTTP status or the Open Graph tags, the checks that need it are skipped and listed in the run summary instead of flagging every page, but I have not yet confirmed those field names against every crawler's current output.
Speed and cost, also on synthetic data
On a laptop, the audit step took about half a second for 10,000 generated pages. In the cloud, a run on 2,000 synthetic pages took 4.0 seconds end to end at 1 GB: loading the dataset about 1.6 s, the audit 0.14 s, writing the results about 0.6 s. Reading and writing the dataset is the slow part, not the auditing.
Two design choices worth copying
Skip, don't guess. If the dataset has no heading data, the H1 rules do not run, and the run summary says why. Flagging "no H1" on every page of a crawl that simply did not save headings would be a false alarm.
Explain every point. Each issue has a code, a message, evidence and a fix, and the score is 100 minus the listed penalties, with caps per category. You can reproduce any score by hand.
Try it
https://apify.com/hiredhandai/seo-metadata-auditor ($1.50 per 1,000 pages, pay per use). The prefilled example audits six inline pages in a few seconds. If you run it on a real crawl and it misses something or flags something wrong, I would like to hear about it.
I'm the author, and I used AI assistance to draft and test this.
Top comments (1)
Nice write-up, and good that you're upfront about it being a self-consistency test. Since you asked about real crawls, here are shapes I see a lot that synthetic sites rarely have:
Listing skipped checks at the top of the summary, not only in the run log, would also help people notice them.