TL;DR: Making a fix is not proof it shipped. Crawl before you change anything, save that crawl, re-crawl after with identical settings, and compare the two. The comparison reports each issue as resolved, reduced, or unchanged, and surfaces anything your fix broke elsewhere. One real before-and-after below.
You shipped the fix and closed the ticket. That is not the same as proving the change is live, correct, and free of collateral damage. The proof is a second crawl, compared against the first.
A crawl comparison loads your re-crawl against a saved baseline and reports every issue as resolved, reduced, or unchanged, URL by URL. Here is the procedure, then a real worked result, then the part most people skip: using the comparison to catch what your fix broke somewhere else.
The procedure
Verify a technical SEO fix with a crawl comparison (Screaming Frog)
1. Set storage mode to Database. Required for saved, comparable crawls.
2. Crawl the site. Save it as your dated baseline.
3. Make the fixes. Deploy. Let the page cache or CDN clear.
4. Re-crawl with the IDENTICAL config and the IDENTICAL segments.
5. Switch to Compare mode and select the baseline and the re-crawl.
6. Read the Issues comparison. Filters: Added, New, Removed, Missing.
Crawl comparison needs database storage mode and a paid license. The free build is capped at 500 URLs with the config restricted, so this is a licensed-tool workflow from step one.
What it proved
This is a real cycle on a production WordPress property: a June 11 baseline against a June 22 re-crawl, identical settings, every delta taken from Screaming Frog's Compare mode.
| Finding | Before | After | Result |
|---|---|---|---|
| Invalid HTML elements in head | 39 | 0 | Resolved |
| Rich result validation errors | 34 | 0 | Resolved |
| Rich result validation warnings | 18 | 2 | Reduced 89% |
| Internal redirect links | 9 | 0 | Resolved |
| External redirect links | 20 | 0 | Resolved |
| Internal client error (4xx) | 1 | 0 | Resolved |
| Duplicate H2 across pages | 4 | 0 | Resolved |
| Non-indexable internal URLs | 14 | 4 | Cleaner |
| External links resolving directly to 200 | 78 | 100 | +22 |
Two of those rows need reading carefully rather than at face value. The four remaining non-indexable URLs are the XML sitemap files, which are non-indexable by design, not a problem to chase. And the 78 to 100 line is a side effect of the redirect cleanup, because internal links that pointed at redirecting legacy slugs were repointed to their final 200 destination, not a separate win.
The high-priority cluster, head validation, schema errors, and broken or redirecting links, closed in full. Those are the items that decide whether the rest of a page's signals are read correctly, which is why they go first.
The rule that makes the comparison mean anything
The baseline and the re-crawl have to use the same spider config, the same segments, and the same prompts if you run any. Change the measurement between runs and a later difference could come from the instrument instead of the work.
Treat the config as a ruler. You are measuring the same site twice and comparing the readings, so the difference is only attributable to your fixes if the ruler did not change. The failure mode to watch for is a fake win: you re-crawl with a tighter include rule, the non-indexable count drops, and it looks like your directive cleanup worked when you simply stopped crawling the URLs that carried the problem. Lock the config before the baseline and do not touch it until after the re-crawl.
Reading the comparison
The overview gives you current, previous, change, and change percentage on every total. The issues view is where verification happens: each finding from the baseline reads as resolved, reduced, or unchanged, with the high-priority items flagged so you can see at a glance whether the changes that matter actually closed. A number that did not move is telling you a fix did not land, which is exactly what you want to know before you call the work done.
The change detection runs on four filters: Added, New, Removed, Missing. That is what lets you see which specific URLs moved and in which direction, not just that a count went down.
The benefit most people skip: regression detection
A single audit snapshot tells you what is broken now. It cannot tell you what your last fix broke. That is the argument for re-crawling even when you are sure the change worked.
A comparison surfaces new and added issues alongside the resolved ones, and the new ones are where regressions hide:
- A redirect fix orphans a page that was only reachable through the old URL.
- A schema template change passes on the template you edited and breaks validation on a different template that shared the component.
- A noindex directive leaks onto pages you meant to keep indexable.
- An internal link repoint creates a fresh redirect chain you did not have before.
None of these show up if you only look at the issue you set out to fix.
Prove it without exposing private data
The worked cycle published zero Search Console figures and zero analytics figures and still proved every structural result. Response codes, the canonical profile, the directive profile, and schema validation all come from the crawl's own data. For client work that means you can show a fix shipped and nothing regressed without disclosing indexation counts or traffic numbers. The crawl-level evidence stands on its own.
When a fix does not show up in the re-crawl
Usually one of these:
- A page cache or CDN is still serving the old version. Re-crawl after it clears, or verify against a direct origin pull.
- The change deployed to staging and never reached production.
- A segment rule built on a URL pattern silently skipped a slug that did not match the naming convention. Validate your segment counts before trusting per-segment diffs.
- The re-crawl covered a different URL set than the baseline. Back to holding the config constant.
- A rendering difference between the two runs, so the output you are comparing is not the same output.
Work that list before concluding a fix failed. More often the fix is fine and the measurement drifted.
Make it recurring
Once you have a saved baseline, a fixed config, and fixed segments, the whole thing becomes a loop: baseline, fix, re-crawl, compare. Because the config and segments are held constant, every new crawl compares cleanly against the last, drift gets caught early, and each fix stays measured rather than assumed to hold. Screaming Frog can even auto-compare the last two crawls on a scheduled run, which turns the loop into close to passive monitoring.
A crawl finds problems. A re-crawl compared against a saved baseline proves they were solved. The gap between those two is where most audits stop.
Originally published at https://www.techseoexperts.com/crawl-discovery-diagnostics/crawl-comparison-verify-seo-fixes/
Top comments (0)