Free Website SEO Audit: Finding What's Actually Broken

Your site can rank for nothing and you won't know why — unless you run this free audit. Here is the exact checklist I run, plus what the AI search engines want in 2026.

Six months ago, I opened Search Console for a site I'd inherited from a client handoff and found something ugly: 31 pages returning 404s, most of them indexed. And that was only the visible damage. The crawl data showed roughly 900 URLs Google had discovered that I had never built — phantom links, pagination fragments, and a parameter mess that made the site look three times larger than it was.

That audit cost nothing. No paid tool, no agency report. Just the free stack — Search Console, a crawl log, and page-speed data — plus the knowledge of what to look at first. The fix took a week, and the site's crawl efficiency jumped visibly within two cycles.

Here is how to get found and cited in 2026: fix what is actually broken, then make your content trivially quotable for the AI crawlers. This article walks both halves — the free technical audit and the platform-by-platform checklist for AI search visibility.

The Free Technical Audit in Six Steps

Run these in order. Each one needs zero dollars and catches a different class of failure.

Step 1: Check Indexability

Open Google Search Console → Pages report. Filter by the "Not indexed" tab and read the reasons. The four that matter:

• Crawled — currently not indexed: Google looked, decided low quality or duplicate, moved on. Usually a content or canonical problem.
• Discovered — currently not indexed: Google found the URL but never crawled it. Crawl budget is the likely culprit.
• Duplicate, Google chose a different canonical: you have a canonicalization mistake.
• Soft 404s: pages that return 200 but display "not found." Google treats these as thin pages.

If your money pages sit in any of these, fix the reason, not the symptom. A "crawled not indexed" page needs better content or a better internal link; a "discovered" page needs fewer junk URLs so the crawler reaches it.

Step 2: Find the 404s and Orphan Pages

Grab your server access log or use Google Search Console's "Page indexing" report for the URLs Google has already flagged. A free crawler like Screaming Frog's free tier (500 URLs) or a quick log analysis will surface the 404s that matter.

Then hunt orphans — pages with zero internal links. I found 900 phantom URLs because of bad pagination and filters. The fix was noindex,follow on parameter pages, clean canonical tags, and an XML sitemap that lists only the 200 pages worth ranking.

The discipline here is to fix the reason behind each status, not the page. A page Google "discovered but never crawled" needs fewer junk URLs competing for crawl budget, not a new meta description. A page "crawled but not indexed" needs better content and a stronger internal link. If you only chase symptoms, the same URLs come back in the next cycle and you have learned nothing.

Step 3: Fix Core Web Vitals (The 75th Percentile Wins)

Google's thresholds are public, and they are the same thresholds the AI crawlers use as a rough quality signal:

• LCP under 2.5s — the largest element (usually an image) must load fast. Lazy-load below the fold, fetchpriority="high" on the hero.
• INP under 200ms — how responsive your page is to taps and clicks. The biggest fixes are removing third-party script chains and keeping main-thread work low.
• CLS under 0.1 — layout shift. Reserve space for images and ads so the page does not jump.

Use PageSpeed Insights (free) and run your top ten pages. For most WordPress or Next.js sites, the top three fixes are: serve next-gen images, defer non-critical JavaScript, and self-host or remove tracking scripts.

Step 4: Make Every Page Crawlable

The basics, checked in five minutes: one H1 per page, a unique title and meta description, a canonical tag pointing to itself, no meta robots "noindex" on pages you want ranked, and internal links from at least one relevant page. Every money page should be reachable within three clicks of the homepage.

Step 5: Kill the Silo-Killers — Duplicate Content

Run a site: search on your domain for a few core terms. If two URLs return for the same query, that is a duplicate-signal problem. Consolidate with 301s or canonical tags and merge the content into a single stronger page.

Step 6: Audit Your Schema and Structured Data

Use Google's Rich Results Test (free) on your article and product pages. In 2026, the schema that earns you AI citations is: Article (with datePublished, dateModified, author), FAQPage, HowTo, and Organization. Structured data is how you hand the AI crawlers a clean map of who wrote what, when, and what it answers.

Step 6b: Check robots.txt Actually Lets the AI Crawlers In

This step did not exist three years ago and it matters now. Several high-profile sites have spent years blocking AI crawlers, then wondered why they vanished from AI answers. Check your robots.txt for the major user agents — GPTBot, PerplexityBot, Google-Extended (the Gemini crawler), ClaudeBot, CCBot (Common Crawl, which trains and is read by several systems), and Bytespider (TikTok). If you want AI citations, they need permission to fetch your pages:

User-agent: GPTBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: ClaudeBot
Allow: /

You can scope it by path if some areas are off-limits — Disallow: /admin/, for instance — but blocking an AI crawler from your public article pages is a choice with a measurable cost. It is one of the cheapest wins on this list: most sites are blocked by default because a template shipped with broad Disallow rules, not because anyone decided deliberately.

Then confirm your sitemap is submitted and clean. An XML sitemap listing only canonical, indexable pages is the fastest way to tell every crawler — Google's and the AI ones — which URLs are real and worth reading.

Where the AI Search Engines Actually Look

The old SEO checklist gets you indexed and ranked in classic results. The 2026 question is whether AI assistants cite you at all. They behave differently, and each platform has its own biases.

Google AI Overviews

AI Overviews pulls heavily from pages that rank top-three for the query, plus pages with strong structured data. The playbook: rank well traditionally first, because the Overview is generated from the same index — then reinforce with FAQPage schema and a clear, direct answer in the first paragraph.

Checklist:

• [ ] Top-3 ranking for your target query (classic SEO still gates the AI result)
• [ ] FAQPage or Article schema validated with Rich Results Test
• [ ] The answer to the question appears in the first 50 words, not buried mid-page
• [ ] dateModified updates when content changes, so the AI treats it as fresh

ChatGPT Search

ChatGPT's browse mode reads full pages and prefers content it can quote verbatim. That rewards clear, self-contained answers with named facts, dates, and numbers — things a model can lift as a citation.

Checklist:

• [ ] Self-contained paragraphs: each section answers its headline without requiring the whole page
• [ ] Concrete data points (numbers, dates, percentages) that read as quotable facts
• [ ] Your brand or author name attached to the claim ("Gulshan Yadav notes…")
• [ ] No heavy JavaScript-only rendering — if your page is a client-side app, it often cannot be read

Perplexity

Perplexity cites sources inline and ranks by a mix of authority and recency. It aggressively values freshness — a slightly lower-authority page published this month beats a stale authority page. Update your money pages on a schedule and say so in the page.

Checklist:

• [ ] Visible, honest publication and modification dates
• [ ] A "last updated" note that changes when content changes
• [ ] High-quality outbound citations to authoritative sources (Perplexity rewards verifiable claims)
• [ ] Pages that read like briefs: a direct answer up top, detail below

Google Gemini

Gemini's grounded answers pull from a blend of web results and structured data, and it gives extra weight to sources that machine-read cleanly. Semantic HTML, logical heading order, and schema are not optional here.

Checklist:

• [ ] Proper heading hierarchy (h1 → h2 → h3, no skipping)
• [ ] Semantic HTML (<article>, <time>, <address> for authors)
• [ ] Organization + Article schema with author fields
• [ ] Plain language: Gemini truncates and rewrites, so shorter paragraphs survive citation better

The Metrics to Track (and the Only Numbers That Matter)

After the fixes, watch four things in Search Console and analytics for a month:

1. Indexed pages vs. discovered pages — the gap should shrink every cycle. This is the crawl-efficiency number.
2. Clicks from position 5–20 — these move first when indexability fixes land, before rankings.
3. Average position on your money keywords — the lagging indicator, but the one that pays.
4. AI citation share — query Perplexity and ChatGPT for your core terms monthly and count how often your site is cited. There is no built-in report for this, so it is a manual check — but it is the metric that tells you whether the AI work is working.

A fifth signal is free if your analytics tool splits crawler traffic: watch whether GPTBot, PerplexityBot, and ClaudeBot are actually hitting your site, and at what frequency. If a page gets crawled by Google but never by an AI crawler, check your robots.txt rules again. Visibility into crawler behavior turns the AI work from guesswork into a measurable loop.

If you want the whole thing automated, I built a free audit tool at MisarSEO that runs the crawl-and-indexability pass for you in one click — it reads Search Console and surfaces the "crawled not indexed" and 404 lists without the spreadsheet work. Free tier handles a starter site, and you keep the paid-free workflow above as the manual checklist.

The brutal truth of technical SEO: your site almost certainly has a handful of obvious failures — 404s, duplicate canonicals, noindex mistakes, slow images — and you cannot rank well until you clear them. The good news is the entire audit costs an afternoon and nothing else. Run the six steps, fix the reasons behind the reports, and give the AI crawlers the clean, quotable pages they are looking for. That is the whole game.

One more habit worth stealing: schedule the audit, do not wait for a traffic drop. A quarterly free pass through these six steps takes two hours and turns technical SEO from a firefight into maintenance. The sites that win the AI-citation game are rarely the biggest — they are the ones that are structurally clean, freshly updated, and allowed to be read by every crawler that matters. That is a competitive advantage you can build for the price of a lunch break.

---

*Gulshan Yad