DEV Community: SEO by Subham

Your Backlinks Are Indexed But Invisible to AI Search — I Built a Free Checker to Prove It

SEO by Subham — Fri, 28 Aug 2026 00:31:23 +0000

The Backlink That Google Loved and ChatGPT Never Heard Of

A few weeks back I published a Google Sheets tool to check backlink indexation with SerpApi — a simple, manual-only script that tells you whether a backlink has actually been picked up by Google, instead of guessing.

It worked well. One backlink I was tracking got indexed by Google in under 72 hours. Good news, right?

Then, out of curiosity, I opened ChatGPT and asked it about the exact page that backlink pointed to.

It had no idea the page existed.

Not "I don't have real-time data." Not a vague guess. It simply didn't know the content was there — despite Google indexing it days earlier.

That gap is the entire premise of this post: being indexed by Google and being visible to AI search are two completely different things, and almost nobody is measuring the second one.

So I extended my original checker to test for both.

Why "Indexed" Doesn't Mean "Visible" Anymore

For over a decade, indexation was the finish line. If Google crawled your page and added it to the index, you were in the game — rankings were just a matter of time and optimization from there.

That assumption is breaking down.

A growing share of search now happens inside AI systems — ChatGPT, Perplexity, and Google's own AI Overviews — that don't work like the traditional index. They don't list ten blue links; they synthesize an answer from a handful of sources they trust enough to cite. A page can be perfectly indexed and still never make it into that shortlist.

For anyone doing SEO or link building, that means a new failure mode nobody's watching for: a backlink that "worked" by every traditional metric, but contributes zero visibility where a growing number of searches actually happen.

I wanted a way to see that gap directly, on my own sites, without relying on vibes.

What I Built: An Indexation + AI Visibility Checker

The original tool answered one question per backlink: is this indexed?

The updated version answers three:

Is the backlink indexed by Google? (unchanged from the original tool — SerpApi site: query)
Does the target page appear in Google's AI Overview for the keywords it's meant to rank for?
Does a general-purpose LLM (ChatGPT/Perplexity via API) recognize the page or domain at all when asked directly?

Each backlink row in the Google Sheet ends up with three simple signals instead of one, plus a combined score so I can sort and prioritize at a glance.

The Technical Walkthrough

Same stack as before: Python, the SerpApi library, and Google Sheets as the interface — no dashboard, no database, nothing to maintain.

1. Indexation check (recap)

This part is unchanged from the original tool — a straightforward site: search through SerpApi to confirm the URL is indexed:

import requests

def check_indexation(url, serpapi_key):
    params = {
        "engine": "google",
        "q": f"site:{url}",
        "api_key": serpapi_key
    }
    response = requests.get("https://serpapi.com/search", params=params)
    results = response.json()
    return bool(results.get("organic_results"))

2. Google AI Overview check

SerpApi's Google engine returns an ai_overview block when one is present for a query. I check whether my domain shows up anywhere inside its sources:

def check_ai_overview(keyword, target_domain, serpapi_key):
    params = {
        "engine": "google",
        "q": keyword,
        "api_key": serpapi_key
    }
    response = requests.get("https://serpapi.com/search", params=params)
    results = response.json()

    ai_overview = results.get("ai_overview", {})
    sources = ai_overview.get("references", [])

    cited = any(target_domain in src.get("link", "") for src in sources)
    return {
        "has_ai_overview": bool(ai_overview),
        "domain_cited": cited
    }

If there's no AI Overview at all for that keyword yet, I record that too — it's a useful signal on its own.

3. Direct LLM recognition check

This one is more experimental, and I want to be upfront about that. I send a small, consistent prompt to the OpenAI API asking whether the model has knowledge of the page or domain, and log the raw response for manual review rather than trying to auto-parse a yes/no:

from openai import OpenAI

client = OpenAI(api_key="YOUR_OPENAI_KEY")

def check_llm_recognition(url, domain):
    prompt = (
        f"Are you aware of the website or page at {url} "
        f"(domain: {domain})? If so, briefly describe what it's about. "
        f"If you have no knowledge of it, say so directly."
    )
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        max_tokens=150
    )
    return response.choices[0].message.content

I run the same prompt structure against Perplexity's API too, since its answers are grounded in live search rather than training data alone — it tells me something different: not "does the model remember this," but "would this page surface in a real-time AI search right now."

4. Combining it into one sheet

Each backlink gets logged as a row with:

Backlink URL	Indexed?	AI Overview exists?	Domain cited in Overview?	LLM recognizes page?	Notes

I don't collapse this into a single fake "score out of 100" — SEO tools do that too often and it hides more than it reveals. Instead I flag any row where Indexed = Yes but both AI columns are No. Those are the interesting rows. That's the actual gap.

What I Found Running It

Out of the backlinks I re-checked from my last batch:

Every single one was indexed by Google, as expected.
Roughly a third appeared in an AI Overview for their target keyword.
Almost none were recognized directly by the LLM when asked cold — which makes sense, since a backlink built last month isn't in any model's training data yet.

The AI Overview number was the useful one. It's the closest thing to a real-time "does AI search actually see this" signal available right now, and it's a much smaller percentage than the indexation rate. That gap is exactly what I was trying to surface.

Why This Is Still Manual-Only

Same reasoning as the original tool, and it applies even more here:

API costs add up fast. LLM calls aren't free, and neither are SerpApi credits. Automating this into a daily cron job for hundreds of URLs would get expensive quickly for very little marginal insight.
AI answers aren't stable. Ask the same question twice and you can get different phrasing, or a different verdict entirely. Automating on top of a non-deterministic signal just produces noisy automation, not a reliable pipeline.
This is a diagnostic, not a dashboard. I run it in batches when I want a checkpoint, review the raw answers myself, and move on. That's a better use of the signal than pretending it's precise enough to monitor continuously.

Manual-only isn't a limitation here — it's the correct way to use a signal that's still this noisy.

Limitations, Honestly

A few things worth being upfront about, since overselling this would defeat the point of writing it:

LLM "recognition" is not a real index. There's no API that tells you definitively what a model does or doesn't know — asking it directly is a proxy, not ground truth.
AI Overviews don't appear for every query, and their presence/absence can shift between runs.
None of this replaces actual traffic and conversion data. It's a leading indicator, not proof of impact.

If you're building something similar, treat every AI-visibility signal as directional, not authoritative.

Try It Yourself

The full script (indexation + AI Overview + LLM recognition check) is a single Python file plus a Google Sheet template — no server, no signup, nothing to host. If you want to run it on your own backlinks, I'm happy to share the setup in the comments or a follow-up post.

If you've noticed the same indexed-but-invisible gap on your own sites, I'd genuinely like to hear what you found — drop it below.

I write and build tools like this as part of my link-building work at SEO by Subham — if you're curious about the manual, white-hat approach behind the original indexation checker too, it's all there.

I built a Google Sheets tool to check backlink indexation with SerpApi, and why I made it manual-only

SEO by Subham — Sun, 23 Aug 2026 10:40:55 +0000

I do SEO and link-building work at seobysubham.com, and one task kept feeling unnecessarily repetitive: checking whether the pages containing backlinks were actually indexed by Google.

The process is simple in theory. Take a list of backlink URLs, search for each one, and determine whether Google has the page in its index. Doing that manually for a large list is slow, especially when the URLs are already sitting in a Google Sheet. That was the main reason I decided to build the checker directly inside Sheets rather than as a separate app.

I didn't want another small web application with its own login, database, hosting, and UI. The people using the data already had a spreadsheet. Putting the results there meant there was no extra interface to learn, no separate auth flow, and the rest of the team could see results in the same document they already worked from.

Why Google Apps Script?

Apps Script was a natural fit because it sits directly inside Google Workspace. A spreadsheet can have an Apps Script project attached to it, and that script can read and write cells using the Sheets API available through the Apps Script runtime. It also lets me add a custom menu to the spreadsheet, so the checker starts from the UI instead of running as a completely separate application.

The architecture is deliberately small:

Google Sheet
    |
    | Backlink URLs
    v
Apps Script
    |
    | HTTP request
    v
SerpApi
    |
    | Search response
    v
Apps Script
    |
    | Parse result
    v
Google Sheet
    |
    +-- Indexed
    +-- Not Indexed
    +-- Error

No dedicated server involved. The spreadsheet is effectively the interface, and Apps Script handles the application logic. Backlink URLs live in one column, indexation results get written into another.

The core logic

The production version has more validation and error handling, but the basic implementation is fairly straightforward. For every backlink URL, the script sends a request to SerpApi. The response is JSON, so the script can inspect the returned search results and decide whether the target URL appears in Google's results.

A simplified version looks something like this:

function checkBacklinkIndexation() {
  const sheet = SpreadsheetApp.getActiveSpreadsheet()
    .getSheetByName("Backlinks");

  const apiKey = PropertiesService
    .getScriptProperties()
    .getProperty("SERPAPI_KEY");

  const rows = sheet.getDataRange().getValues();

  // Assume column A contains backlink URLs
  for (let i = 1; i < rows.length; i++) {
    const backlinkUrl = rows[i][0];

    if (!backlinkUrl) {
      continue;
    }

    const query = `"${backlinkUrl}"`;

    const endpoint =
      "https://serpapi.com/search.json" +
      "?engine=google" +
      "&q=" + encodeURIComponent(query) +
      "&api_key=" + encodeURIComponent(apiKey);

    try {
      const response = UrlFetchApp.fetch(endpoint, {
        muteHttpExceptions: true
      });

      const data = JSON.parse(response.getContentText());

      const indexed = data.organic_results?.some(result =>
        result.link === backlinkUrl
      );

      sheet.getRange(i + 1, 2).setValue(
        indexed ? "Indexed" : "Not Indexed"
      );

    } catch (error) {
      sheet.getRange(i + 1, 2).setValue("Error");
    }
  }
}

This is intentionally simplified. In a real implementation, I wouldn't rely on an exact string comparison alone, since URLs can differ by trailing slashes, query parameters, redirects, encoding, or canonical versions. The important part is the data flow rather than this exact comparison: read the rows, skip empty URLs, build a search request, send it to SerpApi, parse the JSON response, determine whether the target appears in the returned results, and write the status back into the spreadsheet.

I also keep the API key in Apps Script's script properties rather than hardcoding it into the source. That makes configuration easier to manage and avoids accidentally leaving the key inside a function that might get copied around.

Why I intentionally made it manual-only

This was probably the most deliberate engineering decision in the project.

Apps Script supports time-driven triggers, so technically I could make the checker run automatically every hour, every day, or on some other interval. I decided not to. The reason is API cost.

Every backlink check results in an external API query, and SerpApi charges per search. An automatic trigger can quietly consume the available quota. Imagine a spreadsheet with 500 URLs — if an automated process checks all of them daily, that's potentially 15,000 searches a month, and the user might not notice until the quota is already gone.

For this particular tool, automation isn't necessarily an improvement. So instead, I added a custom menu and the user explicitly starts the process:

Open spreadsheet
       |
       v
Review backlink list
       |
       v
Run "Check Indexation"
       |
       v
API requests are made
       |
       v
Results appear in the sheet

This makes API usage visible, and it gives me better control when testing. I can add a few URLs, run the checker, inspect the output, and stop there. The trade-off is obvious: manual execution is less convenient than a scheduled job. But here I preferred predictable API usage over invisible automation.

The technical problem I ran into

One issue appeared once I started testing with larger backlink lists. The first implementation treated the spreadsheet almost like a normal JavaScript array and made an individual request for every row, followed immediately by an individual spreadsheet write. That works fine for a small test. It becomes inefficient with hundreds of URLs, because there are two different external operations happening repeatedly — a sheet read, an API request, a sheet write, over and over.

Apps Script has execution time limits, and every network request adds latency. Writing to Sheets cell-by-cell also creates unnecessary overhead.

The fix was to read the spreadsheet data once and collect results in memory instead of writing on every iteration:

const results = [];

for (const url of urls) {
  const status = checkUrl(url);
  results.push([status]);
}

Then write the complete result range in a single operation:

sheet
  .getRange(2, 2, results.length, 1)
  .setValues(results);

Small change, but it matters once the spreadsheet gets larger. I also learned that API response handling needs to be defensive — a successful HTTP request doesn't automatically mean the expected JSON structure exists. An API can return an error, a quota message, or a response missing the expected fields. So the production version checks the HTTP response and expected fields before deciding a URL is "Not Indexed." An API failure and a confirmed non-indexed URL are two completely different states, and conflating them silently would've made the data untrustworthy.

What I'd improve next

A few things I'd change if I keep developing this.

Batching. Large lists should be processed in controlled chunks rather than treating the entire spreadsheet as one job.

Caching. If I checked a URL yesterday and it hasn't changed, there's little reason to spend another API query re-checking it. Storing the last-checked date and result would let me re-check only URLs that actually need another lookup.

Better error reporting. Instead of just writing "Error," the sheet could store the actual HTTP status, a quota message, or a note on the malformed response — useful context instead of a dead end.

A progress indicator. Since API requests take time, something like "42 / 200 checked" would make a manual run easier to follow.

The basic version is already useful because it removes a repetitive lookup process from the browser and puts the result directly next to the backlink data it's describing. The bigger lesson for me was that a small internal tool doesn't always need a complicated architecture — Google Sheets, Apps Script, and one external API were enough. The more interesting engineering decisions were around API cost, execution limits, error states, and knowing when automation actually helps versus when it just hides a cost you'll notice later.

Making the tool manual-only wasn't a limitation I forgot to solve. It was the feature that kept the system predictable.