I Built an AI Search Visibility Checker — and Found Out My Own Site Was Invisible
I launched my first Apify Actor a few weeks ago. Spent time on SEO, structured data, the whole checklist.
Then I asked ChatGPT: "What's the best tool for X in my space?"
My product wasn't mentioned. Claude didn't know I existed. Gemini had the wrong description.
Not because my site was bad. Because I had no idea what AI "sees" when it looks at a page.
So I built something to find out.
The Problem Nobody's Talking About Yet
Most SEO tools measure Google rankings. Keyword positions, backlinks, domain authority. The usual stuff.
But more people are asking ChatGPT directly instead of typing keywords into Google. And LLMs don't evaluate your site the same way search engines do.
A site can have perfect Google rankings, great Core Web Vitals, strong backlinks — and still be invisible to AI. Broken structured data, a misconfigured robots.txt, or the LLM simply never heard of you.
I found this out the hard way. My own site scored 45/100 on the first audit I ever ran.
What I Built
GEO Auditor runs two things in one pass:
Technical audit — crawlability, robots.txt, sitemap, structured data, social meta, performance. Standard stuff, automated.
LLM probes — sends targeted queries to ChatGPT, Claude, and Gemini (via OpenRouter) and checks if each one knows your site exists, and whether it gets the facts right.
The output is a report with scores. JSON, Markdown, or a self-contained HTML page with radar charts if you want the visual version.
Honestly, the hardest part wasn't building the crawler or hooking up the APIs. It was deciding what a "good" score even means when nobody has defined what AI visibility looks like yet.
Things I Learned
robots.txt is the silent killer. The first site I tested was my own blog. The culprit? An old robots.txt I'd copied from a template years ago that blocked /api/. I'd forgotten it was even there. AI crawlers treated the whole site as lower priority because of it.
Structured data parsers are stricter than you think. One beta tester ran the audit and found their JSON-LD had a syntax error that Google's validator didn't catch, but Claude silently failed on. Their products were invisible to AI because the data was technically there but unparseable for some models.
LLM responses vary wildly. I ran the same URL through the tool 10 times over 3 days. ChatGPT found it 8/10 times. Claude 6/10. Gemini 3/10. A single manual check is essentially meaningless. You need a sample size.
Most people don't know they're invisible. I shared the tool with a few indie hacker friends. Everyone thought their site was fine. Most weren't. Nobody checks for this.
What I'd Do Differently
If I were starting this over:
- Test with real users earlier. I spent too long polishing the HTML report before validating that anyone actually wanted LLM probes
- Ship a simpler MVP first. Could have launched with just the technical audit and added LLM probes in v2
- Price it properly. Every LLM probe costs OpenRouter credits. At what I'm charging, it barely breaks even. I'm eating the cost for now to see if people actually use it
How to Check Your Own Site
If you're curious where your site stands, it's on the Apify Store. You paste your URL and your OpenRouter API key, wait about 8 minutes (the LLM probes take time), and get a report.
A few honest limitations:
- You need your own OpenRouter key — I didn't want shared rate limits to mess with results
- ~8 minute runtime, not real-time
- Single page per run right now
- One run is a snapshot, not a definitive verdict
- Only works on public pages
What's Next
I'm still figuring out what "AI visibility" should look like as a metric. A few ideas I'm playing with:
- Tracking visibility over time with weekly scans
- Adding Perplexity and Grok probes
- A simple checklist for non-technical founders who don't want to run a tool
If you've built something similar or have thoughts on how AI citation should work — I'd genuinely like to hear.
Built in public. I'm also on fork.work where I write about building things with Apify.
Top comments (0)