DEV Community

Cover image for I built a tool to check what AI agents actually understand about your website
Tommy
Tommy

Posted on

I built a tool to check what AI agents actually understand about your website

There's a gap between what a website says about itself and what an AI agent can actually verify from its structured data.
I got curious about this after noticing that different LLMs were giving inconsistent answers about the same company. Some hallucinated services that didn't exist. Others missed the core business entirely. The common thread: the sites had no llms.txt, incomplete JSON-LD, or structured data that contradicted the homepage copy.
So I built Lagotto Meter — it fetches llms.txt, llms-full.txt, and JSON-LD from any URL, passes them to Llama 3.3 70B (or Gemini 2.5 Flash as fallback), and scores how well the structured data supports the claims the site makes about itself.
It's not an SEO tool. It doesn't crawl content. It only reads what you've explicitly declared for AI agents — and checks whether those declarations are coherent and complete.
The scoring is:

llms.txt presence and quality: 0–25
llms-full.txt: 0–15
JSON-LD @graph completeness: 0–20
robots.txt / sitemap: 0–10
Semantic coherence (LLM judgment): 0–30

Sites with perfect technical structure but no verifiable claims about clients or results get penalized on coherence. The model is looking for proportionality between what you declare and what you can prove.
Try it on your own site or any site you're curious about:

https://lake8.dev/lagotto-meter
Prompt is public and replicable. Free, 1 analysis per IP per 24h.

Top comments (0)