DEV Community

VOXA
VOXA

Posted on

I added schema, llms.txt and IndexNow to 116 pages. Here is what changed

#ai #seo #webdev

cover

I added structured data, llms.txt and IndexNow to 116 pages of a new site. Here is what actually changed in Search Console.

Not a growth story. A list of what I added, what the numbers look like after a month, and the parts of AEO advice that turned out to be noise.

The starting point

The site is a one-month-old publication about AI tools. 116 published pages when I started this pass: 43 guides, 31 news posts, 23 notes and 19 utility pages, about 89,000 words of body copy, built as hand-written HTML on Cloudflare Pages. No CMS, no plugins.

Search Console at the start: a handful of impressions, a few clicks, most pages discovered but not indexed.

The problem I was trying to solve was not ranking. It was that AI answers were quoting other sites for questions this site already answered, and the obvious reason is structural: a crawler that reads the page cannot tell who wrote it, when, what question it answers, or whether it is allowed in.

What I added

Five things, in this order.

1. One JSON-LD block per page, and only one. Every article carries a single Article or NewsArticle block. Every page with visible Q&A carries a FAQPage block generated from the questions on the page itself. The most common mistake I made early was adding a second Article block from a template while the page already had one, which makes both blocks invalid.

{
  "@context": "https://schema.org",
  "@type": "NewsArticle",
  "headline": "What the post is about",
  "datePublished": "2026-11-09T18:30:00+05:30",
  "author": { "@type": "Organization", "name": "VOXA" },
  "mainEntityOfPage": { "@type": "WebPage", "@id": "https://example.com/the-page" }
}
Enter fullscreen mode Exit fullscreen mode

2. Dates that match the visible byline. If the schema says 9 November and the page says 6 November, you have told the crawler that the page is inconsistent. This one is boring and it is the one I got wrong most often.

3. An llms.txt file. A short markdown file at the root, listing who publishes the site, what it covers, and where the main pages are. It is not an official standard and Google does not read it. I added it because it costs ten minutes and some AI crawlers do look for it. I would not spend more than ten minutes on it.

4. A robots.txt that names the crawlers. Instead of one wildcard rule, the file lists GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot and ClaudeBot with an explicit Allow. If you block AI crawlers by accident - which many security plugins do by default - nothing else in this list matters.

5. IndexNow. A small script that pings the IndexNow endpoint when a page is published, so participating search engines hear about it in minutes rather than waiting for a crawl. Google does not participate. Bing and several others do. It is a two-line call and it is free.

The numbers after a month

From Search Console, last 28 days, read the day I wrote this:

  • 110 pages indexed, out of 116 submitted
  • 2,610 impressions
  • 11 clicks
  • average position 18.4
  • 209 distinct queries, the top ones being tool pricing questions

Six pages were still sitting in "Discovered - currently not indexed" or "Crawled - currently not indexed". Two URLs were 404s that Google had found somewhere on the web. One of those was a URL that never existed on the site at all, which is a reminder that Search Console reports on the whole web's opinion of your domain, not just your sitemap.

What that means honestly: the structure work did not produce traffic on its own. Eleven clicks is eleven clicks. What it did produce is a site where every page is legible to a machine, and a clean baseline to measure against. If you are starting from zero, the ceiling here is low and the work is unglamorous.

What did not work, or is not what people say it is

Schema does not rank you. It can make a page eligible for a rich result, and it can help an AI system understand the page. It will not move a page from position 40 to position 3.

llms.txt is not a standard. Treat claims about it as claims, not as documentation.

IndexNow is not Google. If your plan is "get indexed fast", Search Console's URL inspection is still the tool that matters for Google.

Adding schema to a thin page makes a thin page with schema. Nothing in this list substitutes for the page answering a question better than the next result.

The checklist I use before publishing

  1. One H1, and it matches the title closely
  2. Canonical points at this URL, not a variant
  3. Exactly one Article/NewsArticle block, with a date matching the byline
  4. FAQPage only if the questions are visible on the page
  5. Author and publisher named in the schema
  6. robots.txt allows the AI crawlers you want
  7. Sitemap entry added and submitted
  8. Internal links: at least two, pointing at related pages
  9. The page returns 200 with no redirect chain
  10. Rich Results Test passes before it goes live

That is the whole setup. It took a few evenings and it is now a template I reuse.

I packaged the pieces I use into a $29 kit - six JSON-LD templates, an llms.txt starter, a robots.txt snippet, an IndexNow script, the checklist and an 8-page setup guide for plain HTML, WordPress and Shopify. It is my own product, so treat this paragraph as a disclosure rather than a recommendation: voxahq.in/aeo-kit.

Originally published at voxahq.in.

Top comments (0)