DEV Community

Multita
Multita

Posted on

Getting a brand-new site read by Google and AI: IndexNow, llms.txt and structured data

A new domain has no authority and no crawl history, so the default is to wait weeks for Google to notice you. You can skip a lot of that wait. When we launched Multita, three cheap and boring pieces did most of the work of getting indexed and getting quoted by AI assistants.

IndexNow: stop waiting for the crawler

IndexNow lets you ping search engines the moment a page changes, instead of waiting to be crawled. You drop a key file at your root and POST the changed URLs. On Cloudflare it is a few lines in a Worker. New pages got picked up in hours, not days.

llms.txt: a map for the models

llms.txt is a plain text file at your root that points AI crawlers at your most useful pages, in clean markdown. It is young as conventions go, but it costs nothing and it tells a model "this is what the site is about" without making it fight your nav and footer to find out.

Structured data: say what you are in a language machines trust

JSON-LD is the part both Google and the models lean on. An Organization and a WebSite block state your name, your URL and what you do, as data instead of guesswork.

<script type="application/ld+json">
{ "@context":"https://schema.org", "@type":"Organization",
  "name":"Multita", "url":"https://multita.com.ar" }
</script>
Enter fullscreen mode Exit fullscreen mode

The payoff is clearer entity recognition, a real shot at a brand panel, and a much higher chance that an AI answer cites you correctly instead of inventing something.

None of this buys authority. It buys legibility, which for a new site is the thing standing between you and being found. Ship all three on launch day.

Top comments (1)

Collapse
 
citedy profile image
Dmitry Sergeev

didn't know about llms.txt, definitely adding that to my next project. wonder if it actually helps with the crawl budget though?