Our IndexNow script will not run against localhost, and fetches our own key file before it submits

A sitemap is a standing statement about which pages exist. IndexNow is an event saying which ones just changed, and the engines that take part act on it in minutes rather than whenever they next feel like crawling. Bing, Yandex, Seznam and Naver participate. Google does not, so it is additive to a Search Console sitemap and never a replacement.

The protocol itself is about as small as a protocol gets. You publish a key at a URL on your own host, then POST a JSON body naming the host, the key, the URL the key is published at, and the pages that changed. Ownership is proved by the fact that both halves agree.

Which is lovely until something is wrong, because the failure modes all arrive as the same three digits:

403 Forbidden

That is the entire response. It means the receiving end could not fetch your key file, or it fetched it and the contents did not match. It does not say which. Our key file lives at pub-trivia.app/2382c7da66fd467eb45e9c97aadf605a.txt, and you can read it, because being readable is the whole mechanism: the key is a proof of control over the host, not a secret.

So the submitter for pub-trivia.app is a 90-line script that spends most of its length refusing to send the request.

Guard one: not the production site

if (!IS_PRODUCTION_SITE) {
    fail(
        `SITE_URL is ${SITE_URL}, not the production site. ` +
            'Run via `pnpm indexnow` so NEXT_PUBLIC_WEBSITE_URL is loaded.'
    )
}

SITE_URL falls back to localhost outside production. A submission of localhost URLs comes back as a 422, and a 422 in this protocol also means "the key does not match", so the ten minutes you then spend staring at your key file are ten minutes the script could have saved you by reading one environment variable.

Guard two: a path that is not in the sitemap

const unknown = paths.filter((path) => !INDEXABLE_ROUTES.includes(path))

The URL list is read from the same constant the sitemap is generated from, rather than typed out again in the script. That is not tidiness. Three route lists on this site were once kept by hand and they disagreed, which is how a page ended up in the sitemap while every crawler that followed the link got bounced to the login page.

With the list shared, naming a path that is not in it has exactly two causes, and both are worth stopping for: you typed it wrong, or the page is not published yet and the thing to fix is the registry entry rather than the submission.

Guard three: fetch our own key file first

const response = await fetch(url, { redirect: 'manual' })

if (response.status >= 300 && response.status < 400) {
    const target = response.headers.get('location') ?? 'elsewhere'
    return {
        ok: false,
        url,
        detail: `redirects (${response.status}) to ${target} ` +
            '- the key file must return 200 with the key as its body',
    }
}

redirect: 'manual' is the load-bearing option. The likeliest failure on this particular site is not a typo in the key, it is the auth gate deciding the key file is a page that needs a session and answering with a 307 to the login form. Follow redirects and that reads as a 200 with some HTML in it; leave them unfollowed and the script can tell you the URL it was sent to instead.

It also checks the body, byte for byte, against the constant, because serving the key with a trailing newline from the wrong template is a failure the API reports as the same bare 403.

Guard four: one stray origin fails all of them

const offOrigin = paths.filter((path) => !path.startsWith('/'))
if (offOrigin.length > 0) {
    throw new Error(
        `IndexNow: paths must be root-relative, got ${offOrigin.join(', ')}.`
    )
}

Every URL in a submission has to be on the host the submission names. One that is not does not get skipped, it rejects the whole batch with a 422. Seventy-two URLs failing because of the seventy-third is worth catching locally.

Then it says what the status meant

export const INDEXNOW_STATUS_MEANING: Record<number, string> = {
    200: 'accepted',
    202: 'accepted, key validation pending',
    400: 'bad request - the payload is malformed',
    403: 'forbidden - the key file could not be fetched or did not match',
    422: 'unprocessable - a URL is not on this host, or the key does not match',
    429: 'rate limited - too many submissions',
}

Six numbers and six sentences, so that the one time a year this runs and fails, nobody has to go and find the spec. The success line ends with the other thing worth remembering:

✓ 72 URLs submitted - 200 accepted
  Bing reflects this in Webmaster Tools within a few hours; Google ignores it.

The default is the whole sitemap, and that is wrong after the first time

pnpm indexnow with no arguments submits everything. pnpm indexnow /guides/pub-quiz-format submits one page. --dry-run prints the payload and sends nothing.

The first of those is right exactly once. After that, re-submitting seventy-two unchanged pages on every deploy is a signal that says "everything changed" every time, which teaches the receiving end to weight it accordingly. IndexNow is a claim about change, and a claim made indiscriminately is not worth much.

Ten tests cover the parts that can be tested without the network: that the key matches the shape the spec accepts, that the path is named after the key, that a route exists behind it, that the file responds as text/plain with the key and nothing else, and that the whole sitemap fits inside one request. One of them is just this:

it('is reachable without a session', () => {
    expect(isPublicRoute(INDEXNOW_KEY_PATH)).toBe(true)
})

That assertion exists because the symptom of its absence is a 403 from a third party, days later, with nothing in it pointing at the auth gate.

Check it yourself

Both halves of this are public, so you can run the same checks we do:

# the key file: 200, text/plain, the key as the entire body
curl -i https://pub-trivia.app/2382c7da66fd467eb45e9c97aadf605a.txt

# the URL list the submitter reads, as the sitemap renders it
curl -s https://pub-trivia.app/sitemap.xml | grep -c "<url>"

The second one answers 72 today, and that number is the submitter's argument list. Most of those lastmod values come from the date the page's content was actually edited rather than from the build clock, which is the other half of not crying wolf: one of them says 2026-10-03 because the about page is the page that changed that day, and that is the single path the script would have been handed. The three with a timestamp rather than a date are the legal documents, which have no content-registry entry to read a date from.

If you want to see what the submitted pages actually are, the guides hub and the free quiz night tools are two of the seventy-two, and the tools pages are the ones worth a click: they run entirely in your browser, so you can plan a running order or print table QR codes without an account.