Notify search engines on every GitHub Pages deploy with IndexNow

When I looked on 27 October 2026, Bing Webmaster Tools no longer showed a URL Submission page. It pointed to IndexNow instead. So I added a small GitHub Actions workflow to the repository behind data.yappman.com. After each GitHub Pages deploy, it tells IndexNow which pages changed. This article walks through it. It is our own code.

What IndexNow needs

IndexNow lets a site tell participating search engines that URLs have changed, instead of waiting for them to find out. You publish a key as a text file at your site root. The file is named after the key and contains the key.

That key is public by design. It is not a secret and does not belong in repository secrets. Each request names the file's address in a keyLocation field, so the receiver can fetch it. In our repository the workflow finds the key by looking for a 32-character hex name ending in .txt.

Why only the changed pages? Sending every URL on every deploy tells a search engine nothing about what is new. A diff keeps each submission to what changed. The workflow also needs no secret and no token. Its permissions are contents: read and nothing else.

The trigger and the gate

on:
  workflow_run:
    workflows: ["pages-build-deployment"]
    types: [completed]
  workflow_dispatch:
    inputs:
      base:
        description: "Git ref to compare with HEAD (default: the commit before HEAD)"
        required: false

The workflow runs when GitHub's own pages-build-deployment workflow completes. You can also start it by hand, with an optional base ref to compare against.

if: ${{ vars.INDEXNOW_ENABLED == 'true' && (github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success') }}

The job runs only if the repository variable INDEXNOW_ENABLED is true. Within that, a manual start always passes, and a deploy-triggered start passes only if the deploy succeeded. The file's own header calls it a draft that does nothing until you set the variable (Settings, Secrets and variables, Actions, Variables).

The job checks out the deployed commit with full history (fetch-depth: 0), because the next step needs to compare commits. A concurrency group keeps two pings from running at once.

Work out the changed pages

BASE="$BASE_INPUT"
[ -n "$BASE" ] || BASE="HEAD~1"
git diff --name-only "$BASE" HEAD -- '*.html' > changed.txt
node docs/release/indexnow-urls.js $(cat changed.txt) > urls.txt

With no base given, it compares against the commit before HEAD. git diff --name-only lists the changed .html files. A small Node script then turns file paths into public URLs.

The script is short. It accepts only .html paths. It skips _held/, docs/, assets/ and .github/, the 404 page and the Google verification file. index.html becomes the site root. folder/index.html becomes https://data.yappman.com/folder/. It removes duplicates, sorts the list and prints one URL per line. The step then writes the count of URLs to the step output.

Ping IndexNow

KEY_FILE=$(ls | grep -E '^[0-9a-f]{32}\.txt$' | head -n 1)
KEY="${KEY_FILE%.txt}"
test -n "$KEY"
test "$(curl -fsS "https://data.yappman.com/$KEY_FILE")" = "$KEY"

This step runs only if the count is not zero. It finds the key file, then fetches it from the live site and checks the content matches the key. If the file is not served, the step fails before anything is sent.

curl -fsS -X POST "https://api.indexnow.org/indexnow" \
  -H "Content-Type: application/json; charset=utf-8" \
  --data @body.json -w "\nHTTP %{http_code}\n"

A short inline Node command builds body.json with the host, the key, the key location and the URL list. Then curl posts it to https://api.indexnow.org/indexnow and prints the HTTP status.

Testing it

Set INDEXNOW_ENABLED to true, then start the workflow by hand from the Actions tab. Leave base empty to compare with the previous commit. Or give it an older ref, such as a tag, to cover several deploys in one go. Read the log. The changed-pages step prints the URL list, and the ping step ends with the HTTP status.

First run

Our first run submitted 20 changed URLs and got HTTP 202. I read that as accepted for processing. It does not mean the pages are indexed. Check your own search engine tools later to see what they did with the URLs.

Limits

• IndexNow is a hint, not a guarantee. A search engine decides whether and when to crawl and index a page.
• Google has not announced support for IndexNow. For Google, the sitemap and Search Console remain the routes.
• The workflow only sees .html files that follow the folder/index.html pattern. Changes to other file types are not submitted.
• git diff --name-only also lists deleted files, and the script does not check that a file still exists. A removed page would be submitted as a changed URL.
• If a deploy changes nothing in HTML, nothing is sent.

About the site

data.yappman.com publishes a free weekly UK and Australian shelf-price index. Product-level shelf prices on demand come from the UK Supermarket Price Scraper on the Apify Store: https://apify.com/yappman/uk-supermarket-price-scraper

Drafted with an AI assistant. The workflow described is our own code, and the figures come from our own first run on 27 October 2026.