DEV Community: Momen Abdelhafez

Bulk Indexing Your URLs, Automated (No More One-by-One API Calls)

Momen Abdelhafez — Wed, 01 Jul 2026 21:23:33 +0000

If you've ever tried to get more than a handful of URLs indexed by Google, you already know the pain: Google's Indexing API works fine for one page — and then falls apart the moment you're dealing with hundreds.

Here's what that actually looks like, why it breaks down at scale, and how I ended up automating it.

The One-URL Version Works Fine

Google's Indexing API is straightforward for a single request:

async function submitUrl(url) {
  await jwtClient.authorize();
  const res = await fetch(
    'https://indexing.googleapis.com/v3/urlNotifications:publish',
    {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${jwtClient.credentials.access_token}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ url, type: 'URL_UPDATED' }),
    }
  );
  return res.json();
}

Call it once, get a 200, move on. Nice and clean.

Where It Breaks Down

Now do that for 500 URLs — new blog posts, product pages, backlinks you built last week. A few problems show up fast:

Quota limits. The API caps requests per day. Loop over 500 URLs naively and you'll hit the ceiling.
No batch status view. The API tells you a request was accepted — not whether the URL is actually indexed. Those are different things.
Retry logic. Some submissions fail silently or need a re-try. You end up building your own queue.
Tracking over time. Indexing isn't instant. You need something polling status days later, not just firing requests once. None of this is hard, exactly — it's just infrastructure you have to build and maintain for a problem that isn't your core product.

What I Built to Skip This

I ended up wrapping all of this into GoogleIndexer — free, handles the batch/quota/retry/tracking problem so you don't have to:

Paste or upload a list of URLs — submits the whole batch
Tracks each URL's status automatically (pending → indexed)
Confirms the moment each one goes live in search
Works for pages and backlinks alike It's the same underlying API problem, just with the queueing and status-polling already handled.

When to Roll Your Own vs Use a Tool

Honestly — if you're only ever submitting a handful of URLs, the raw API snippet above is all you need. Where a tool like this starts to matter is once you're publishing or building links at real volume and don't want to own a whole submission pipeline just to know whether your content is actually discoverable.

If you want to try it: google-indexer.com — free, no credit card.

Would love to hear how others here are handling bulk indexing — anyone built their own queue system for this, or found other tooling that works well?

Get Indexed on Google for free and in less than 24h!

Momen Abdelhafez — Wed, 01 Jul 2026 21:20:05 +0000

Momen Abdelhafez

Jul 1

Why Your Pages Aren't Showing Up on Google (And How I Fixed It)

#seo #webdev #googleindexing

2 min read

Why Your Pages Aren't Showing Up on Google (And How I Fixed It)

Momen Abdelhafez — Wed, 01 Jul 2026 21:05:46 +0000

You ship a page. You check Google a few days later. Nothing.

If you've built a product, a blog, or a side project and wondered why it's invisible in search, this post is for you. It's not about "SEO tricks" — it's about a step almost every dev skips: confirming your pages are actually indexed.

Publishing ≠ Indexed

This trips up a lot of technical people, because it feels like it should just work. It doesn't. Before Google can show your page in search results, it goes through three stages:

Discovery — Google learns the URL exists
Crawling — Googlebot fetches and reads the page
Indexing — Google stores it in its searchable database Only step 3 makes you findable. A page that's live on your server but not indexed is functionally invisible to Google — zero impressions, zero clicks, zero rankings. Same goes for backlinks: a link on a page that isn't indexed passes no authority at all.

The Real Fix: Google's Indexing API

Google actually offers a way to skip the wait — the Indexing API. Instead of hoping Googlebot finds your page on its normal crawl schedule, you notify Google directly that a URL is ready.

The basic flow:

# 1. Enable the Indexing API in Google Cloud Console
# 2. Create a service account, download the JSON key
# 3. Add the service account as an Owner in Search Console
# 4. Authenticate and POST to the endpoint

const { google } = require('googleapis');

const jwtClient = new google.auth.JWT(
  serviceAccount.client_email,
  null,
  serviceAccount.private_key,
  ['https://www.googleapis.com/auth/indexing'],
  null
);

async function submitUrl(url) {
  await jwtClient.authorize();
  const res = await fetch('https://indexing.googleapis.com/v3/urlNotifications:publish', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${jwtClient.credentials.access_token}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ url, type: 'URL_UPDATED' }),
  });
  return res.json();
}

This works well — for one or two URLs. Where it gets annoying: managing service accounts, handling quotas, writing your own status-tracking logic, and checking whether each URL actually made it into the index (submitting ≠ indexed, same problem as before, just one layer down).

What I Built Instead

I got tired of wiring this up manually every time, so I built GoogleIndexer — a free tool that wraps the whole workflow:

Submit URLs (and backlinks) in bulk — no API setup required
Automatic status tracking per URL — pending → indexed
Clear confirmation when a page actually goes live in search It's the same underlying problem — submit, then verify — just without writing and maintaining the plumbing yourself. Free, no credit card, built because I needed it for my own projects.

TL;DR

Publishing a page doesn't mean it's indexed. Confirm, don't assume.
Google's Indexing API lets you request indexing directly instead of waiting on the crawl schedule.
Setting it up yourself is doable but has real overhead (auth, quotas, tracking).
GoogleIndexer handles bulk submission + tracking for free if you'd rather skip the setup. Curious how others here handle indexing at scale — anyone rolled their own tracking system, or found a good OSS option?