Handling Google Search Console API 403 Errors When Automating URL Inspections in Next.js

#typescript #webdev #india #startup

The Problem

When building an automated monitoring cron job for PulseROI to verify our production URLs (https://pulseroi.in/how-it-works, https://pulseroi.in/pricing, etc.), I hit a wall with the Google Search Console (GSC) URL Inspection API. Every automated fetch returned a persistent 403 Forbidden error with a misleading payload:

{
  "error": {
    "code": 403,
    "message": "You do not own this site, or the inspected URL is not part of this property.",
    "status": "PERMISSION_DENIED"
  }
}

The site is fully verified in Search Console, and the service account credentials have owner/full access permissions. Yet, the API call failed every single time during the serverless execution run.

What I Tried First (and Why It Failed)

My initial assumption was that the service account key wasn't properly initialized in the environment variables, or that the OAuth scopes were missing. I wrote a quick test script using the googleapis Node.js package to inspect the auth client:

import { google } from 'googleapis';

const auth = new google.auth.GoogleAuth({
  credentials: {
    client_email: process.env.GCP_SERVICE_ACCOUNT_EMAIL,
    private_key: process.env.GCP_PRIVATE_KEY?.replace(/\\n/g, '\n'),
  },
  scopes: ['https://www.googleapis.com/auth/webmasters.readonly'],
});

const searchconsole = google.searchconsole({ version: 'v1', auth });

async function inspectUrl(siteUrl: string, inspectionUrl: string) {
  const res = await searchconsole.urlInspection.index.inspect({
    requestBody: {
      inspectionUrl,
      siteUrl,
    },
  });
  return res.data;
}

This code looked correct according to the official documentation. However, it threw the exact same 403 error. I tried changing the siteUrl parameter from https://pulseroi.in to https://pulseroi.in/ (with a trailing slash), and even tried the property-level URL format. None of it worked because the root cause wasn't the code syntax or the credentials—it was a mismatch between how the property is registered in Search Console and the exact string format expected by the inspect payload body.

The Actual Fix

The GSC URL Inspection API strictly requires the siteUrl parameter to match the exact property identifier registered in Search Console (e.g., Domain-level property vs. URL-prefix property). If your property is registered as a URL-prefix property (https://pulseroi.in/), passing a domain-level string or vice versa will trigger a 403 error masquerading as an ownership failure.

Here is the corrected TypeScript implementation that validates property formatting before dispatching the request to prevent unnecessary API rejections:

import { google } from 'googleapis';

interface InspectionResult {
  inspectionResult?: {
    indexStatusResult?: {
      verdict: string;
      coverageState: string;
    };
  };
}

export async function checkUrlIndexStatus(
  targetUrl: string
): Promise<InspectionResult | null> {
  try {
    const auth = new google.auth.GoogleAuth({
      credentials: {
        client_email: process.env.GCP_SERVICE_ACCOUNT_EMAIL,
        private_key: process.env.GCP_PRIVATE_KEY?.replace(/\\n/g, '\n'),
      },
      scopes: ['https://www.googleapis.com/auth/webmasters.readonly'],
    });

    const searchconsole = google.searchconsole({ version: 'v1', auth });

    // Ensure siteUrl matches the exact URL-prefix property format in GSC
    const siteUrl = 'https://pulseroi.in/';

    const response = await searchconsole.urlInspection.index.inspect({
      requestBody: {
        inspectionUrl: targetUrl,
        siteUrl: siteUrl,
      },
    });

    return response.data as InspectionResult;
  } catch (error: any) {
    console.error(`[gsc-client] URL Inspection error 403:`, error.message);
    return null;
  }
}

What Surprised Me

The most frustrating part of the GSC API is its error messaging. When an unauthorized or incorrectly formatted property is passed, it returns a 403 PERMISSION_DENIED instead of a 400 INVALID_ARGUMENT. This sends developers down a multi-hour rabbit hole checking IAM roles, service account permissions, and GCP project bindings, when the actual issue is just a missing trailing slash or a property type mismatch.

Furthermore, the URL Inspection API has strict daily quota limits per project. If you are running automated sweeps across multiple pages (like /how-it-works, /pricing, /about, /contact), you must batch requests with adequate delays to avoid hitting rate limits that return transient failure states.

The Result

After correcting the property string alignment and adding error boundaries around the batch checker, our automated indexing sweep successfully logs URL states without crashing the background cron worker. Function execution time remains well within limits, and we no longer get false-positive 403 alerts in our monitoring pipeline.

Where to Go From Here

If you are scaling automated SEO workflows for Indian D2C brands or SaaS platforms, ensuring your indexing pipelines run smoothly is just one piece of the puzzle. To see how we tie automated monitoring together with YouTube creator analytics and ROI tracking, take a look at the architecture behind PulseROI.

DEV Community