How to run bulk PageSpeed Insights checks for hundreds of URLs (and export Core Web Vitals to CSV)

Hill Country Bots — Thu, 12 Nov 2026 14:17:38 +0000

The PageSpeed Insights website is fine when you have one URL. It is a slow way to check a sitemap, a client list, or every page you shipped last month. You want one table with Core Web Vitals, a performance score, and a file you can open in a spreadsheet.

There are two honest ways to do that. The first is Google's official PageSpeed Insights API, which you can call from a short Python script. The second is a bulk runner we built on Apify, Bulk PageSpeed Checker, which calls that same API, retries failures, and lets you download CSV from the console if you do not want to write code.

Lab data and field data are not the same thing

A PageSpeed response has two kinds of numbers, and they answer different questions.

Lab data comes from Lighthouse. Google loads the page in a simulated browser, usually a phone, and reports metrics from that one run: Largest Contentful Paint (LCP), Cumulative Layout Shift (CLS), Total Blocking Time, First Contentful Paint, Speed Index, and a performance score from 0 to 100. That score can move from run to run, because it is one test on one day.

Field data comes from the Chrome User Experience Report, often called CrUX. When Google has enough real-user traffic for the page, you get the 75th percentile for LCP, Interaction to Next Paint (INP), and CLS, each marked FAST, AVERAGE, or SLOW. That is what people actually experienced, not what one lab test guessed.

Google's PageSpeed Insights API docs say they plan to stop including CrUX data in this API, and they point people to the CrUX API instead. There is no public cutoff date on that page. A run we did on November 11, 2026 still returned field data, and the examples below use those numbers. If those fields go away later, lab scores will still be there, and the field columns may come back empty unless you also call the CrUX API.

Call the official API yourself

You can try the API without a key, but Google tells you to use a key for frequent, automated queries. Create one in the Google Cloud Credentials page, enable the PageSpeed Insights API, and keep the key in an environment variable. Do not paste it into the script.

The official endpoint is a GET request:

import os
import time
import csv
import requests

API = "https://www.googleapis.com/pagespeedonline/v5/runPagespeed"
KEY = os.environ["PSI_API_KEY"]
URLS = [
    "https://www.python.org/",
    "https://www.wikipedia.org/",
    "https://web.dev/",
]

def p75(metrics, name):
    item = (metrics or {}).get(name) or {}
    return item.get("percentile")

def check(url, strategy="mobile"):
    response = requests.get(
        API,
        params={"url": url, "strategy": strategy, "key": KEY},
        timeout=120,
    )
    response.raise_for_status()
    data = response.json()
    field = (data.get("loadingExperience") or {}).get("metrics") or {}
    categories = (data.get("lighthouseResult") or {}).get("categories") or {}
    perf = (categories.get("performance") or {}).get("score")
    return {
        "url": url,
        "performance": int(perf * 100) if perf is not None else None,
        "field_lcp_ms": p75(field, "LARGEST_CONTENTFUL_PAINT_MS"),
        "field_inp_ms": p75(field, "INTERACTION_TO_NEXT_PAINT"),
        "field_cls": p75(field, "CUMULATIVE_LAYOUT_SHIFT_SCORE"),
    }

rows = []
for url in URLS:
    rows.append(check(url))
    time.sleep(1.5)

with open("pagespeed.csv", "w", newline="") as handle:
    writer = csv.DictWriter(handle, fieldnames=rows[0].keys())
    writer.writeheader()
    writer.writerows(rows)

The sleep is there on purpose. Check the Quotas tab for the PageSpeed Insights API in your own Google Cloud project, rather than guessing a daily limit, and retry when you get a 429. A 500 can also show up under load. Give that request another try after a pause, then record the failure and move on. One dead URL should not stop the rest of the list.

This is enough for a short list. At a few hundred URLs, the boring parts take over. You need pacing, retries, a timeout, and a stable CSV schema. The raw JSON is also nested, with different key names for lab audits and field metrics, so flattening it by hand gets old fast.

A no-code path, or a few lines with apify-client

That is why we built the Bulk PageSpeed Checker. It is a small paid Actor on Apify. We are Hill Country Bots, and we should say that plainly. The Actor still calls Google's official PageSpeed Insights API. It does not invent its own scores.

If you do not want to write code, open the Actor, paste your URLs, choose mobile, desktop, or both, and click Start. When the run finishes, download the dataset as CSV or Excel from the Apify console. Failed audits show up as a row with a status and an error message, and those rows are not charged. The live price is $3 per 1,000 successful audits. You can cap a run with maxResults so a long list cannot run away with the bill. For a very large or daily job, you can pass your own Google API key; it is stored encrypted and sent only to Google.

If you already write Python, the official apify-client package does the same job:

import os
import csv
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("hillcountrybots/bulk-pagespeed-checker").call(
    run_input={
        "urls": [
            "https://www.python.org/",
            "https://developer.mozilla.org/",
            "https://www.gov.uk/",
            "https://www.wikipedia.org/",
            "https://web.dev/",
        ],
        "strategy": "mobile",
    }
)

rows = list(client.dataset(run["defaultDatasetId"]).iterate_items())

with open("pagespeed.csv", "w", newline="") as handle:
    writer = csv.DictWriter(
        handle,
        fieldnames=[
            "url",
            "status",
            "performance",
            "field_lcp_ms",
            "field_inp_ms",
            "field_cls",
            "error",
        ],
    )
    writer.writeheader()
    for row in rows:
        field = row.get("fieldData") or {}
        writer.writerow(
            {
                "url": row.get("url"),
                "status": row.get("status"),
                "performance": (row.get("scores") or {}).get("performance"),
                "field_lcp_ms": (field.get("lcpMs") or {}).get("p75"),
                "field_inp_ms": (field.get("inpMs") or {}).get("p75"),
                "field_cls": (field.get("cls") or {}).get("p75"),
                "error": row.get("errorMessage"),
            }
        )

Put your Apify token in APIFY_TOKEN. Do not hard-code it. New Apify accounts get a small monthly credit, which is enough to try this on a short list.

What a real run looks like

The JSON below is from a real run of this Actor (run 1SBNdSduv5ONdxgpr, build 0.1.11) on November 11, 2026, at 7:38 PM CT (America/Chicago). It used Lighthouse 13.6.0, on mobile. Five public pages came back fine. One made-up domain was included on purpose so we could see the error row. That failed URL was not charged. At the live price, the five successful audits would cost about 1.5 cents.

Here is the python.org row, shortened to the fields you would actually put in a spreadsheet:

{
  "url": "https://www.python.org/",
  "strategy": "mobile",
  "finalUrl": "https://www.python.org/",
  "lighthouseVersion": "13.6.0",
  "scores": { "performance": 90 },
  "lab": {
    "lcpMs": 3301,
    "cls": 0.002625429900180301,
    "tbtMs": 0,
    "fcpMs": 2109,
    "speedIndexMs": 2109
  },
  "fieldData": {
    "overallCategory": "FAST",
    "lcpMs": { "p75": 1319, "category": "FAST" },
    "inpMs": { "p75": 89, "category": "FAST" },
    "cls": { "p75": 0, "category": "FAST" }
  },
  "opportunities": [
    {
      "title": "Render-blocking requests",
      "estimatedSavingsMs": 1600
    }
  ],
  "status": "ok"
}

The same run, as a CSV would look after flattening the field p75 values:

url,status,performance,field_lcp_ms,field_inp_ms,field_cls
https://www.python.org/,ok,90,1319,89,0
https://developer.mozilla.org/,ok,100,1979,164,0.04
https://www.gov.uk/,ok,99,707,107,0
https://www.wikipedia.org/,ok,100,1301,100,0
https://web.dev/,ok,46,3047,235,0.01

Two details are worth sitting with. First, python.org's lab LCP was 3,301 ms, while real-user LCP at the 75th percentile was 1,319 ms. A lab test is one simulated phone load. Field data is what people actually saw. They will not always agree, and that is why you export both. Second, web.dev scored 46 for performance in the lab, and its field data came back SLOW, with a p75 LCP of 3,047 ms. A well known site can still miss the mark on a given day, which is a good reason to work from a table instead of from memory.

The sixth row in that run was https://hcb-nonexistent-domain-zz9q.com/, with status set to error and errorCode set to LIGHTHOUSE_FAILED. The job finished, the bad URL did not wipe the other five rows, and it did not add to the bill.

A few extras if you want to go further

Runnable Python and Node.js scripts, including this PageSpeed flow and a sitemap-to-PageSpeed chain, are in our apify-seo-examples repo on GitHub. The sitemap example pulls URLs from a public sitemap, then sends them to the checker, so you can find the slow pages on a whole site without pasting them by hand.

If you only need one public page, and you do not want an account at all, there is a free site check on our site. It asks PageSpeed Insights to test the page on a phone and on a desktop, then shows the scores and what to fix first.