DEV Community

Cover image for How to benchmark a SERP API: six axes and a cost model that survives n=100
Ricardo Batista
Ricardo Batista

Posted on Originally published at cloro.dev

How to benchmark a SERP API: six axes and a cost model that survives n=100

Key takeaways

  • Score six axes separately and never average them. A provider can be fast and shallow, or excellent on head terms and blind on local packs.
  • Compute cost at your own depth. DataForSEO is half cloro's price at n=10 and level with it at n=100 with AI Overview, and SerpApi's gap widens tenfold.
  • Score a missing block and an absent block differently, or your trend lines encode the parser's blind spots as real changes in Google.

Most SERP API comparisons publish one number: a price per 1,000 calls, or a median latency off a handful of queries. Both are true and both describe the easy case.

A SERP API is a parser sitting on scraping infrastructure. It can be fast and shallow. It can be cheap at ten results and ruinous at a hundred. It can be excellent on commercial head terms and blind on local packs. Average those into one score and you have hidden every failure mode that will actually bite you.

Run it in this order:

  1. Build the query set. Fifty queries, evenly split across the four classes. Freeze the list, because you will re-run it against every provider and again in three months.
  2. Fix the request shape. Same country, language, depth and AIO flag everywhere. Any provider-specific parameter is a variable you can no longer attribute.
  3. Capture ground truth by hand for ten of them, so a parser miss is distinguishable from an absent block.
  4. Score the six axes as six columns. Do not average.
  5. Compute cost at your depth using each vendor's own unit, at both n=10 and the depth you actually need.
  6. Measure latency under concurrent load, reporting p50 and p95. A sequential median tells you nothing about queue behaviour.
  7. Ask the two questions the harness cannot answer: how schema changes are versioned, and where the data comes from.

Six axes, scored separately

Never collapse them into a single number. What you weight depends on your workload: a rank tracker running nightly batches cares about cost at depth, an agent doing grounding cares about latency, a GEO platform cares almost entirely about AI Overview field fidelity.

Axis What you are actually testing
AI Overview fidelity Not "does it return AIO" but in what shape. Cited sources with URLs and positions, a text-only summary, or raw HTML you parse yourself. Only the first survives a Google DOM change without work on your side
Multi-engine coverage Whether each surface is a real endpoint or a wrapper around the same Google call. Coverage claims are cheap
Envelope completeness Ads, People Also Ask, related searches, knowledge panel, shopping. Ten of the twelve we tested return the full envelope, so this rarely separates the field. It does catch the ones quietly returning organic only
Geolocation precision Country, city, or UULE-precise. This is where a local SEO tracker gets burned late, because a national average and a metro result diverge on exactly the queries a client checks by hand
True cost per 1,000 At your depth and feature set, not the advertised base rate. Its own section below
Latency p50 and p95 The p95 is the one that matters. Dispatch faster than the queue clears and the slowest 5% balloon past the median. A 2-second median hides a 15-second tail

Fifty queries, four classes

Same request shape through every provider. Commercial head terms, which stress the envelope. AI-Overview-eligible informational queries, which separate real parsing from a text summary. Local and maps-heavy, which stress geolocation. Long tail, where thin SERPs show up as ambiguous nulls rather than errors.

The classes matter more than the count. A provider that looks identical to its competitors on head terms can be missing a third of AI Overviews on informational ones, and an aggregate success rate will never show you that.

One thing worth stealing: score a missing block and an absent block differently. When a provider returns nothing for an AI Overview, you cannot tell whether the SERP had no Overview or the parser missed it. Capture ground truth by hand for ten queries so you can tell the two apart. Skip that and your trend lines encode the parser's blind spots as real changes in Google.

The cost model, where most evaluations go wrong

Three things decide your bill, and the headline rate is none of them.

Depth. Google removed &num=100 on September 11, 2025, so everyone paginates at 10 results a page. Locomotive Agency put the infrastructure cost increase at roughly 10x at top-100 depth. If you need 100 results you are buying ten pages, and how a vendor bills those pages is now the dominant term.

AI Overview enrichment. Usually a surcharge, not an included field. DataForSEO gates it behind load_async_ai_overview at $0.0006 per keyword, roughly doubling the first-page price. Others include it. Credit-pool vendors charge extra credits.

Billing unit. Per-call, per-successful-result, and per-credit are not comparable until you do the arithmetic at your own depth. A credit pool turns a fixed cost into a variable one. Per-successful-result billing turns your failure rate into a price.

Provider Unit n=10 n=100 + AIO
cloro (Hobby, $0.40/1k credits) 3 credits at n=10, 23 at n=100 with AIO $1.20/1k $9.20/1k
DataForSEO (Standard queue) per call, +$2/1k for AIO on page one $0.60/1k $9.20/1k
SerpApi (Big Data tier) per search, one search = one 10-result page (pricing) $9.17/1k $91.70/1k

The ranking inverts between those two columns. DataForSEO is half cloro's price at n=10 and level with it at n=100 with AIO. SerpApi's gap widens by an order of magnitude, because each batch of ten bills as its own search. Run this before you shortlist. The provider that wins your evaluation at n=10 can be the most expensive thing you run in production, and the whole category is priced this way here.

Two things the harness cannot tell you

Whether the schema will hold. A benchmark captures one moment. It says nothing about whether the vendor renames a field next quarter and breaks your parsers. You cannot measure this from outside, so ask: how are schema changes versioned, and what is the deprecation window? No answer means a maintenance cost you have not priced.

Where the data comes from. Google sued SerpApi on December 19, 2025, alleging circumvention of anti-bot protections under DMCA 1201 (announcement, Search Engine Land has the detail). SerpApi moved to dismiss in February 2026, arguing it returns only what a signed-out user sees. Unresolved as of August 2026. No court has ruled that scraping public results is unlawful and the suit targets circumvention rather than scraping, but provenance is still a procurement question: official feed, scraped results pages, or the vendor's own index. Where the legal line actually sits is worth reading before you pick on this axis.

Both official routes are closing anyway. Bing's search APIs retired in August 2025, and Google's Custom Search JSON API is closed to new customers with a January 1, 2027 shutdown per Google's own docs. What is actually being retired and what is not is worth checking, because most write-ups get it wrong.

If you want the filled-in version, 12 providers scored on all six axes with the cost table at four depths, it is in Best SERP APIs 2026. Check vendor pricing pages yourself before committing. The category has repriced twice since num=100 and will again.

Top comments (1)

Collapse
 
alexshev profile image
Alex Shev

A SERP API benchmark needs more than accuracy. Freshness, location fidelity, retry behavior, result normalization, cost at scale, and how failures are reported all matter if the data drives product decisions.