Originally published on proxymint.
How to scrape Google search results in 2026: the Custom Search JSON API and its limits, what Google's terms say, parsing, pacing and setting location.
TL;DR
- Google has one official results API, and it is closing. The Custom Search JSON API is closed to new customers and is discontinued on January 1, 2027, per Google's overview page updated February 18, 2026.
- The API caps depth at 100 results per query. Each call returns at most 10 results, so top-100 depth for one keyword takes 10 billable calls.
- Google's rules prohibit automated queries without permission. The spam policies name rank-checking scrapes as machine-generated traffic, the Terms of Service bar automated access against robots.txt, and robots.txt disallows /search.
- Own-site ranking data needs no scraping. The Search Console API returns query, page, country and device rows, up to 25,000 per request, with Google's permission.
Key takeaways
- Start with the official route. If a Custom Search key exists, use it until January 1, 2027, and budget 100 free queries a day, then billing per 1,000 queries up to 10,000 a day.
- Answer own-site questions from Search Console. Average position by query, page, country and device is free and inside Google's rules.
- Parse by structure and test on fixtures. A heading inside a link survives layout changes that break hardcoded class names, and a saved fixture makes the break visible.
- Treat a captcha or /sorry/ page as a stop. Record the query as no data, back off, and never retry through a new exit to get past it.
- Pick the route by permission, not by tooling. No proxy tier changes what Google's terms allow; licensed data and engines that permit automated access are the routes that hold up.
Read the full guide on proxymint, with the commands, tables and sources.
Top comments (0)