DEV Community

coreclaw
coreclaw

Posted on

How to Scrape Google Maps Without Getting Blocked in 2026

How to Scrape Google Maps Without Getting Blocked in 2026

Google Maps is the most valuable public dataset that nobody wants you to access programmatically. Every day, businesses spend thousands of dollars on lead generation tools that are essentially thin wrappers around Google Maps scraping — and in 2026, getting this right is harder than ever.

I've spent the last six months building and breaking Maps scrapers. Here's what actually works, what's a waste of time, and why most tutorials you'll find are dated and dangerous.

Why Google Maps Scraping Is So Difficult Now

Google's anti-bot stack for Maps has evolved into a multi-layer defense system. It's not just reCAPTCHA anymore — it's a sequence of checks that start before your browser even renders a pixel.

Layer 1 — TLS Fingerprinting: Before any JavaScript runs, Google inspects your TLS handshake. Python's requests library, curl, and even headless Chromium have distinct JA3/JA4 fingerprints that Google matches against known automation patterns. If you fail here, you don't even get HTML back — just a 429 or a redirect loop.

Layer 2 — Browser Environment: Google runs JavaScript checks on navigator.webdriver, chrome.runtime, WebGL renderer strings, font enumeration, and canvas fingerprinting. Headless Chrome returns SwiftShader as the WebGL renderer — real Chrome returns your GPU model. That difference alone flags you.

Layer 3 — Behavioral Analysis: Mouse trajectories, scroll patterns, timing between actions. Real users don't click search results in exactly 2.3 seconds every time. Real users don't scroll with perfect linearity.

Layer 4 — Cross-Session Correlation: If the same "browser fingerprint" appears from 50 different IPs across 3 continents in 10 minutes, Google connects the dots.

The net result: standalone headless Chrome gets detected with 95%+ accuracy in 2026. Even with full stealth patches, raw success rates hover around 40-70% for sustained scraping.

The Tools That Actually Work (and the Ones That Don't)

What's Broken

Plain requests + BeautifulSoup: Dead on arrival. Google's TLS fingerprint check kills this before you get a single byte of Maps data. The requests library's TLS signature hasn't matched a real browser since 2023.

Puppeteer with default headless: Detected in under 10 requests. Google specifically looks for --headless mode artifacts.

Datacenter proxies: Instant blocks on Google Maps. Google maintains blocklists of known datacenter IP ranges and refreshes them aggressively.

What Works (With Effort)

Playwright + playwright-stealth + residential proxies: This is the current baseline. playwright-stealth patches about 15-19 detection vectors, including navigator.webdriver, chrome.runtime, and permissions API spoofing. Combined with residential proxies and proper rate limiting, this gives you a fighting chance.

Here's a production-ready setup:

from playwright.sync_api import sync_playwright
from playwright_stealth import stealth_sync
import time
import random

PROXIES = [
    "http://user:pass@residential-1.proxy:8000",
    "http://user:pass@residential-2.proxy:8000",
    "http://user:pass@residential-3.proxy:8000",
]

def scrape_google_maps(query: str, location: str, max_results: int = 50):
    results = []
    proxy_idx = 0
    requests_per_session = 0

    with sync_playwright() as p:
        browser = p.chromium.launch(
            headless=True,
            proxy={"server": PROXIES[proxy_idx]},
            args=[
                "--disable-blink-features=AutomationControlled",
                "--disable-dev-shm-usage",
            ]
        )
        context = browser.new_context(
            viewport={"width": 1366, "height": 768},
            user_agent=(
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/126.0.0.0 Safari/537.36"
            ),
        )
        page = context.new_page()
        stealth_sync(page)

        page.goto("https://www.google.com/maps", wait_until="domcontentloaded")
        time.sleep(random.uniform(3, 6))

        search_box = page.locator("#searchboxinput")
        search_box.fill(f"{query} in {location}")
        search_box.press("Enter")
        time.sleep(random.uniform(4, 8))

        feed = page.locator('div[role="feed"]')
        for _ in range(8):
            feed.evaluate("el => el.scrollTop = el.scrollHeight")
            time.sleep(random.uniform(2, 4))
            requests_per_session += 1

            # Rotate proxy every 20-30 requests
            if requests_per_session >= random.randint(20, 30):
                browser.close()
                proxy_idx = (proxy_idx + 1) % len(PROXIES)
                requests_per_session = 0
                # Re-launch browser with new proxy
                browser = p.chromium.launch(
                    headless=True,
                    proxy={"server": PROXIES[proxy_idx]},
                    args=["--disable-blink-features=AutomationControlled"],
                )
                context = browser.new_context(
                    viewport={"width": 1366, "height": 768},
                    user_agent=(
                        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
                        "AppleWebKit/537.36"
                    ),
                )
                page = context.new_page()
                stealth_sync(page)

        cards = page.locator('a[href*="/place/"]')
        for i in range(min(cards.count(), max_results)):
            title = cards.nth(i).inner_text().strip()
            url = cards.nth(i).get_attribute("href")
            results.append({"title": title, "url": url})

        browser.close()
    return results
Enter fullscreen mode Exit fullscreen mode

curl_cffi for API-level scraping: If you're willing to give up browser automation and work with Google's internal ProtoJSON format, curl_cffi with impersonate="chrome" perfectly mimics Chrome's TLS handshake. The tradeoff: you'll need to parse Protobuf-serialized JSON arrays that are prefixed with )]}'\n (Google's anti-JSON-hijacking guard), and review data comes back as cryptic hex tokens that need separate resolution.

from curl_cffi import requests

# This TLS handshake is indistinguishable from real Chrome
resp = requests.get(
    "https://www.google.com/maps/preview/place/...",
    impersonate="chrome",
    headers={
        "Accept": "application/json",
        "Accept-Language": "en-US,en;q=0.9",
    }
)
# Response format: )]}'\n followed by nested JSON arrays
data = resp.text[5:]  # strip the anti-hijack prefix
Enter fullscreen mode Exit fullscreen mode

The Smarter Approach: Don't Build It Yourself

Honestly, after maintaining a Maps scraper for six months, I've come to a conclusion that would have offended me two years ago: building your own Google Maps scraper is a waste of engineering time in 2026.

Between proxy costs ($10-15/GB for residential), CAPTCHA solving (~$2-3 per 1,000 challenges), infrastructure overhead ($50-200/month for browser farms), and the endless cat-and-mouse game with Google's detection updates, the cost per successful search lands around $0.01-$0.05. A decent SERP API costs $0.005 per query and just works.

If you need Google Maps data at scale, use a platform that already solved these problems. CoreClaw's Google Maps Scraper pulls business records in bulk — reviews, reviewer details, photos, contact info including full name, email, job title, opening hours, and prices — and you get structured CSV/JSON output without touching a single proxy configuration.

The same logic applies to their MCP server integration: if you're building an AI agent that needs Maps data, pointing it at a pre-built worker via MCP is faster and more reliable than teaching the agent to navigate Google's anti-bot stack.

Proxy Strategy: The Difference Between 40% and 95% Success Rate

The single biggest lever you can pull is your proxy setup. Here's what the data shows:

Configuration Success Rate Cost/GB
No proxy 70-80% (small scale only) Free
Datacenter proxies 80-90% (degrades fast) $1-3
Rotating residential 95%+ $8-15
Mobile proxies Highest (niche use) $20+

But raw success rate isn't the full picture. The real art is in the rotation strategy:

Don't rotate per request. Google notices when a "user" changes IP mid-session. Rotate per browser session — one proxy for 20-30 business detail pages, then switch.

Cool-down periods matter. When Google throws a CAPTCHA or a soft block, that IP is burned for at least 20-40 minutes. Continuing to hammer it just extends the cool-down.

Geo-match your proxies. Searching for "restaurants in Tokyo" from a US IP? That's a red flag. Your proxy should match the search region.

Mistake Consequence
Using datacenter proxies for Maps Blocked within minutes
Wrong country geo-location Wrong market data, higher detection rate
Rotating proxy mid-pagination Session breaks, inconsistent results
Default/blank User-Agent Immediate block

Handling Google's Result Limit: Grid Partitioning

Google Maps caps search results at around 120 places per query — and that's only at tight zoom levels. For broader searches, you get 30-50 results before hitting "You've reached the end of the list."

The workaround is grid partitioning:

  1. Divide your target area into smaller grid cells (about 1.5 km at zoom level 16)
  2. Run a separate search within each cell
  3. De-duplicate results that appear in overlapping cells

A city like San Francisco needs roughly 400 grid cells at zoom 16. At 120 results per cell and 400 cells, you're looking at 48,000 potential data points. But you're also looking at 400 separate search operations, each requiring its own proxy session and rate-limiting compliance.

The smarter approach used by platforms like CoreClaw is adaptive hexagon partitioning (inspired by Uber's H3 geospatial indexing): only subdivide cells that hit the result ceiling, skip cells under 20 results immediately, and eliminate about 90% of unnecessary API calls compared to uniform grids.

Code That Extracts Reviews (The Hard Part)

Reviews are the most valuable but trickiest data to extract. Google loads them dynamically — the initial HTML shows maybe 10-20 reviews, and you need to scroll to trigger loading more:

def extract_reviews(page, max_reviews=50):
    """Pull reviews from a Google Maps place detail page."""
    # Click into the reviews tab
    reviews_btn = page.locator('button[aria-label*="Reviews"]')
    if reviews_btn.count() > 0:
        reviews_btn.first.click()
        time.sleep(random.uniform(2, 4))

    # Scroll to load reviews
    panel = page.locator('div.m6QErb.DxyBCb')
    for _ in range(max_reviews // 5):
        if panel.count() > 0:
            panel.first.evaluate("el => el.scrollTop = el.scrollHeight")
            time.sleep(random.uniform(0.8, 2))

    reviews = []
    for el in page.locator('div.jftiEf').all()[:max_reviews]:
        name_el = el.locator('div.d4r55')
        rating_el = el.locator('span.kvMYJc')
        text_el = el.locator('span.wiI7pd')
        date_el = el.locator('span.rsqaWe')

        reviews.append({
            "reviewer": name_el.inner_text() if name_el.count() else None,
            "rating": rating_el.get_attribute("aria-label") if rating_el.count() else None,
            "text": text_el.inner_text() if text_el.count() else None,
            "date": date_el.inner_text() if date_el.count() else None,
        })
    return reviews
Enter fullscreen mode Exit fullscreen mode

What Breaks and How Fast

Here's my empirical failure timeline from running Maps scrapers in production:

Time Elapsed What Happens
0-5 min Normal operation with residential proxy
5-15 min Intermittent CAPTCHAs on search requests
15-30 min Consistent CAPTCHAs, some empty results
30-60 min Hard rate limit (429 errors)
60+ min IP-level block on the Maps endpoint

Recovery: soft throttles clear in 1-4 hours, hard rate limits in 24-48 hours, and IP blocks can last a week.

The Bottom Line

Google Maps scraping in 2026 is a classic build-vs-buy decision, and the economics have shifted decisively toward buying. Between the TLS fingerprinting arms race, the multi-layer behavioral detection, the proxy costs, and the ongoing maintenance burden of updating CSS selectors every time Google tweaks their frontend, the engineering time you'll spend maintaining a scraper will exceed the cost of using a platform that already solved these problems.

If you need one-off data, grab a pre-built worker. If you need ongoing monitoring, use a platform with scheduling and API access. If you're building this into a product, integrate via MCP or REST API. Save your engineering hours for the thing that actually differentiates your business — not for reverse-engineering Google's anti-bot stack.

Top comments (0)