DEV Community: Jeffrey Turov

Your scraper forgets everything between runs. Here's a review monitor that doesn't.

Jeffrey Turov — Tue, 01 Sep 2026 17:55:54 +0000

Your scraper forgets everything between runs. Here's a review monitor that doesn't.

A scraper that pulls Google Maps reviews once is a toy. What businesses actually pay for is a monitor: "tell me the moment a bad review lands." The difference between the two is one unglamorous feature — remembering what you've already seen.

I built Review Radar, an Apify Actor that watches a list of Google Maps places, scrapes recent reviews on a schedule, and posts a Slack alert the moment a new review at or below your star threshold appears. The Slack side uses Apify's MCP connectors, so the Actor never touches a token. Everything below comes from real runs — including the two state bugs that almost shipped.

What we're building

Review Radar:

takes a list of Google Maps place URLs (or a search query),
scrapes the latest reviews of each place with Playwright,
diffs them against a persistent store of already-seen review IDs,
pushes only genuinely new reviews to the dataset, flagged isNew: true, and
posts a Slack alert for each new review at or below your threshold — via an MCP connector, so no Slack token ever enters the Actor's code.

Why Google Maps reviews? Because for hotels, restaurants, and local agencies, a 1-star review answered within an hour is recoverable; the same review discovered three weeks later is a lost customer. Review monitoring is a product businesses already pay monthly for — and the official Google Business API only covers businesses you own, not your competitors or your clients' portfolios.

The three parts that actually matter

Scraping the reviews panel

On a Maps place page, reviews live behind the "Avis"/"Reviews" tab. The extraction selectors that survived contact with production:

Review blocks: div[data-review-id] — with a trap I detail below
Author: div.d4r55
Rating: span.kvMYJc[role="img"] — parse the number from the aria-label ("4 étoiles" / "4 stars")
Text: span.wiI7pd
Relative date: span.rsqaWe

If no review block is visible on load, click the tab first: button[aria-label*="Avis"] or button[aria-label*="Reviews"]. Then scroll the panel (div.m6QErb.DxyBCb.kA9KIf.dS8AEf) until you have enough blocks. Google Maps requires a residential proxy — datacenter IPs get consent-walled or blocked outright.

State: the difference between a scraper and a monitor

The Actor keeps a named key-value store (review-radar-state) with one record per place: the set of review IDs already seen. Each run:

hashes each review to a stable ID (the native data-review-id when present, otherwise a SHA-1 of author+date+text),
flags isNew: true only for IDs not in the store,
persists the updated set at the end.

Proof from two consecutive real runs on the same restaurant. First run:

Cindy P. | 5★ | isNew=True
Raquel G. Urbano | 4★ | isNew=True
Chri Cou1967 | 5★ | isNew=True

Second run, minutes later, identical reviews:

Cindy P. | 5★ | isNew=False
Raquel G. Urbano | 4★ | isNew=False
Chri Cou1967 | 5★ | isNew=False

That False is the entire product. A scheduled run now only surfaces what changed.

Slack alerts without a token in sight

Same connector model as my previous build: the user authorizes Slack once in Apify Console → Settings → Integrations. The Actor declares the connector in its input schema and receives a connector ID at runtime — never a token:

"slackConnector": {
  "title": "Slack connector (optional)",
  "type": "string",
  "resourceType": "mcpConnector",
  "mcpServers": [
    { "url": "*", "tools": { "required": ["*message*", "*chat*", "*post*", "*send*"], "readOnly": false } }
  ]
}

The mcpServers declaration does double duty: it filters which connectors the picker offers, and the proxy refuses any tool call outside that list. At runtime the Actor lists the connector's actual tools and picks the first one matching post/chat/message/send, then formats the alert:

🚨 Nouvel avis 2★ sur *La Maison Lefèvre* (J. Dupont, il y a 2 jours)
Service décevant, attente de 40 minutes...
https://google.com/maps/place/...

Making the connector input optional ("required": []) was deliberate: without it, the Actor still produces the full dataset — which also makes the Actor testable without touching your Slack workspace.

The two bugs that almost shipped

Bug 1 — key-value store key charset. I keyed place records by the Maps feature ID (0x45d3f1...:0x...). Apify record keys only allow a-zA-Z0-9!-_.'() — the colon is illegal. The Actor scraped everything perfectly, then crashed on the very last line of the run. One regex fixed it, but it's exactly the class of bug that only appears after a full successful scrape:

place_key = re.sub(r"[^a-zA-Z0-9!\-_.'()]", "_", place_key)

Bug 2 — duplicated review blocks. Querying both div[data-review-id] and the legacy div.jftiEf.fontBodyMedium selector returns overlapping containers — the same review twice. My first test dataset had Cindy P. duplicated. Fix: dedupe by review ID inside the scrape loop, before anything reaches the dataset.

What I'd do differently at scale

Relative dates ("il y a 2 semaines") don't sort. For precise alerting windows, resolve them against the run date.
The Slack tool argument names vary by connector (text, message, content). I inspect the tool's input schema and map fields — brittle but workable until connector schemas stabilize.
Photos in reviews aren't extracted yet — for hospitality clients, a photo of a dirty room matters more than the text.

Try it

The Actor is review-radar on my Apify account. Inputs: place URLs (or a search query), reviews per place, star threshold, optional Slack connector. First run reports the existing batch as new; every run after that reports only what changed. Point a daily schedule at it and you have a review monitoring product for the cost of a few compute units.

A note on terms: Google Maps scraping sits uneasily with Google's ToS. Keep volumes polite (a handful of places, daily cadence), use a residential proxy, weigh the risk for production use, and prefer official sources where they cover your need — the Business Profile API works fine for businesses you own, it just can't watch anyone else's.

From scraper to stack: pushing Google Maps leads straight into GitHub with MCP connectors

Jeffrey Turov — Mon, 31 Aug 2026 23:39:38 +0000

From scraper to stack: pushing Google Maps leads straight into GitHub with MCP connectors
Every lead-generation pipeline I build ends the same way: a dataset full of freshly scraped businesses… that I then have to export, transform, and commit somewhere by hand. The scrape is automated; the last mile never is.
When Apify launched MCP connectors - a new kind of Actor input that lets Actors securely call third-party services like GitHub, Notion, or Slack during a run - I rebuilt that last mile. This article walks through a working Actor that scrapes businesses from Google Maps and commits the results straight into a GitHub repository, without the Actor code ever seeing a credential.
Everything below comes from a real build: the pitfalls are ones I actually hit, and the fixes are what actually solved them.

Full source code: Codeberg repository (canonical mirror, readable without an account). Also available as a source ZIP. The GitHub original at github.com/jeffreyturov-dev/apify-scraping-toolbox is currently unreachable for logged-out viewers (account-level restriction, under review with GitHub Trust & Safety) - use the mirror.
What we're building
Maps to Stack: an Actor that

takes a Google Maps search query (e.g. restaurants Esch-sur-Alzette),
scrapes each place page with Playwright (name, address, phone, website, rating, GPS),
pushes the results to the dataset, and
writes a JSON snapshot into your GitHub repo via an MCP connector - one commit per run, versioned by Git. Why GitHub as the destination? Because for lead-gen pipelines, a repo is a free CRM: versioned, diffable, and already wired into everything else (CI, dashboards, git-based CMSs). The same pattern works identically for Notion or Slack - only the connector changes. Prerequisites
An Apify account (the free plan works for testing this)
A GitHub account with a repository to write into, authorized as an MCP connector in Apify Console → Settings → Integrations (one OAuth flow, two clicks)
A residential proxy for the Google Maps scraping half - Maps blocks datacenter IPs outright. A note on terms: routing around Google's consent wall and IP blocks sits uneasily with Google Maps' Terms of Service. For production use you should weigh that risk, keep request volumes polite, and consider official sources such as the Places API where they cover your need; the residential proxy here is what makes the unoffical path technically reliable, not legally bulletproof.
Five minutes to read the Actor source top to bottom; it's intentionally small The security model that makes this interesting The classic way to do this is to pass a GitHub token as an Actor input. That means every run log, every shared input JSON, and every fork of your Actor is one leak away from a compromised token. MCP connectors invert this:
You authorize GitHub once in Apify Console → Settings → Integrations. The credential lives with the connector, not with your code.
At run time, the Actor receives a connector ID (a string like ebw4ThD4cQbEKzC2l) - not a token.
The Actor talks to the Apify MCP proxy at ${ACTOR_MCP_CONNECTOR_BASE_URL}/, authenticating with the run's own APIFY_TOKEN.
The proxy enforces the tool permissions your Actor declared in its input schema. The Actor physically cannot call tools outside its declaration. Declaring the connector input In .actor/INPUT_SCHEMA.json, set resourceType: "mcpConnector". The mcpServers rule list both filters which connectors the picker offers and caps which tools the proxy will let the Actor call: "githubConnector": { "title": "GitHub connector", "description": "MCP connector to your GitHub account. The Actor only sees a connector ID.", "type": "string", "resourceType": "mcpConnector", "mcpServers": [ { "url": "", "tools": { "required": ["create_", "push_", "update_", "write_", "commit_", "get_*"], "readOnly": false } } ] }

At run time the input value is just the connector ID string.
Verified in the run log: the proxy filtered the connector's tool list down to exactly what matched my declaration. My run saw 7 tools - create_branch, create_or_update_file, create_pull_request, create_repository, push_files, update_pull_request, update_pull_request_branch - and nothing else. The constraint layer works.
Connecting from the Actor (Python)
Two environment variables are injected into every run: ACTOR_MCP_CONNECTOR_BASE_URL (the proxy) and APIFY_TOKEN (the starter's token). You use the standard MCP Python SDK - nothing Apify-specific:
import os
import httpx
from mcp import ClientSession
from mcp.client.streamable_http import streamable_http_client

base_url = os.environ["ACTOR_MCP_CONNECTOR_BASE_URL"]
token = os.environ["APIFY_TOKEN"]
proxy_url = f"{base_url}/{connector_id}"

async with httpx.AsyncClient(
headers={"Authorization": f"Bearer {token}"}
) as http:
async with streamable_http_client(proxy_url, http_client=http) as streams:
read, write = streams[0], streams[1]
async with ClientSession(read, write) as session:
await session.initialize()
tools = (await session.list_tools()).tools

(Import verified against the installed SDK: mcp.client.streamable_http exposes both streamable_http_client and the alias streamablehttp_client, so the line above runs as pasted; the docs showcase the alias, which is worth knowing given the drift discussed below.)
Pitfall #1 - SDK drift. The docs unpack streamable_http_client into three values (read, write, _), but the mcp version my image installed yields two. ValueError: not enough values to unpack (expected 3, got 2). Indexing the tuple (streams[0], streams[1]) works across versions. Same story for the result object: the error flag is result.is_error (snake_case), not isError - the docs' TypeScript casing leaks into expectations.
Writing the file - and surviving reality
With the session up, pick the file-writing tool from whatever the connector actually exposes (don't hard-code - the tool list is filtered by your schema and by what the server advertised at authorization time):
patterns = ["create_or_update_file", "push_files", "create_file", "update_file", "write_file"]
tool = next((t for p in patterns for t in tools if p in t.name), None)

owner, _, repo_name = actor_input["repo"].partition("/")
args = {
"owner": owner,
"repo": repo_name,
"path": actor_input.get("path", "leads/output.json"),
"content": payload_json,
"message": f"leads: {query} ({date.today()})",
"branch": actor_input.get("branch", "main"),
}
result = await session.call_tool(tool.name, arguments=args)

Two failures hit me immediately in real runs - both easy to fix once you see them:
Pitfall #2 - Branch main not found. My repo's default branch is master, not main. The MCP tool doesn't fall back to the repo's default; it fails loudly. Cheap robust fix: detect the failure and retry once with the other common branch name.
Pitfall #3 - File already exists… provide the current file's SHA. The second run collided with the first run's file. The GitHub API wants the blob SHA for updates - but the connector I authorized only exposes write tools (no get_file_contents to fetch the SHA). Instead of fighting it, I leaned into a better pattern for pipelines: immutable, timestamped snapshots. Every run writes a new dated file; Git itself becomes the history. If a read tool is available, the code uses the SHA path instead:
text = "".join(getattr(c, "text", "") for c in (result.content or []))
if "already exists" in text.lower():
get_tool = next((t for t in tools if "get_file_contents" in t.name), None)
if get_tool:
... # fetch sha, retry with args["sha"] = sha
else:
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
args["path"] = re.sub(r".json$", f"-{stamp}.json", args["path"])
result = await session.call_tool(tool.name, arguments=args)

Worth knowing: the connector's tool set is fixed at authorization time ("Layer 1" in the docs). If you authorize a connector and later wish it exposed more tools, re-authorize it - the discovered set doesn't refresh on its own.
The scraping side, briefly
The Maps half of the Actor uses Crawlee's PlaywrightCrawler with a residential proxy (Google blocks datacenter IPs outright). Three details that make it reliable, all learned the hard way on earlier Actors:

Kill the consent wall first. consent.google.com intercepts the first navigation; click "Accept" before waiting for any results selector.
Block heavy resources in a pre-navigation hook. The load event never fires on Maps (continuous analytics), so route-abort images/fonts/media, or every navigation times out.
Don't trust a green run. A SUCCEEDED run can still hold zero useful items - always assert on dataset item count and log per-item progress.
The first two are a few lines each, and they are the difference between an Actor that works and one that silently dies on every third navigation:
@crawler.pre_navigation_hook
async def optimize(context: PlaywrightCrawlingContext):
page = context.page
page.set_default_navigation_timeout(120_000)

async def _abort(route):
await route.abort()

The load event never fires on Maps (continuous analytics).

Without this, every navigation hits the navigation timeout.

await page.route(
"*/.{png,jpg,jpeg,gif,webp,svg,ico,woff,woff2,ttf,mp4,webm,avi}",
_abort,
)

@crawler.router.default_handler
async def search_handler(context: PlaywrightCrawlingContext):
page = context.page

# consent.google.com intercepts the first navigation.
# Click "Accept" before waiting for any results selector.
if "consent.google" in page.url:
    for sel in ('button[aria-label*="Accept"]',
                'button[aria-label*="Tout accepter"]',
                'button[aria-label*="Accepter"]',
                'form[action*="consent"] button'):
        btn = await page.query_selector(sel)
        if btn:
            await btn.click()
            await page.wait_for_timeout(2000)
            break

await page.wait_for_selector('div[role="feed"]', timeout=15000)

The full scraper is ~150 lines; the dataset holds one item per business with name, category, address, phone, website, rating, coordinates, and the Maps URL.
What a real run looks like
Input:
{
"query": "restaurants Esch-sur-Alzette",
"maxResults": 3,
"githubConnector": "ebw4ThD4cQbEKzC2l",
"repo": "jeffreyturov-dev/apify-scraping-toolbox",
"path": "leads/esch-restaurants.json"
}

Run log (abridged):
Scraped 3 businesses
Connector exposes 7 tools: ['create_branch', 'create_or_update_file', ...]
Using tool: create_or_update_file
Branch 'main' not found - retrying 'master'
Timestamped snapshot: leads/esch-restaurants-20260903T220554Z.json
Done - create_or_update_file wrote 3 leads

And the commit lands in GitHub with the full JSON: 3 businesses, ratings, phone numbers, coordinates - queryable, diffable, and already where the rest of my tooling lives.

Where this pattern earns its keep

Lead-gen pipelines: scrape on a schedule, each run commits a dated snapshot; a git log is your change history of who opened/closed in a neighborhood.
Any-to-any delivery: swap the connector for Notion (a database row per business) or Slack (a summary message) - the Actor code changes by a dozen lines; the security model stays identical.
Untrusted Actor code: if you consume third-party Actors, connectors are the only sane way to give them access to your services - the schema's tool constraints and the proxy's enforcement mean a malicious or sloppy Actor can't exceed its brief. Why the connector model is the only sane way to share access That last point deserves its own paragraph, because it generalizes beyond Apify. Every integration platform faces the same dilemma: users want automations that touch their GitHub, their CRM, their Slack - but handing a raw token to third-party code is an unacceptable blast radius. The connector pattern resolves it with three properties that are hard to get simultaneously any other way. First, mediation: every call passes through a proxy that authenticates the caller and authorizes the tool, so the credential is never exposed to the code that uses it. Second, declared least privilege: the input schema caps what the Actor may do, and the cap is enforced outside the Actor, where the Actor cannot rewrite it. Third, auditability: because calls flow through one chokepoint, every tool invocation is attributable to a specific run. Tokens in code give you none of these; connectors give you all three for the price of one OAuth flow. If you build or consume automations that touch shared services, this is the shape the access layer should have. Try it The Actor source (schema, scraper, connector logic) is intentionally small - read it top to bottom in five minutes. The moving parts that matter:
resourceType: "mcpConnector" in the input schema, with mcpServers tool constraints.
${ACTOR_MCP_CONNECTOR_BASE_URL}/ + APIFY_TOKEN bearer, via the stock MCP SDK.
Defensive write logic: branch fallback, SHA-if-available, timestamped snapshot otherwise. The connector model removes the part of pipeline building I liked least - sprinkling credentials through code - and replaces it with one authorization, one ID, and a proxy that keeps everyone honest.

Useful links: the Apify input schema reference, the pay-per-event monetization docs, the Apify MCP integration docs, and the Apify MCP server page.

I built 8 pay-per-use scraping APIs that AI agents can call directly (Google Maps, TikTok, Instagram, YouTube, LinkedIn) — here's what I learned

Jeffrey Turov — Tue, 28 Jul 2026 09:28:47 +0000

A few weeks ago I published a set of Actors on Apify Store. Today they're all AI-agent ready: any LLM agent (Claude, GPT, Cursor, LangChain, n8n) can discover and call them through the Apify MCP server — no custom integration code needed.

This post is the full playbook: what the tools do, how the pay-per-event monetization works, the bugs I hit, and how AI agents actually consume them.

The toolbox

Actor	What it extracts	Price (pay-per-event)
Google Maps Business Scraper	Names, phones, websites, ratings, reviews, GPS	$0.005 / business
TikTok Profile & Video Scraper	Followers, likes, bio, per-video stats	$0.01 / profile
Instagram Profile Scraper	Followers, bio, verified, engagement	$0.01 / profile
YouTube Video & Channel Scraper	Views, likes, subscribers, search results	$0.002 / video
LinkedIn Profile Scraper	Headlines, companies, skills, experience	$0.02 / profile
RAG Web Browser	Clean Markdown from any URL + Google search	$0.003 / page
Fuel Prices France API	Real-time prices, 9,800 stations, GPS	FREE
Hotel Rate Monitoring	Competitor rates, parity checks	FREE

The two free ones are deliberate lead magnets — more on that below.

Why "AI-agent ready" changes everything

The old model: a human finds your scraper on the store, reads the docs, clicks buttons.

The new model: an AI agent gets a task ("find me 50 plumbers in Austin with their phone numbers"), searches the Apify Store via MCP, reads the actor's README and input schema, and calls it — end to end, no human.

For that to work, three things must be true:

Your README is written for an LLM, not just humans. Mine now all start with a "Use this tool when..." section — that's what the agent pattern-matches against the user's request.
Your input schema has a description on every field. The agent constructs the JSON input from those descriptions. No description = hallucinated parameters = failed runs = no revenue.
Your output is documented field by field. The agent needs to know what it gets back to reason over it.

Here's the actual flow with the Apify MCP server (https://mcp.apify.com — add it to Claude Desktop or Cursor in 30 seconds):

User: "Get me the follower counts of these 5 TikTok creators"
Agent: → search-actors("tiktok profile")
       → fetch-actor-details (reads README + input schema)
       → call-actor(travelmonitorlab/tiktok-scraper,
                    {"profiles": [...], "maxVideosPerProfile": 0})
       → returns structured JSON

Or skip MCP entirely — every actor is a single synchronous HTTP call:

curl -X POST "https://api.apify.com/v2/acts/travelmonitorlab~google-maps-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["plumbers Austin TX"], "maxResults": 50}'

Monetization: pay-per-event (and the trap that cost me hours)

Apify offers several pricing models. For new actors, PRICE_PER_DATASET_ITEM is rejected — you must use PAY_PER_EVENT. The model is better anyway: you define events (e.g. business-scraped at $0.005) and charge explicitly in code:

await Actor.charge(event_name="business-scraped")
await Actor.push_data(item)

The trap: put the charge AFTER crawler.run() and your event loop may already be closed — the charge silently vanishes, you deliver data for free. Charge inside the handler, right before pushing data. Always verify with chargedEventCounts in the run object after a test run.

Setting pricing is pure API:

PUT /v2/acts/{actorId}
{"pricingInfos": [{
  "pricingModel": "PAY_PER_EVENT",
  "reasonForChange": "Launch pricing",
  "pricingPerEvent": {"actorChargeEvents": {
    "business-scraped": {
      "eventTitle": "Business scraped",
      "eventDescription": "One Google Maps business record",
      "eventPriceUsd": 0.005
    }}}}
]}

Apify takes 20%. Compute is paid by the user; you pocket the event fees.

Battle scars (so you don't get them)

Crawlee 1.8 breaking changes: purge_on_start and navigation_timeout_secs are no longer valid kwargs — use page.set_default_navigation_timeout() in a pre_navigation_hook.
Google Maps never fires load: analytics keep streaming forever, so navigation always times out. Fix: abort images/fonts/media via page.route() (the handler must be a coroutine, not a lambda).
Residential proxies are mandatory for Google Maps, and you must pass actor_proxy_input= as a named argument to Actor.create_proxy_configuration().
French number formats will crash your floats: "4,8" → replace comma; "1 234" reviews can use \xa0 or \u202f as thousand separator.
A "SUCCEEDED" run can contain zero useful data. Always check itemCount + sample the dataset + read the end of the log.
Don't run 7 queries × 25 results in one run. Split into parallel runs of ≤5 queries × 15 results; retry failed ones sequentially (residential proxy tunnels occasionally die).

The distribution strategy

Publishing on the store is step 0. What actually moves the needle:

Free lead magnets — the fuel-price and hotel-rate actors are 100% free. Free tools get users, ratings, and store ranking; ranked actors surface in MCP search results; MCP visibility drives paying users to the paid actors.
README written for LLMs — agents choose tools whose docs they can parse. Clear "use when", typed inputs, example I/O.
Niche SEO titles — "Google Maps Scraper" is saturated; "Fuel Prices France API" has zero competition on the store.
Dogfooding — I use my own Google Maps actor to build lead lists I sell elsewhere. Every sale is also a demo.

Try them

All 8 actors are live on Apify Store. If you build agents, add https://mcp.apify.com to your MCP client and just ask for the data — the agent will find the tools.

Feedback, bugs, feature requests: open an issue on any actor page, I answer fast.

DEV Community: Jeffrey Turov

Your scraper forgets everything between runs. Here's a review monitor that doesn't.

From scraper to stack: pushing Google Maps leads straight into GitHub with MCP connectors

The load event never fires on Maps (continuous analytics).

Without this, every navigation hits the navigation timeout.

I built 8 pay-per-use scraping APIs that AI agents can call directly (Google Maps, TikTok, Instagram, YouTube, LinkedIn) — here's what I learned

The toolbox

Why "AI-agent ready" changes everything

Monetization: pay-per-event (and the trap that cost me hours)

Battle scars (so you don't get them)

The distribution strategy

Try them