DEV Community: Chaz Eden

[Boost]

Chaz Eden — Tue, 14 Jul 2026 00:11:33 +0000

Chaz Eden

Jul 14

I built a tool that checks whether ChatGPT recommends your brand (Python + Apify)

#ai #python #seo #webscraping

5 min read

I built a tool that checks whether ChatGPT recommends your brand (Python + Apify)

Chaz Eden — Tue, 14 Jul 2026 00:11:17 +0000

Your customers have stopped Googling "best note-taking app." They're asking ChatGPT, Perplexity, and Gemini instead — and getting back a short list of three or four products. If your brand isn't on that list, you're invisible, and unlike a Google ranking you can't even see where you stand.

That's the problem I set out to measure. This post is the build breakdown: five AI answer engines, one uniform result shape, a mention-detection core that doesn't lie to you, and the honest gotchas I hit around cost and billing. The whole thing runs as a paid Apify Actor written in async Python.

The niche has a name now — GEO (Generative Engine Optimization) or AEO (Answer Engine Optimization). Think SEO, but the search engine is a language model and the "ranking" is whether you get named in the answer.

The core question

Give the tool a brand, its competitors, and the buyer-intent questions your customers actually type:

{
  "brand": "Notion",
  "competitors": ["Obsidian", "Coda", "Evernote"],
  "prompts": ["best note taking app for students", "Notion vs Obsidian which should I use"],
  "engines": ["perplexity", "chatgpt", "gemini", "claude", "aiOverview"],
  "samplesPerPrompt": 3
}

It asks each engine each prompt (several times, because LLM answers vary run-to-run), then analyzes every answer for: were you mentioned, how early, were you recommended or just listed, what's the sentiment, who else got named, and — the part incumbents skip — which domains each engine cited. That last one is the actionable output: it tells you which websites the AI trusts for your category, i.e. where you need coverage.

Architecture: one shape to rule them all

The trick that keeps the whole thing sane is that every engine adapter — whether it's a clean REST API or a messy HTML scrape — returns the exact same record shape:

{
    "engine": "perplexity",
    "prompt": "best note taking app for students",
    "sampleIndex": 1,
    "responseText": "...",
    "citations": [{"url": "...", "domain": "zapier.com", "title": "..."}],
    "model": "sonar",
    "tokensUsed": 481,
    "costEstimate": 0.0055,
}

Once every adapter conforms to that, the analysis and aggregation layers never have to know which engine produced a record. Adding a sixth engine later is a self-contained file.

The adapters

Four of the five engines are official APIs with web search or grounding built in — never scrape a consumer chat UI (that's a ToS violation and a CAPTCHA arms race you'll lose):

Engine	How	Citations from
Perplexity	`sonar` model over HTTPS	native `search_results[]`
ChatGPT	OpenAI Responses API + `web_search` tool	`url_citation` annotations
Gemini	Gemini API + Google Search grounding	`grounding_metadata`
Claude	Anthropic API + web search tool	`web_search_tool_result` blocks
Google AI Overviews	scrape `google.com/search` via residential proxies	parsed source links

The Perplexity adapter is the whole pattern in miniature — build this one first, it's the cheapest and the API returns citations natively:

async def run(prompt, sample_index, ctx):
    api_key = require_env("PERPLEXITY_API_KEY")
    async with httpx.AsyncClient(timeout=90) as client:
        resp = await client.post(API_URL,
            headers={"Authorization": f"Bearer {api_key}"},
            json={"model": "sonar", "messages": [{"role": "user", "content": prompt}]})
    data = resp.json()

    record = base_record("perplexity", prompt, sample_index)
    record["responseText"] = data["choices"][0]["message"]["content"]
    for result in data.get("search_results") or []:
        record["citations"].append(make_citation(result["url"], result.get("title")))
    return record

Google AI Overviews is the one true scrape. Google's markup shifts every few weeks, so I isolated every selector and heuristic into a single file — when it breaks, there's exactly one place to fix. It also parses defensively: not every query triggers an AI Overview, and "no overview appeared" is recorded as data (aiOverviewPresent: false), not thrown as an error. That absence is something users genuinely want to know.

Fan-out with isolation and throttling

Every (engine × prompt × sample) combination is a coroutine, all fired concurrently with asyncio.gather:

tasks = [
    run_engine(engine, prompt, sample_index, ctx)
    for engine in engines
    for prompt in prompts
    for sample_index in range(1, samples + 1)
]
results = await asyncio.gather(*tasks)

Two things make firing 300 requests at once safe. First, a per-provider semaphore caps concurrency so I never hammer one API with more than ~5 in flight. Second, an isolation boundary: run_engine wraps each adapter so one engine's bad day can never kill the run — a failure becomes a record with an error field instead of an exception that takes down the other 299 queries.

async def run_engine(engine, prompt, sample_index, ctx):
    async with ctx.semaphore(engine):          # ~5 concurrent per provider
        try:
            return await registry[engine](prompt, sample_index, ctx)
        except Exception as exc:               # never let one query kill the run
            record = base_record(engine, prompt, sample_index)
            record["error"] = f"{type(exc).__name__}: {exc}"
            return record

The part that actually matters: mention detection

This is the credibility core. If the tool says "you were mentioned" when you weren't, nobody trusts a single number it produces. So detection lives in pure functions with unit tests, and it fights two failure modes.

Substring false positives. Naive "Coda" in text matches "Codash." Word-boundary regex fixes it:

def _pattern(name):
    return re.compile(r"(?<!\w)" + re.escape(name) + r"(?!\w)", re.IGNORECASE)

Common-word false positives. This is the sneaky one. Half the good SaaS names are ordinary English words — Notion, Coda, Monday, Slack, Arc, Square. "The notion of productivity is vague" is not a mention of Notion. So a lowercase occurrence of a name that's also a common word is treated as prose, while the capitalized brand ("Notion is great") counts:

def _is_false_positive(name, matched_text):
    return (
        name.lower() in COMMON_ENGLISH_WORDS
        and name[:1].isupper()
        and matched_text.islower()
    )

Mention position falls out of the same machinery: find the first genuine occurrence of the brand and of every competitor, then rank them. Position 1 means you were the first product named — the thing buyers actually remember.

brand_index = find_first_mention(text, brand, aliases)
earlier = sum(1 for idx in competitor_indexes.values() if idx < brand_index)
mention_position = earlier + 1

Sentiment and "recommended vs. merely listed" come from one cheap batched LLM call per response returning strict JSON — never a second full-price generation.

The scorecard

Aggregation rolls the per-query records into the number a marketing team reads:

{
  "brand": "Notion",
  "visibilityScore": 87.5,
  "visibilityByEngine": { "perplexity": 100.0, "chatgpt": 50.0, "gemini": 100.0, "claude": 100.0 },
  "shareOfVoice": { "Notion": 38.0, "Obsidian": 41.0, "Evernote": 21.0 },
  "topCitedDomains": [{ "domain": "zapier.com", "count": 6 }, { "domain": "reddit.com", "count": 4 }]
}

Per-engine visibility is just the mention rate for that engine; the blended visibilityScore is the mean across engines. The interesting insight is almost always in visibilityByEngine — "you're at 100% everywhere except ChatGPT, where you're at 50%" is a concrete, fixable finding.

Two gotchas worth your time

LLM web search is expensive and volatile. The token cost isn't the problem — the per-search fee and the size of the search-result context are. ChatGPT with web search pulled 10k–20k input tokens per query in my tests and occasionally fired multiple searches, swinging from ~$0.015 to ~$0.04 for a single query. Claude with an unbounded search did the same until I capped it to one search on the small model. If you build anything on top of web-search tools, meter cost per query per provider from day one — the variance will surprise you.

Billing sneaks up on you. On Apify's pay-per-event model, one synthetic event charges automatically for every item written to the default dataset. Push a failed query or a summary row there and you've billed your user for junk. The fix is a discipline: the default dataset gets only successfully analyzed responses; failures and the scorecard go to a separate output record that isn't billed. Whatever platform you're on, know exactly which write is the one that costs your user money.

Try it

Actor: AI Brand Visibility Monitor on Apify

If you're doing GEO/AEO work — tracking brand mentions in ChatGPT, measuring AI share of voice, figuring out which sources the models cite for your category — I'd love to hear how you're approaching it. What would you want a tool like this to measure?

Comparing the job-posting APIs of Workday, Greenhouse, Lever, Ashby, SmartRecruiters, and Recruitee (2026)

Chaz Eden — Mon, 13 Jul 2026 01:09:55 +0000

If you're building a job board, a hiring-signal tool, or a "who's hiring" newsletter, the good news is that you usually don't need to scrape HTML at all. The five biggest applicant tracking systems (ATSs) — Workday, Greenhouse, Lever, Ashby, SmartRecruiters, and Recruitee — all publish public, no-authentication JSON APIs for their job boards. Companies want these to be read; it's how their jobs get syndicated.

The bad news: all six APIs are completely different — different shapes, different pagination, different date formats, and some genuinely surprising quirks. I recently built an aggregator across all six and tested against dozens of live boards. Here's the field guide I wish I'd had.

Five endpoints at a glance

ATS	Endpoint	Auth	Pagination	Salary data
Greenhouse	`boards-api.greenhouse.io/v1/boards/{token}/jobs?content=true`	none	no (one response)	rarely
Lever	`api.lever.co/v0/postings/{company}?mode=json`	none	no	sometimes
Ashby	`api.ashbyhq.com/posting-api/job-board/{board}?includeCompensation=true`	none	no	usually
SmartRecruiters	`api.smartrecruiters.com/v1/companies/{company}/postings`	none	yes (`limit`/`offset`)	rarely
Recruitee	`{company}.recruitee.com/api/offers/`	none	no	rarely

Try one right now:

curl "https://boards-api.greenhouse.io/v1/boards/stripe/jobs" | jq '.jobs | length'
# 500+ open roles, in one response

Greenhouse: the workhorse (with one weird trick)

Greenhouse powers a huge share of tech job boards. One GET returns every job — no pagination even for 500+ role boards. Add ?content=true for full descriptions.

The quirk: the content field is HTML… but HTML-escaped. You'll receive <p>We are hiring</p> and need to run it through html.unescape() before parsing. Nearly every Greenhouse scraper tutorial misses this.

Dates come as ISO-8601 with offsets (2026-06-02T08:58:57-04:00), and department/office arrays are consistently populated — the most structured taxonomy of the five.

Lever: cleanest API, millisecond timestamps

Lever returns a flat JSON array — no wrapper object. Everything interesting hides in categories (department, team, commitment, location) plus a top-level workplaceType (remote / hybrid / on-site) that's more reliable than parsing location strings.

The quirks: createdAt is epoch milliseconds, not ISO. And unknown company slugs correctly return HTTP 404 — remember that; it becomes relevant below.

Ashby: the one with salary data

Ashby is the youngest of the five and increasingly common among well-funded startups (OpenAI, Linear, Ramp, Notion). Its killer feature: ?includeCompensation=true returns structured salary ranges — min, max, currency, and pay interval — on most postings.

How rich is that in practice? In a July 2026 pull of 24 well-known tech companies across all five ATSs, about 700 of 4,265 postings had structured salary ranges — and nearly all of them came from Ashby boards. (Median posted midpoint for US salaried roles: $286,000. Highest max: $585,000 for a research engineer role.)

The compensation object nests salary inside summaryComponents — sometimes at the top level, sometimes inside compensationTiers[].components. Parse both paths defensively.

SmartRecruiters: the only one that paginates (and lies about 404s)

SmartRecruiters — common among European enterprises — is the most work:

The list endpoint paginates (limit max 100, offset), returning {"totalFound": N, "content": [...]}.
The list response has no descriptions. Full text requires one extra request per posting (/postings/{id}), where the description arrives split into titled sections (companyDescription, jobDescription, qualifications) you must stitch back together.

The trap: request postings for a company that doesn't exist and you get… HTTP 200 with totalFound: 0. Not a 404. If you're auto-detecting which ATS a company uses by probing endpoints, SmartRecruiters will happily "confirm" every company name you throw at it. Require totalFound > 0 before believing it.

Recruitee: the subdomain one

Recruitee (popular with EU scale-ups) serves each company's API from its own subdomain: {company}.recruitee.com/api/offers/. That means a wrong company slug isn't a 404 — it's a DNS resolution failure. Your HTTP client raises a connection error, not an HTTP error; handle both.

Dates come as "2026-05-29 14:07:42 UTC" — not ISO-8601. And employment types arrive as composite codes like fulltime_fixed_term that need your own normalization table.

The real cost isn't fetching — it's normalizing

Any of these APIs is maybe 20 lines of Python to fetch. The actual engineering is making five schemas agree:

Dates: ISO with offsets (Greenhouse), epoch ms (Lever), ISO-ish (Ashby), "... UTC" strings (Recruitee).
Remote: a boolean (isRemote, Ashby), an enum (workplaceType, Lever), a location flag (SmartRecruiters), or nothing but the word "Remote" in a location string (Greenhouse).
Employment type: full-time vs FullTime vs permanent vs fulltime_fixed_term.
Stable IDs for deduplication and change tracking across runs.

Plus retries with backoff on 429/5xx, per-company error isolation (one dead slug shouldn't kill a 200-company run), and — if you run on a schedule — delta logic so you only process new/changed postings.

Or: one call for all six

I packaged all of the above into a single Apify actor — Job Scraper API — Workday, Greenhouse, Lever, Ashby + More. You give it company names or careers-page URLs; it auto-detects each company's ATS (including the SmartRecruiters false-positive trap and the Recruitee DNS dance), fetches everything, and returns one normalized schema:

{
  "id": "ashby:openai:8fb1615c-...",
  "title": "Technical Program Manager, Compute Infrastructure",
  "company": "openai",
  "locations": ["San Francisco"],
  "remote": false,
  "employmentType": "full-time",
  "compensation": { "min": 257000, "max": 335000, "currency": "USD", "interval": "year" },
  "postedAt": "2026-03-12T16:38:15+00:00",
  "url": "https://jobs.ashbyhq.com/openai/8fb1615c-..."
}

It costs about $1.50 per 1,000 jobs, and a delta mode returns only new/changed postings between scheduled runs — the thing you actually want if you're building alerts or tracking hiring velocity.

If you'd rather build it yourself, everything above is the map. Either way: skip the HTML scraping — the JSON was there all along.

Data points in this post are from live API pulls in July 2026. The ATS platforms can change their APIs at any time — verify against a live board before shipping.