DEV Community

Devil Scrapes
Devil Scrapes

Posted on

BuiltIn Salary Data: Pulling Structured Comp Bands Without the Copy-Paste

BuiltIn salary data is sitting in plain sight on every job card — and there's still no bulk export button for it, just 25 cards per page and a "load more" click.

If you've tried to build any kind of comp-band analysis, competitor hiring tracker, or recruiting pipeline off BuiltIn's public listings, you've hit the same wall: the site is a great read-only aggregator and a terrible bulk-data source. Salary ranges, seniority, top skills, and work-mode all live inside inconsistently formatted card markup — and BuiltIn changes the DOM shape often enough that a one-off scraper written last quarter is a coin flip whether it still works.

We built an Actor that does the extraction for you and hands back typed rows: salary_min, salary_max, seniority, work_mode, top_skills, categories — all normalized, all queryable, all yours in JSON, CSV, or Excel. This post walks through what the data looks like, why the naive approach breaks down faster than it looks, and exactly what it costs to run at scale.

What the data looks like

Here's one row from a search for "engineer":

{
  "job_id": "10451551",
  "title": "Sr. Finance Manager, Supply Chain",
  "url": "https://builtin.com/job/sr-finance-manager-supply-chain/10451551",
  "company_name": "Aerospace Corporation",
  "location": "El Segundo, CA, USA",
  "locations": ["El Segundo, CA, USA"],
  "work_mode": "Hybrid",
  "posted_text": "7 Minutes Ago",
  "posted_at_estimate": "2026-07-30T14:23:00Z",
  "salary_raw": "142K-204K Annually",
  "salary_min": 142000,
  "salary_max": 204000,
  "seniority": "Senior level",
  "categories": ["Aerospace", "Machine Learning", "Cybersecurity", "Defense"],
  "top_skills": ["Excel", "SAP", "Forecasting"],
  "description_snippet": "The Aerospace Corporation is seeking a Senior Finance Manager..."
}
Enter fullscreen mode Exit fullscreen mode

Notice salary_min / salary_max are already integers — parsed and expanded from BuiltIn's "142K-204K Annually" text — and posted_at_estimate is a real ISO-8601 timestamp derived from BuiltIn's relative-time strings like "7 Minutes Ago". Neither of those is free; both are exactly the kind of thing that breaks the first time you try to pandas.read_html() your way through this site.

The naive approach, and why it stalls

The instinct is reasonable: open devtools, find the job cards, write a quick requests.get() + BeautifulSoup loop, done in an afternoon. Three things derail that plan pretty fast.

First, multi-location listings collapse into a tooltip. A card showing "3 Locations" isn't three rows — it's one card whose real location list only renders on hover, which means a plain HTML parse either drops the extra cities or has to reconstruct them from a truncated string. Second, salary text isn't structured — "142K-204K Annually" needs its own parser to become two usable integers, and BuiltIn is not consistent about whether a listing has one number, two, or none at all. Third, posted-time is always relative ("7 Minutes Ago", "2 Days Ago") — useful to a human scanning the page, useless for any freshness filter or dedup job unless you convert it to an absolute timestamp at scrape time, because the string itself goes stale the moment the page ages.

None of that is exotic engineering. It's just enough plumbing that it eats a weekend, and enough of a moving target that it needs re-checking whenever BuiltIn tweaks card markup — which, on a site this actively maintained, happens more than you'd like.

The Actor

We built this once so you don't have to rebuild it every time BuiltIn nudges a class name. [Screenshot suggestion: Apify Console run page mid-execution, showing the dataset item counter climbing and the "Input" panel with search: "data scientist" visible.]

Run it via the Apify Python SDK:

from apify_client import ApifyClient

client = ApifyClient("APIFY_TOKEN")
run = client.actor("DevilScrapes/builtin-jobs-scraper").call(
    run_input={
        "search": "data scientist",
        "category": "Data Science",
        "maxResults": 200,
    }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item["salary_min"]:
        print(item["title"], item["salary_min"], "-", item["salary_max"])
Enter fullscreen mode Exit fullscreen mode

Or the raw JSON input if you're calling it from the Apify Console or another Actor:

{
  "search": "engineer",
  "city": null,
  "category": null,
  "maxResults": 200,
  "proxyConfiguration": { "useApifyProxy": true }
}
Enter fullscreen mode Exit fullscreen mode

We rotate curl-cffi browser TLS impersonation (Chrome, Firefox, Safari fingerprints) across requests so the target sees ordinary browser traffic, and we rotate proxies through Apify Proxy with a fresh session on any block. Retries use exponential backoff on 408 / 429 / 5xx, capped at 5 attempts, and we honour Retry-After when BuiltIn sends it. If a run only gets partway through before the target pushes back, the run reports exactly how many rows it landed instead of quietly returning an empty dataset — you always know what you paid for.

What you'd actually use this for

  • Recruiting-agency sourcing — pull fresh openings for a target role across dozens of employers in one feed, filtered by keyword, city, or category, instead of manually checking each employer's careers page.
  • Comp-band benchmarking — aggregate salary_min/salary_max across a category or city to see where a role's market rate actually sits this quarter, not last year's survey data.
  • Sales intel for HR-tech vendors — a spike in postings for a specific category or seniority level is a clean buying-signal proxy for "this company is actively hiring."
  • Hiring-mode tracking — track the remote/hybrid/onsite split (work_mode) for a role or region over time as return-to-office policies shift.
  • Feed normalization — drop a clean BuiltIn slice into a larger multi-source job dataset alongside other boards, using the same typed schema across sources.

Pricing — the actual numbers

Pay-Per-Event: $0.02 per run (one-off warm-up) plus $0.0015 per result row. A 1,000-result pull costs $1.52 total. There's no subscription and no minimum — Apify gives every new account $5 of free trial credit, no card required, which covers roughly 3,000 rows before you spend anything.

The part that's genuinely fiddly

The most interesting failure mode isn't blocking — it's silence. An unmatched city or category slug doesn't error; BuiltIn just returns zero results for a filter that looks perfectly valid. We pass your free-text filter through normalized (so "Data Science" and "data-science" both resolve the same way), but a typo still comes back as an empty, technically-successful run rather than a 404 you can catch. We surface a clear status message either way — "0 results for this filter" reads very differently from "target blocked us mid-run" — but it's worth building your own sanity check (a known-good baseline search) into any pipeline that runs unattended.

What this doesn't do

This Actor scrapes what's visible on the listing card — it doesn't follow through to each job's own detail page or the employer's company profile, so if you need the full job description body rather than the snippet, or company-level metadata like headcount, that's a separate crawl. There's also no enumerated taxonomy of valid cities or categories to validate against ahead of time; BuiltIn doesn't publish one, so filters are resolved live, same as if you typed them into the site's own search bar.

Try it

The Actor is live on the Apify Store: https://apify.com/DevilScrapes/builtin-jobs-scraper. Start with the $5 free trial credit — no card required — and a 500-row test run costs well under a dollar.

If you're doing comp-band research or building a recruiting feed off BuiltIn today, I'd genuinely like to know what fields you wish were in the output — company headcount, funding stage, and full JD text are the three most-requested candidates so far. Drop a comment or open an issue on the Actor's Apify Console listing.


Sources referenced in this post: BuiltIn (the target site), Apify Python SDK docs (client usage shown above), SHRM's pay-transparency law tracker (context on why more employers now publish salary bands up front, which is part of why this dataset exists to scrape in the first place).

Top comments (0)