DEV Community

Noble Ronin
Noble Ronin

Posted on

Skip LinkedIn/Indeed: most companies' job boards have a public JSON API

#webscraping #python #api #jobs

If you've ever tried to pull job listings by scraping LinkedIn or Indeed, you know the pain: anti-bot systems, CAPTCHAs, rotating proxies, and scripts that silently break every few weeks.

Here's the thing — you usually don't need any of that.

Companies don't post jobs on LinkedIn first. They post them in their ATS (Applicant Tracking System) — Greenhouse, Lever, Ashby, Workday, etc. — and most ATS platforms expose the company's board as a public JSON endpoint. No key, no login, no browser. It's the company's own source of truth, so it's cleaner and fresher than any aggregator.

The endpoints

A few that work with a plain GET ({company} = the company's slug):

  • Greenhouse — https://boards-api.greenhouse.io/v1/boards/{company}/jobs?content=true
  • Lever — https://api.lever.co/v0/postings/{company}?mode=json
  • Recruitee — https://{company}.recruitee.com/api/offers/
  • Breezy HR — https://{company}.breezy.hr/json

SmartRecruiters, Ashby, BambooHR and Personio have their own equivalents. Workday is the one annoying exception — it's a POST and needs the full board URL (tenant + datacenter + site), so you can't guess it from a bare company name.

Example: pulling Stripe's open roles (Python)

Stripe uses Greenhouse:

import requests

company = "stripe"
url = f"https://boards-api.greenhouse.io/v1/boards/{company}/jobs?content=true"
jobs = requests.get(url).json()["jobs"]

for j in jobs[:5]:
    print(j["title"], "—", j["location"]["name"])
Enter fullscreen mode Exit fullscreen mode

That's it. No Selenium, no proxy, no CAPTCHA solver. Runs in ~200ms and won't break next Tuesday because Cloudflare changed something.

Auto-detecting the ATS

If you don't know which ATS a company uses, just try them in order and take the first one that returns jobs. A bare 404 means "not this ATS, try the next." Greenhouse → Lever → Ashby → SmartRecruiters → Recruitee → Breezy covers a huge chunk of tech companies.

Gotchas

  • Rate limits are lenient but real — be polite, set a User-Agent.
  • Descriptions: Greenhouse/Lever include full HTML descriptions in the list; Breezy's list endpoint doesn't (you'd fetch each posting for that).
  • Workday: POST-only, needs the full board URL — worth special-casing.

If you don't want to build/maintain 9 of these

I got tired of keeping a fetcher per ATS in sync, so I bundled 9 of them (Greenhouse, Lever, Ashby, SmartRecruiters, Workday, BambooHR, Personio, Recruitee, Breezy) into one thing that auto-detects the ATS from a bare company name or board URL and returns normalized jobs: https://apify.com/ponderable_hydrometer/multi-ats-jobs

But honestly, the endpoints above are the real value — build your own if you'd rather.

Which ATS should I add next? Considering Workable, Teamtailor and JazzHR.

Top comments (0)