DEV Community

Andre van Schalkwyk
Andre van Schalkwyk

Posted on AI-assisted

How to get every job from Greenhouse, Lever and Ashby career pages (Python)

#python #webscraping #api #tutorial

Most tech companies don't host their own job listings. They use an applicant tracking system (ATS), and the big ones (Greenhouse, Lever and Ashby) each expose a public job-board feed that powers the company's careers page. If you want job data for a list of companies, these feeds beat scraping LinkedIn or Indeed: they're the source of truth, they're structured, and they don't block you.

The catch: every ATS is different

  • Greenhouse boards live at boards.greenhouse.io/<board>; pay ranges only appear if you ask for them and the company publishes them.
  • Lever boards live at jobs.lever.co/<company>, with "commitment" and "workplaceType" fields and a different salary shape.
  • Ashby boards live at jobs.ashbyhq.com/<company> and often include compensation, in yet another format.

And you first need to know which ATS a company uses. "Stripe" doesn't tell you that.

Option 1: do it yourself

For a single ATS, it's a few lines of requests against the public board endpoint plus a mapping function. Multiply that by eight platforms, add platform detection, salary parsing from free text, seniority inference and de-duplication for "new jobs only", and it becomes a small project to maintain.

Option 2: one normalised API

Disclosure: I built this. I wrapped all of that into a hosted API on Apify that takes company names, careers URLs or platform:board pairs and returns one schema for all eight platforms:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("cape-data-labs/company-career-jobs-api").call(
    run_input={"companies": ["https://boards.greenhouse.io/airbnb", "lever:palantir", "ashby:ramp"]}
)
for job in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(job["company"], job["title"], job["workplaceType"], job["salary"])
Enter fullscreen mode Exit fullscreen mode

Each job comes back with title, department, location, remote/hybrid/on-site, seniority, employment type, salary (min/max/currency/period), posted date and apply URL. Turn on onlyNewJobs and schedule it daily for alerts; you only pay for postings you haven't seen ($1.50 per 1,000 jobs).

More examples (CSV export, Slack alerts, remote-with-salary search, hiring-intent lead scoring): https://github.com/andrevs/ats-jobs-api-examples

What I learned building it

  • Platform detection by name is a guessing game; careers-page URLs are always faster and more reliable.
  • Pay ranges are more common than you'd think: in one test run, 145 of Ramp's 171 open roles had one. But every ATS stores them differently.
  • Job ads are a surprisingly good technographic source: companies name their CRM, warehouse and cloud in them. That became a second API, Hiring Signals.

Top comments (0)