DEV Community

Siftwright Data
Siftwright Data

Posted on

How to pull fresh job openings straight from company career pages (Greenhouse, Lever, Ashby and more)

#webscraping #jobs #python #api

Most job boards repost listings days after they appear on the company's own careers page. If you're a recruiter, job-seeker or building a job product, the freshest source is the applicant tracking system (ATS) the company itself uses: Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee and Personio all publish public job feeds.

This post shows how to query those feeds in one go.

The quick way: one call, every ATS

We published a tool on the Apify Store, ATS Jobs Search, that searches all seven feeds at once, detects which ATS a careers page uses, and can return only jobs that are new since your last run.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("siftwright_data/ats-jobs-search").call(run_input={
    "searchDirectory": True,
    "keywords": ["data engineer"],
    "locations": ["Remote", "New York"],
    "onlyNewJobs": True,
})
for job in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(job["companyName"], "|", job["title"], "|", job["url"])
Enter fullscreen mode Exit fullscreen mode

New to Apify? The free plan's monthly credit is enough to try it on thousands of jobs.

Doing it by hand (for one company)

Each ATS has a public JSON endpoint, for example:

  • Greenhouse: https://boards-api.greenhouse.io/v1/boards/{company}/jobs
  • Lever: https://api.lever.co/v0/postings/{company}?mode=json
  • Ashby: https://api.ashbyhq.com/posting-api/job-board/{company}

That works for a handful of companies. The hard part is knowing which companies use which ATS and what their board slug is, which is what the tool's built-in directory solves.

Need something custom?

If you need a feed for a specific niche delivered to a spreadsheet or Slack every morning, email siftwright.data@gmail.com and we'll quote a fixed price.

Top comments (0)