DEV Community

Martin Petrov
Martin Petrov

Posted on

Track new jobs at any Greenhouse, Lever or Ashby company daily with Python

#python #api #jobs #automation

You want to know the day a company you care about opens a new role, and checking ten careers pages by hand every morning is not a plan.

Maybe you are a job seeker with a short list of dream companies. Maybe you sell to engineering teams and "they just opened 4 data engineer roles" is your best buying signal. Maybe you run a niche job board. In every case the data is public, but it is spread across different applicant tracking systems (ATS), each with its own page layout and its own JSON shape.

I built a small tool for this, and in this post I show how to use it from Python: pull jobs from Greenhouse, Lever and Ashby (plus Workday and six other ATS) in one call, save them to CSV, and run it every day so you only see new postings.

Why the usual approach hurts

The usual path looks like this:

  1. Scrape the careers page HTML. It breaks on the next redesign, and big career sites are rendered with JavaScript.
  2. Find the ATS behind it. Greenhouse has boards-api.greenhouse.io, Lever has api.lever.co, Ashby has its own posting API, Workday has its own paginated endpoint per tenant. Each one returns different field names for the same thing.
  3. Write a normaliser per ATS, a dedupe step, and some state so tomorrow's run knows what you already saw.

Steps 2 and 3 are where the time goes. None of it is hard, but it is a weekend of glue code, and then you have to maintain it.

What I built

ATS Jobs Scraper is an Apify actor that reads the public job-board endpoints of 10 ATS (Workday, Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee, Personio, Teamtailor, Breezy HR) and returns every job in one schema: title, department, location, remote flag, salary range when the company publishes one, posting date and apply link. No login, no browser, no CAPTCHA, because it only uses the endpoints the careers pages themselves call.

You give it a list like this:

greenhouse:stripe
lever:palantir
ashby:ramp
https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite
Enter fullscreen mode Exit fullscreen mode

A plain careers URL works too (https://boards.greenhouse.io/stripe, https://jobs.lever.co/palantir), and for most ATS a bare company slug is auto-detected.

Step 1: get an Apify token

Create a free Apify account and copy your API token from Settings -> API & Integrations. Put it in an environment variable so it never ends up in your code:

export APIFY_TOKEN="your-token-here"
Enter fullscreen mode Exit fullscreen mode

Step 2: one call with curl

The run-sync-get-dataset-items endpoint starts the actor, waits for it to finish and returns the results in the same HTTP response:

curl -s -X POST \
  "https://api.apify.com/v2/acts/martin10291~ats-jobs-aggregator/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "companies": ["greenhouse:stripe", "lever:palantir", "ashby:ramp"],
    "titleKeywords": ["engineer"],
    "maxJobsPerCompany": 2,
    "maxItems": 6,
    "includeDescription": false
  }'
Enter fullscreen mode Exit fullscreen mode

I ran exactly this input today (2026-11-05). The run took 5.5 seconds. These are the 6 rows it returned, trimmed to the main fields:

company title location postedAt salary
Stripe AI Engineer Chicago 2026-10-07 -
Stripe Android BSP Engineer Taipei, Taiwan 2026-09-29 -
Palantir Backend Software Engineer - Application Development London, United Kingdom 2024-03-11 -
Palantir Backend Software Engineer - Application Development New York, NY 2024-03-11 -
Ramp Security Engineer, Cloud New York, NY (HQ) 2026-04-07 211,400-290,600
Ramp Mobile Engineer, Android New York, NY (HQ) 2025-07-31 189,000-330,000

And one full row, exactly as it came back:

{
  "id": "greenhouse:stripe:8144196",
  "ats": "greenhouse",
  "companySlug": "stripe",
  "company": "Stripe",
  "title": "AI Engineer",
  "department": "1155 SA Transformation",
  "location": "Chicago",
  "locations": ["US"],
  "isRemote": false,
  "salaryMin": null,
  "salaryMax": null,
  "salaryCurrency": null,
  "postedAt": "2026-10-07T15:39:36Z",
  "url": "https://stripe.com/jobs/search?gh_jid=8144196",
  "applyUrl": "https://stripe.com/jobs/search?gh_jid=8144196",
  "scrapedAt": "2026-11-05T20:16:52Z"
}
Enter fullscreen mode Exit fullscreen mode

The run also writes a RUN_SUMMARY record to its key-value store. Today it said Stripe had 763 open jobs, Palantir 308 and Ramp 170, and that 2 were saved from each because of my limits. That summary is handy when you want to chart how many roles a company has open over time.

Step 3: the same thing in Python, saved to CSV 

import csv
import os

import requests

TOKEN = os.environ["APIFY_TOKEN"]
URL = ("https://api.apify.com/v2/acts/martin10291~ats-jobs-aggregator"
       "/run-sync-get-dataset-items")

payload = {
    "companies": [
        "greenhouse:stripe",
        "lever:palantir",
        "ashby:ramp",
        "https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite",
    ],
    "titleKeywords": ["engineer", "data"],
    "excludeKeywords": ["intern"],
    "locations": ["London", "Berlin", "Remote"],
    "includeRemoteWithLocations": True,
    "postedWithinDays": 14,
    "includeDescription": False,
    "maxItems": 200,
}

resp = requests.post(URL, params={"token": TOKEN}, json=payload, timeout=300)
resp.raise_for_status()
jobs = resp.json()
print(f"{len(jobs)} jobs")

fields = ["company", "title", "location", "isRemote", "salaryMin",
          "salaryMax", "salaryCurrency", "postedAt", "applyUrl"]
with open("jobs.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=fields, extrasaction="ignore")
    w.writeheader()
    w.writerows(jobs)
Enter fullscreen mode Exit fullscreen mode

The filters run inside the actor, so you only pay for the rows you keep. postedWithinDays and minSalary are useful here; note that minSalary drops jobs that publish no salary at all.

If you want the rows in Google Sheets instead of a CSV, gspread takes the same list:

import gspread

gc = gspread.service_account(filename="service_account.json")
ws = gc.open("Job feed").sheet1
ws.append_rows([[j.get(k) for k in fields] for j in jobs])
Enter fullscreen mode Exit fullscreen mode

Step 4: run it daily and get only new jobs

This is the part that saves the most glue code. Add one flag:

payload["onlyNewJobs"] = True
payload["postedWithinDays"] = 0  # let the state decide what is new
Enter fullscreen mode Exit fullscreen mode

With onlyNewJobs the actor stores the IDs it has already returned for that exact company list in a named key-value store in your Apify account. The next run with the same list returns only jobs that were not there before. The first run returns everything (that is your baseline); after that you get the daily delta.

Two ways to schedule it:

  • Apify Schedules: open the actor, save your input as a task, then add a schedule (for example 0 7 * * *). Connect the task to a webhook, Slack, email or Zapier/Make so new jobs land where you work.
  • Your own cron: keep the Python script above and run it from cron or GitHub Actions: 
0 7 * * * cd /home/me/jobfeed && APIFY_TOKEN=xxx python3 feed.py >> feed.log 2>&1
Enter fullscreen mode Exit fullscreen mode

If you change the company list, the state key changes too, so a new list starts a fresh baseline.

Not on Apify? Same data as a plain REST API

I also list the same scraper as a regular REST API on RapidAPI and API.market (search for "Jobs API - Workday, Greenhouse, Lever, Ashby"). That route makes sense if you already have a RapidAPI key and want one HTTP call without an Apify account. The Apify route above is the most direct one and the one I tested for this post.

Honest limits

  • It only sees companies you name. It is not a search engine over all jobs on the internet. You bring the list; the tool reads those boards.
  • Salary is only as good as the source. Ashby, Lever and Recruitee often return structured ranges (like the Ramp rows above). For other ATS the actor tries to detect a range in the text, and many postings simply have none, like the Stripe rows above.
  • Fields mean what the ATS says. Palantir's backend roles show postedAt 2024-03-11 because those are evergreen postings that stay open for years. Stripe's location and locations can differ because Greenhouse stores them separately. I keep the original values instead of guessing.
  • Workday tenants are slower. Big Workday sites have thousands of roles and paginate, so set maxJobsPerCompany or keyword filters if you only need a slice.
  • ATS changes happen. If a company moves from Lever to Ashby, its old slug returns 0 jobs. The RUN_SUMMARY shows jobs per company, so a sudden 0 is easy to spot.

Price

Pay per result: you pay only for jobs saved after your filters, with no start fee. It is $1 per 1,000 jobs until 13 November 2026 and $2 per 1,000 jobs after that. With pay-per-result pricing the Apify compute is included, so the 6-row run above cost $0.006 at today's price. Apify's free plan includes a small monthly credit, enough to try it on your own company list.

Wrap-up

I built this because I wanted one feed of "new roles at these companies" without maintaining ten scrapers. If you try it and a company's board does not parse, or you need an ATS I do not cover yet, leave a comment or open an issue on the actor page and I will look at it.

Next in this series: turning the daily new-jobs feed into a hiring-signal list for sales (which companies just opened roles in a given department).

Top comments (0)