DEV Community

William Jin
William Jin

Posted on

Scrape Greenhouse, Lever and Ashby job boards into one JSON feed (Python)

#python #api #webscraping #career

Open the careers page of almost any startup or scale-up and the job list is usually served by one of three applicant tracking systems: Greenhouse, Lever or Ashby. You can tell from the URL (boards.greenhouse.io/stripe, jobs.lever.co/spotify, jobs.ashbyhq.com/ramp) or from where the "Apply" button points.

All three publish the board as public JSON, so you don't need a browser or HTML parsing. The catch is that each one uses a different shape. This post shows the three endpoints, a script that merges them into one feed, the quirks to normalize, and a hosted option you can also give to an AI agent.

The three endpoints

ATS Endpoint Notes
Greenhouse GET https://boards-api.greenhouse.io/v1/boards/{slug}/jobs Add ?content=true for descriptions (much larger response)
Lever GET https://api.lever.co/v0/postings/{slug}?mode=json createdAt is a Unix timestamp in milliseconds
Ashby GET https://api.ashbyhq.com/posting-api/job-board/{slug}?includeCompensation=true Includes isRemote, workplaceType and pay when the company publishes it

The slug is the company's board name, usually the last part of the careers URL.

One script, three boards 

import requests

def greenhouse(slug):
    r = requests.get(f"https://boards-api.greenhouse.io/v1/boards/{slug}/jobs", timeout=30)
    r.raise_for_status()
    for j in r.json()["jobs"]:
        yield {"company": slug, "title": j["title"], "location": j["location"]["name"],
               "posted": j.get("first_published"), "url": j["absolute_url"]}

def lever(slug):
    r = requests.get(f"https://api.lever.co/v0/postings/{slug}?mode=json", timeout=30)
    r.raise_for_status()
    for j in r.json():
        yield {"company": slug, "title": j["text"], "location": j["categories"].get("location"),
               "posted": j["createdAt"], "url": j["hostedUrl"]}

def ashby(slug):
    r = requests.get(f"https://api.ashbyhq.com/posting-api/job-board/{slug}?includeCompensation=true", timeout=30)
    r.raise_for_status()
    for j in r.json()["jobs"]:
        yield {"company": slug, "title": j["title"], "location": j["location"],
               "posted": j["publishedAt"], "url": j["jobUrl"]}

boards = [(greenhouse, "stripe"), (lever, "spotify"), (ashby, "ramp")]
jobs = [j for fetch, slug in boards for j in fetch(slug)]
data_jobs = [j for j in jobs if "data" in j["title"].lower()]
print(len(jobs), "jobs,", len(data_jobs), "with 'data' in the title")
for j in data_jobs[:3]:
    print(j)
Enter fullscreen mode Exit fullscreen mode

When I ran it, it returned 976 jobs across the three companies, 41 of them with "data" in the title:

{'company': 'stripe', 'title': 'Account Executive, Product Sales (Data)', 'location': 'US-SF-HQ, US-NYC, US-CHI, US-SEA', 'posted': '2026-10-05T12:08:53-04:00', 'url': 'https://stripe.com/jobs/search?gh_jid=8190059'}
{'company': 'stripe', 'title': 'Data Analyst', 'location': 'Canada', 'posted': '2023-11-01T13:50:50-04:00', 'url': 'https://stripe.com/jobs/search?gh_jid=5416444'}
Enter fullscreen mode Exit fullscreen mode

That's a usable feed in about 30 lines. Here's what you'll want to fix before relying on it.

The gotchas

1. Dates mean different things. Greenhouse has updated_at (changes whenever the posting is edited) and first_published (when it went live). Use first_published for "new jobs" alerts, or a job edited yesterday looks brand new. Lever's createdAt is milliseconds since the epoch (1792767640768), so divide by 1000 before converting. Ashby gives an ISO publishedAt.

2. Location is free text. Greenhouse returns whatever the company typed ("US-SF-HQ, US-NYC, US-CHI, US-SEA"), Lever has a categories.location string plus a separate workplaceType (onsite, hybrid, remote), and Ashby has location, secondaryLocations and isRemote. To filter by "remote" or "Germany" reliably, check all of those fields, not just the location string.

3. Pay is only sometimes structured. Ashby returns a compensation object when the company shows pay (for example "$211.4K – $290.6K • Offers Equity" as a summary). Greenhouse and Lever usually put pay ranges inside the description text, so you need a parser that handles $90K, €60,000, ranges and hourly rates.

4. Descriptions are heavy. Greenhouse's ?content=true turned Stripe's board into a 5.6 MB response. Fetch descriptions only when you need them.

5. Unknown slugs return 404. A typo in the slug and a company that moved to a different ATS look the same. Log which boards returned nothing, so an empty result doesn't quietly look like "no jobs".

6. Be polite. These are public endpoints, but they serve the company's real careers page. One request per board per run is plenty; back off on 429 and 5xx.

If you'd rather not maintain it

Disclosure: I packaged this as a hosted tool, Career Site Jobs Scraper on Apify, with the source on GitHub. It covers Greenhouse, Lever, Ashby, Workable and Recruitee, accepts board names or full URLs, and returns one normalized row per job, with dates, remote status and parsed salary.

The input:

{
  "companies": ["stripe", "https://jobs.lever.co/spotify", "https://jobs.ashbyhq.com/ramp"],
  "titleKeywords": ["data"],
  "maxJobsPerCompany": 3,
  "includeDescription": false
}
Enter fullscreen mode Exit fullscreen mode

One row from that run (an Ashby job with published pay), trimmed:

{
  "ats": "ashby",
  "companySlug": "ramp",
  "companyName": "Ramp",
  "title": "Data Scientist",
  "department": "Data",
  "location": "New York, NY (HQ)",
  "locations": ["New York, NY (HQ)", "San Francisco, CA", "Remote (US)"],
  "remote": true,
  "workplaceType": "hybrid",
  "employmentType": "FullTime",
  "postedAt": "2026-06-03T17:53:31.348Z",
  "jobUrl": "https://jobs.ashbyhq.com/ramp/e577622f-6657-4e53-8941-b3a774b04448",
  "salaryText": "$137,120 - $297,330",
  "salaryMin": 137120,
  "salaryMax": 297330,
  "salaryCurrency": "USD",
  "salaryInterval": "year"
}
Enter fullscreen mode Exit fullscreen mode

The run also writes a per-company summary, so you can see why a board returned few rows:

{"input": "stripe", "ats": "greenhouse", "status": "ok", "jobsFound": 719, "jobsMatched": 3, "jobsSaved": 3}
Enter fullscreen mode Exit fullscreen mode

Other filters, applied before anything is charged: excludeTitleKeywords, locationKeywords, departmentKeywords, remoteOnly and postedSince (a date or a period like "7 days").

Call it from Python 

import os
from apify_client import ApifyClient  # pip install apify-client

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("swillz/career-site-jobs-scraper").call(
    run_input={
        "companies": ["stripe", "https://jobs.lever.co/spotify", "https://jobs.ashbyhq.com/ramp"],
        "titleKeywords": ["data"],
        "postedSince": "14 days",
        "includeDescription": False,
    },
    max_total_charge_usd=0.05,
)
for job in client.dataset(run.default_dataset_id).iterate_items():
    print(job.get("companyName"), "|", job.get("title"), "|", job.get("location"))
Enter fullscreen mode Exit fullscreen mode

max_total_charge_usd caps what the run can cost; when it's reached the run stops cleanly and keeps the rows it already saved. (On 1.x versions of the client, use run["defaultDatasetId"].)

Let an AI agent pull the jobs (MCP)

In Claude, Cursor, VS Code or another MCP client, add Apify's hosted MCP server with just this tool:

{
  "mcpServers": {
    "apify": { "url": "https://mcp.apify.com?tools=swillz/career-site-jobs-scraper" }
  }
}
Enter fullscreen mode Exit fullscreen mode

Then ask something like:

"Find remote data or ML jobs posted in the last 7 days at stripe, ramp, notion and figma, and give me a table with company, title, location and link."

The agent fills in companies, titleKeywords, remoteOnly: true and postedSince: "7 days". Tips:

  • Keep answers fast. Have the agent set maxJobs to 50–200 so the run finishes within the MCP call-actor wait (about 45 seconds).
  • Empty results come with a reason. A company with no matching jobs gets a free status row such as "N open jobs found, 0 matched your filters" or "not_found", so the agent can fix the slug or loosen a filter instead of reporting "no jobs".
  • Chain it. The dataset ID from the run can go straight into a Google Sheets export or a Slack webhook.

A daily schedule with postedSince: "1 day" across a list of companies gives you a "new jobs at companies I care about" feed without maintaining any scrapers.

Wrap-up

  • Greenhouse, Lever and Ashby all serve job boards as public JSON; the slug is in the careers URL.
  • Normalize dates (first_published, millisecond createdAt), locations and remote flags before filtering.
  • Pay is structured on Ashby when published; elsewhere it's in the description text.
  • Track boards that return nothing, so a wrong slug doesn't look like "not hiring".

Top comments (0)