DEV Community

Data Drip
Data Drip

Posted on

How to scrape Greenhouse, Lever and Ashby job postings to CSV/JSON

#python #webscraping #api #tutorial

Disclosure: I maintain a hosted version of this as an Apify actor (linked at the end). Everything before that section is free and needs only Python.

If you're building a job board, tracking competitors' hiring, or feeding "hiring signals" into a sales pipeline, the best data source isn't LinkedIn. It's the applicant tracking system (ATS) behind each company's careers page. Three ATSs power a huge share of startup and scale-up careers pages, and all three publish public, unauthenticated JSON:

ATS Careers URL looks like Public API
Greenhouse boards.greenhouse.io/stripe or job-boards.greenhouse.io/stripe GET https://boards-api.greenhouse.io/v1/boards/{slug}/jobs?content=true
Lever jobs.lever.co/palantir GET https://api.lever.co/v0/postings/{slug}?mode=json (EU: api.eu.lever.co)
Ashby jobs.ashbyhq.com/ramp GET https://api.ashbyhq.com/posting-api/job-board/{slug}?includeCompensation=true

No HTML parsing, no headless browser, no proxies. One request returns every open job for that company.

1. Find the slug

The slug is the path segment after the ATS host in the careers URL: stripe, palantir, ramp. If a company's careers page is on its own domain, open any job and look at the "Apply" link or the page source for greenhouse.io, lever.co or ashbyhq.com.

2. What each API returns

Greenhouse: {"jobs": [...], "meta": {"total": N}}. Each job has:

  • title, location.name, absolute_url
  • updated_at, first_published
  • departments
  • with content=true, the description as HTML-escaped HTML in content

Lever: a plain list. Each posting has:

  • text (the title)
  • categories.location / team / commitment
  • hostedUrl, applyUrl
  • createdAt (epoch milliseconds)
  • workplaceType
  • descriptionPlain
  • sometimes salaryRange

Ashby: {"jobs": [...]}. Each job has:

  • title, department, team, location
  • isRemote, workplaceType
  • publishedAt, jobUrl
  • descriptionPlain
  • with includeCompensation=true, a compensation object with a human-readable compensationTierSummary (e.g. "$211.4K – $290.6K • Offers Equity")

3. A complete script: three ATSs → one CSV 

import csv, html, re, requests

def strip_html(s):
    return re.sub(r"\s+", " ", re.sub(r"<[^>]+>", " ", html.unescape(s or ""))).strip()

def greenhouse(slug):
    r = requests.get(f"https://boards-api.greenhouse.io/v1/boards/{slug}/jobs",
                     params={"content": "true"}, timeout=30)
    if r.status_code == 404:
        return None
    r.raise_for_status()
    return [{
        "company": slug, "ats": "greenhouse", "title": j["title"],
        "location": (j.get("location") or {}).get("name"),
        "department": ", ".join(d["name"] for d in j.get("departments", [])) or None,
        "posted": j.get("first_published") or j.get("updated_at"),
        "url": j["absolute_url"], "salary": None,
        "description": strip_html(j.get("content")),
    } for j in r.json()["jobs"]]

def lever(slug):
    for host in ("api.lever.co", "api.eu.lever.co"):     # EU-hosted boards live on api.eu.lever.co
        r = requests.get(f"https://{host}/v0/postings/{slug}", params={"mode": "json"}, timeout=30)
        if r.status_code == 404:
            continue
        r.raise_for_status()
        out = []
        for j in r.json():
            sal = j.get("salaryRange") or {}
            out.append({
                "company": slug, "ats": "lever", "title": j["text"],
                "location": (j.get("categories") or {}).get("location"),
                "department": (j.get("categories") or {}).get("team"),
                "posted": j.get("createdAt"),               # epoch milliseconds
                "url": j["hostedUrl"],
                "salary": f'{sal.get("min")}-{sal.get("max")} {sal.get("currency")}' if sal else None,
                "description": j.get("descriptionPlain"),
            })
        return out
    return None

def ashby(slug):
    r = requests.get(f"https://api.ashbyhq.com/posting-api/job-board/{slug}",
                     params={"includeCompensation": "true"}, timeout=30)
    if r.status_code == 404:
        return None
    r.raise_for_status()
    return [{
        "company": slug, "ats": "ashby", "title": j["title"],
        "location": j.get("location"), "department": j.get("department"),
        "posted": j.get("publishedAt"), "url": j["jobUrl"],
        "salary": (j.get("compensation") or {}).get("compensationTierSummary"),
        "description": j.get("descriptionPlain"),
    } for j in r.json()["jobs"]]

FETCHERS = {"greenhouse": greenhouse, "lever": lever, "ashby": ashby}

def jobs_for(company):
    """'ashby:ramp' -> that ATS only; plain 'notion' -> try each ATS in turn."""
    ats, _, slug = company.rpartition(":")
    if ats:
        return FETCHERS[ats](slug) or []
    for fetch in FETCHERS.values():
        try:
            jobs = fetch(company)
        except requests.HTTPError:
            jobs = None
        if jobs:                      # empty boards can exist on several ATSs, keep looking
            return jobs
    return []

rows = []
for company in ["stripe", "lever:palantir", "ashby:ramp", "notion"]:
    rows += [j for j in jobs_for(company) if "engineer" in j["title"].lower()]

with open("jobs.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=list(rows[0]))
    w.writeheader()
    w.writerows(rows)
print(f"{len(rows)} jobs written to jobs.csv")
Enter fullscreen mode Exit fullscreen mode 
pip install requests
python ats_jobs.py
# e.g. "433 jobs written to jobs.csv"
Enter fullscreen mode Exit fullscreen mode

On 2 November 2026 this returned 433 open engineering roles across Stripe (Greenhouse), Palantir (Lever), Ramp and Notion (Ashby).

4. Gotchas worth knowing

  • Slug collisions. The same slug can exist on more than one ATS. Plain ramp returns a 1-job Greenhouse board, while Ramp's real board is on Ashby with 100+ jobs. When you know the ATS, prefix it (ashby:ramp). When you don't, prefer the board with the most jobs, or check that the company name matches.
  • Lever EU. Some companies are hosted on api.eu.lever.co. A 404 on the US host doesn't mean "no board".
  • Salary. Ashby and Lever expose it as structured fields when the company publishes it. On Greenhouse it's usually only in the description text, so you need a regex like [$€£]\s?\d{2,3}[kK]\s?[-–]\s?[$€£]?\d{2,3}[kK] or an LLM.
  • Dates. Greenhouse uses ISO strings, Lever uses epoch ms, Ashby uses ISO with offset. Normalise before filtering by "posted in the last N days".
  • Descriptions contain personal data sometimes (recruiter emails and phone numbers). Strip them if you store or redistribute.
  • Only open roles. None of these APIs return closed postings, so if you need history, snapshot daily and diff.

5. Going further: enrichment

Raw postings answer "what's open". Most use cases need more:

  • Is this senior?
  • Which skills?
  • Remote or hybrid?
  • Visa sponsorship?

The pragmatic approach is a cheap LLM (e.g. gpt-4o-mini) with a strict JSON schema per job.

6. Hosted option: ATS Jobs Scraper API + AI (what I built)

If you don't want to maintain the glue, I packaged all of the above as an Apify actor: ATS Jobs Scraper API: Greenhouse, Lever & Ashby + AI.

  • companies accepts:
    • careers URLs
    • prefixed slugs (greenhouse:stripe, lever:palantir, ashby:ramp)
    • plain names, which are auto-detected across all three ATSs
  • Filters:
    • keywords
    • locations
    • remoteOnly
    • postedWithinDays
    • maxJobsPerCompany / maxJobsTotal
  • One normalised schema for all three platforms. Salary comes from Ashby/Lever fields, with a fallback to parsing the description text. Emails and phone numbers are stripped from descriptions.
  • Optional AI enrichment with your own OpenAI or OpenRouter key (default gpt-4o-mini):
    • seniority, jobFunction, remotePolicy
    • skills[], yearsExperienceMin
    • salaryEstimate (only when stated)
    • visaSponsorship, languages[]
    • a short summary

Example input:

{"companies":["https://boards.greenhouse.io/stripe","lever:palantir","ashby:ramp","notion"],"keywords":["engineer"],"maxJobsPerCompany":200}
Enter fullscreen mode Exit fullscreen mode

From code (replace <APIFY_TOKEN> with your own token):

curl -X POST "https://api.apify.com/v2/acts/data-drip~ats-jobs-scraper/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"companies":["https://boards.greenhouse.io/stripe","lever:palantir","ashby:ramp","notion"],"keywords":["engineer"],"maxJobsPerCompany":200}'
Enter fullscreen mode Exit fullscreen mode

Pricing is pay per result:

  • $0.0015 per job ($1.50 per 1,000; 10,000 jobs = $15)
  • $0.002 per AI-enriched job, plus your own LLM tokens

So 1,000 jobs with AI cost $3.50 plus tokens. Apify's free plan includes $5 of monthly usage, which covers ~3,333 jobs without AI.

It runs on a schedule (Apify Schedules), exports JSON/CSV/Excel, and plugs into Google Sheets, Zapier, Make, n8n, Clay ("Run Apify Actor") and AI agents via the Apify MCP server.

Limits:

  • Greenhouse, Lever and Ashby only: no Workday, LinkedIn or Indeed.
  • Open postings only.

Wrap-up

  • Before you reach for a browser scraper, check whether the careers page runs on Greenhouse, Lever or Ashby. If it does, there's a free JSON endpoint.
  • Normalise early (title, location, department, posted date, URL, salary) and key on the job URL.
  • Watch for slug collisions and Lever's EU host.

Questions or a company whose board you can't find? Drop it in the comments.

Top comments (0)