Get every open job from Greenhouse, Lever and Ashby in 40 lines of Python

#python #webscraping #jobs #api

If you want to build a job board, track which companies are hiring, or benchmark salaries, you don't need to scrape HTML. Most companies' careers pages are hosted by an applicant tracking system (ATS), and the three most common ones with startups (Greenhouse, Lever and Ashby) all have public, unauthenticated JSON endpoints. The careers page itself calls them.

1. Find out which ATS a company uses

Open the careers page and click any job. The URL tells you:

URL contains	ATS	Slug
`boards.greenhouse.io/stripe` or `job-boards.greenhouse.io/stripe`	Greenhouse	`stripe`
`jobs.lever.co/palantir`	Lever	`palantir`
`jobs.ashbyhq.com/openai`	Ashby	`openai`

If the careers page lives on the company's own domain, view the source and search for those hostnames: the job list is usually embedded from one of them.

2. Call the endpoint

import requests

HEADERS = {"User-Agent": "my-job-tracker/1.0 (you@example.com)"}

def greenhouse(slug):
    r = requests.get(f"https://boards-api.greenhouse.io/v1/boards/{slug}/jobs",
                     params={"content": "true"}, headers=HEADERS, timeout=30)
    for j in r.json()["jobs"]:
        yield {"title": j["title"], "location": j["location"]["name"],
               "department": (j.get("departments") or [{}])[0].get("name"),
               "url": j["absolute_url"], "posted": j.get("first_published")}

def lever(slug):
    skip = 0
    while True:
        page = requests.get(f"https://api.lever.co/v0/postings/{slug}",
                            params={"mode": "json", "skip": skip, "limit": 100},
                            headers=HEADERS, timeout=30).json()
        for j in page:
            yield {"title": j["text"], "location": j["categories"].get("location"),
                   "department": j["categories"].get("team"), "url": j["hostedUrl"],
                   "remote": j.get("workplaceType") == "remote",
                   "salary": j.get("salaryRange")}
        if len(page) < 100:
            break
        skip += 100

def ashby(slug):
    r = requests.get(f"https://api.ashbyhq.com/posting-api/job-board/{slug}",
                     params={"includeCompensation": "true"}, headers=HEADERS, timeout=30)
    for j in r.json()["jobs"]:
        comp = (j.get("compensation") or {}).get("compensationTierSummary")
        yield {"title": j["title"], "location": j.get("location"),
               "department": j.get("department"), "url": j["jobUrl"],
               "remote": j.get("isRemote"), "salary": comp}

for job in ashby("openai"):
    print(job)

That's it: no API key, no headless browser, and no HTML parsing.

3. Salary ranges

Pay-transparency laws in New York, California, Colorado, Washington and soon the EU mean many postings now carry a salary range, and the APIs expose it:

Ashby: compensation.summaryComponents[] has minValue, maxValue, currencyCode and interval per component (salary, equity, bonus).
Lever: salaryRange with min, max, currency, interval (e.g. per-year-salary).
Greenhouse: not in the list response. Fetch each job with ?pay_transparency=true and read pay_input_ranges[] (min_cents, max_cents, currency_type).

def greenhouse_pay(slug, job_id):
    d = requests.get(f"https://boards-api.greenhouse.io/v1/boards/{slug}/jobs/{job_id}",
                     params={"pay_transparency": "true"}, headers=HEADERS, timeout=30).json()
    return [(p["min_cents"] / 100, p["max_cents"] / 100, p["currency_type"])
            for p in d.get("pay_input_ranges", [])]

4. Gotchas I hit

Greenhouse EU boards live on boards-api.eu.greenhouse.io.
Lever EU accounts use api.eu.lever.co.
Slugs are not company names. "Hugging Face" might be huggingface or hugging-face, and on some ATSs a generic slug like remote or figure belongs to a different company. Check the company name the API returns before trusting a guessed slug.
Be polite. These endpoints serve real careers pages. Cache, keep concurrency low and send a User-Agent with contact info.

Beyond these three

Enterprise employers use Workday (a POST API with a 20-item page limit and a total that only appears on page 1), and European companies often use Personio (XML), Recruitee, SmartRecruiters or Workable. I documented the endpoints for 11 ATSs, with their pagination quirks and salary fields, here: github.com/gleanery-works/ats-job-apis.

If you'd rather not maintain eleven integrations, I also run them as a hosted tool, Career Site Jobs Scraper. You give it company names, and it finds each company's ATS and returns one normalized schema with salary min/max. It can also return only jobs that are new since the last run, which is handy for hiring-signal alerts. It's pay per result ($2 per 1,000 jobs), and Apify's free monthly credit covers small projects.

Giving an AI agent access to job data

If the consumer is an LLM agent rather than your own code, you can skip the integration entirely and expose the tool over MCP:

claude mcp add --transport http jobs "https://mcp.apify.com?tools=gleanery_actors/career-site-jobs-scraper"

Then ask: "Which of Datadog, Snowflake and Figma opened data-engineering roles this week, and what do they pay?" The agent calls the tool with {"companies": [...], "titleKeywords": ["data engineer"], "postedWithinDays": 7} and gets normalized JSON back. Setup for Cursor, Claude Desktop and VS Code is in gleanery-works/job-and-supplier-data-mcp.

What are you building with job data? I'm curious which ATSs people still miss.

DEV Community