How to scrape jobs from any company career page (Greenhouse, Lever, Ashby, Workday)

Most tutorials on scraping job listings reach for Selenium, a headless browser, and a proxy pool. Then they break in a month when the HTML changes.

You almost never need any of that. Company career pages are single-page apps, and the JSON they load is fetched from a public API you can call directly. No login, no key, no browser.

Here are the endpoints for the five biggest applicant tracking systems, all verified working as I write this.

Greenhouse

The simplest of the lot. Everything you need is one GET request.

import requests

r = requests.get("https://boards-api.greenhouse.io/v1/boards/stripe/jobs?content=true")
for job in r.json()["jobs"]:
    print(job["title"], "|", job["location"]["name"])

stripe is the board token, which you can read off the company's careers URL (boards.greenhouse.io/stripe). Add ?content=true and you get the full job description as HTML in each record. Stripe alone returns over 500 jobs.

Lever

r = requests.get("https://api.lever.co/v0/postings/spotify?mode=json")
for job in r.json():
    print(job["text"], "|", job["categories"].get("location"))

Lever's payload is generous: descriptionPlain gives you clean text with no HTML stripping, and workplaceType tells you whether the role is remote.

Ashby

Ashby uses GraphQL, but it accepts unauthenticated queries for public job boards.

QUERY = """
query ApiJobBoardWithTeams($organizationHostedJobsPageName: String!) {
  jobBoard: jobBoardWithTeams(organizationHostedJobsPageName: $organizationHostedJobsPageName) {
    jobPostings { id title locationName employmentType }
  }
}
"""

r = requests.post(
    "https://jobs.ashbyhq.com/api/non-user-graphql?op=ApiJobBoardWithTeams",
    json={"operationName": "ApiJobBoardWithTeams",
          "variables": {"organizationHostedJobsPageName": "ramp"},
          "query": QUERY},
)
for job in r.json()["data"]["jobBoard"]["jobPostings"]:
    print(job["title"], "|", job["locationName"])

SmartRecruiters

Paginated, 100 per page, with a totalFound you can loop against.

r = requests.get("https://api.smartrecruiters.com/v1/companies/Visa/postings?limit=100&offset=0")
for job in r.json()["content"]:
    loc = job["location"]
    print(job["name"], "|", loc.get("city"), loc.get("country"))

Workday, and its three traps

Workday is where people give up. It is a POST, and the URL contains a tenant name, a datacenter number, and a site name that all differ per company.

r = requests.post(
    "https://nvidia.wd5.myworkdayjobs.com/wday/cxs/nvidia/NVIDIAExternalCareerSite/jobs",
    json={"appliedFacets": {}, "limit": 20, "offset": 0, "searchText": ""},
    headers={"Content-Type": "application/json", "Accept": "application/json"},
)
data = r.json()
print(data["total"], "total jobs")
for job in data["jobPostings"]:
    print(job["title"], "|", job["locationsText"])

Three things cost me time here, so let me save you the trouble:

1. limit cannot exceed 20. Ask for 100 and Workday returns an empty jobPostings array with no error. It looks exactly like "no more results."
2. It throttles fast paging. If you loop without a pause, a page eventually fails, and if your loop treats a failed page as the end of the list you will silently collect a fraction of the jobs. I lost 1,960 of NVIDIA's 2,000 jobs to this before I noticed. Pause between pages and retry a failed one instead of breaking.
3. The URL parts are not guessable. nvidia, wd5, and NVIDIAExternalCareerSite all come from the company's real careers URL. Copy it, don't invent it.

Some sites fingerprint your TLS handshake

Greenhouse, Lever, Ashby, SmartRecruiters and Workday are all happy to serve a plain requests call. Some other job sites are not: they inspect the TLS handshake itself and block anything that doesn't look like a real browser, no matter what User-Agent header you send.

When a site returns 403 to requests but loads fine in Chrome, that's the signal. curl_cffi impersonates a real Chrome TLS fingerprint:

from curl_cffi import requests as curl_requests

r = curl_requests.get("https://www.example-jobsite.com/jobs", impersonate="chrome")

That single change fixed two sites for me that no amount of header spoofing would.

Detecting which ATS a company uses

You don't need to know in advance. Fetch the careers page and look for the signature:

import re

SIGNATURES = [
    ("greenhouse",      r"boards\.greenhouse\.io/([\w-]+)"),
    ("lever",           r"jobs\.lever\.co/([\w-]+)"),
    ("ashby",           r"jobs\.ashbyhq\.com/([\w.-]+)"),
    ("smartrecruiters", r"careers\.smartrecruiters\.com/([\w-]+)"),
    ("workday",         r"([\w-]+)\.(wd\d+)\.myworkdayjobs\.com/([\w-]+)"),
]

def detect(careers_url):
    html = curl_requests.get(careers_url, impersonate="chrome").text
    for ats, pattern in SIGNATURES:
        m = re.search(pattern, html)
        if m:
            return ats, m.groups()
    return None, None

A word of warning from running this across 42 large employers: most big companies are not on a modern ATS at all. Taleo, Oracle Cloud, Phenom and iCIMS dominate enterprise, and none of them publish an open job API. My hit rate on the Gulf's most sought-after employers was 7 out of 42. Detect first, then decide whether scraping is even possible, rather than writing a parser for a page that will never give you clean data.

Validate your own output before you trust it

Two bugs that got past me, both from naive substring matching on job titles:

• "coo" in title.lower() classified "Coordinator" as a chief operating officer.
• "partner" in title.lower() promoted "Partner Onboarding Specialist" to firm partner.

Use word boundaries (re.search(r"\bcoo\b", title)), and before you publish any aggregate, print the raw titles behind each bucket. Every classifier you write will be wrong in a way that flatters your numbers.

The no-code version

If you'd rather not maintain five API clients, I packaged all of this as a free Apify Actor. It auto-detects the ATS, normalizes all five payload shapes into one schema, adds seniority and function detection, and returns clean JSON or CSV.

{
  "companies": [
    "greenhouse:stripe",
    "https://jobs.lever.co/spotify",
    "https://jobs.ashbyhq.com/ramp"
  ],
  "locationFilter": "Remote"
}

Run it free on Apify — it's free because these APIs need no proxies, so a run costs essentially nothing.

It also works as an MCP tool, so you can connect it to Claude or ChatGPT and just ask "what engineering roles are open at Stripe and Ramp?"

Be a good citizen

These endpoints are public because companies want their jobs discovered. Don't abuse that. Cache aggressively, pause between requests, and don't hammer a small company's board every minute. The reason this all works without keys is that nobody has had to lock it down yet.