Scraping Indeed or LinkedIn Jobs looks like the obvious way to watch who's hiring. In practice you fight Cloudflare challenges, logins, rate limits and HTML that changes every few weeks, and the listings you do get are often days old, reposted from somewhere else.
That somewhere else is usually the company's own careers page, and most careers pages are powered by an applicant tracking system (ATS) that publishes a clean, public JSON feed. Watching those feeds directly is faster, more reliable and fully public.
Step 1: find which ATS a company uses
Open the company's careers page and look at where the "Apply" links point:
| If links go to... | The ATS is | Public feed |
|---|---|---|
boards.greenhouse.io/acme or job-boards.greenhouse.io/acme
|
Greenhouse | https://boards-api.greenhouse.io/v1/boards/acme/jobs |
jobs.lever.co/acme |
Lever | https://api.lever.co/v0/postings/acme?mode=json |
jobs.ashbyhq.com/acme |
Ashby | https://api.ashbyhq.com/posting-api/job-board/acme |
apply.workable.com/acme |
Workable | https://apply.workable.com/api/v1/widget/accounts/acme |
jobs.smartrecruiters.com/Acme |
SmartRecruiters | https://api.smartrecruiters.com/v1/companies/Acme/postings |
The part after the domain (acme) is the board slug.
Step 2: poll the feed and keep only new jobs
import json, pathlib, requests
SEEN = pathlib.Path("seen.json")
seen = set(json.loads(SEEN.read_text())) if SEEN.exists() else set()
def greenhouse(slug):
r = requests.get(f"https://boards-api.greenhouse.io/v1/boards/{slug}/jobs", timeout=30)
for j in r.json()["jobs"]:
yield j["id"], j["title"], j["location"]["name"], j["absolute_url"]
new = []
for slug in ["stripe", "databricks", "notion"]:
for job_id, title, loc, url in greenhouse(slug):
key = f"gh:{slug}:{job_id}"
if key not in seen:
seen.add(key)
new.append((slug, title, loc, url))
SEEN.write_text(json.dumps(sorted(seen)))
for row in new:
print(" | ".join(row))
Run it from cron once a day and you have a "new openings" alert for any list of companies, with no browser, proxies or logins.
Step 3: scale past a handful of companies
The manual approach breaks down when you want hundreds of companies across several ATSs, because every ATS has a different schema and you first need to know each company's slug.
That's the problem we built ATS Jobs Search on Apify to solve. It covers seven ATSs (Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee, Personio), uses one output schema, has a built-in directory of about 7,900 companies' boards, and has an "only new jobs since last run" mode:
from apify_client import ApifyClient
client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("siftwright_data/ats-jobs-search").call(run_input={
"searchDirectory": True,
"keywords": ["data engineer", "analytics engineer"],
"locations": ["Remote"],
"onlyNewJobs": True,
})
for job in client.dataset(run["defaultDatasetId"]).iterate_items():
print(job["companyName"], "|", job["title"], "|", job["url"])
It costs $1 per 1,000 jobs, and Apify's free monthly credit covers a lot of testing.
Prefer a file or a ready-made feed?
- Everything at once: today's full snapshot (about 400,000 open jobs from 7,500+ companies, one CSV) is a $39 instant download.
- A daily niche feed (for example "remote data jobs" or "healthcare roles in Texas") delivered to a spreadsheet or your inbox: email siftwright.data@gmail.com and we'll quote it.
Happy hunting.
Top comments (0)