DEV Community

spooky bat
spooky bat

Posted on

How to get job postings straight from company career pages (without scraping LinkedIn)

#webscraping #javascript #api #career

Job boards are late, full of duplicates and reposts, and most of them forbid scraping. But almost every tech company publishes its openings in the same handful of places: the career page that its applicant tracking system (ATS) hosts. Greenhouse, Lever, Ashby, Personio and others all expose a public job feed for exactly this purpose — so companies can embed their openings on their own website.

That means you can build a clean, legal job dataset from the source. Here is how.

1. Find which ATS a company uses

Open any job on the company's career page and look at where "Apply" leads:

ATS Career page looks like Public feed
Greenhouse boards.greenhouse.io/acme https://boards-api.greenhouse.io/v1/boards/acme/jobs?content=true
Lever jobs.lever.co/acme https://api.lever.co/v0/postings/acme?mode=json
Ashby jobs.ashbyhq.com/acme https://api.ashbyhq.com/posting-api/job-board/acme?includeCompensation=true
Personio acme.jobs.personio.de https://acme.jobs.personio.de/xml?language=en

acme is the company's board id — usually the company name in lowercase.

2. Fetch the feed 

// Node 18+: no dependencies
const res = await fetch('https://boards-api.greenhouse.io/v1/boards/doctolib/jobs?content=true');
const { jobs } = await res.json();
for (const j of jobs.slice(0, 5)) {
  console.log(j.title, '|', j.location?.name, '|', j.absolute_url);
}
Enter fullscreen mode Exit fullscreen mode

Lever returns a plain array of postings, Ashby returns { jobs: [...] } with optional compensation, Personio returns XML. Each has its own field names for location, department and employment type.

3. Things that bite you in production

  • Normalising ten different schemas into one row (title, department, location, country, remote/hybrid, salary, posted date, apply link).
  • Locations are free text: "Berlin, Berlin, Germany", "Remote - EU", "London or Remote".
  • Only new jobs: you need to remember what you already saw, per company, between runs.
  • Being a good citizen: check robots.txt, back off on 429, stop when a site shows a bot challenge, and don't collect personal data (recruiter e-mails and phone numbers often sit inside descriptions).
  • Discovery: the hard part isn't fetching one company — it's knowing which companies to fetch.

4. Or skip the plumbing

I packaged all of the above into an Apify Actor, Career Page Jobs API. It covers 10 ATS (Greenhouse, Lever, Ashby, Personio, Teamtailor, Recruitee, Workable, Workday, BambooHR, Pinpoint), and has two modes:

  • Search without a company list: keywords + location over a built-in list of ~550 verified companies — "software engineer, Germany" returns 118 jobs from 49 companies (German titles like "Softwareentwickler (m/w/d)" included).
  • Track your own list: paste career pages, turn on only new jobs, schedule it daily → a job alert for any set of companies, delivered to Sheets, Slack or your own API.

It costs $1 per 1,000 jobs (you pay only for jobs delivered), and the free Apify credit is enough to try it.

Disclosure: I'm the developer of this Actor. The feeds in sections 1–2 are public and free to use directly — the Actor is just the shortcut if you don't want to maintain ten parsers.

Top comments (0)