DEV Community

Dodo Data
Dodo Data

Posted on Edited on

Every open job at 500 companies, as JSON, without scraping a single job board

Most "job data" projects start by scraping LinkedIn or Indeed, and most of them die there: captchas, logins, lawsuits, and listings that are already three reposts away from the employer.

There is a quieter source. Most tech companies run their career page on an applicant-tracking system: Greenhouse, Lever, Ashby, SmartRecruiters, Recruitee, Workable, Personio, Breezy, Pinpoint, Rippling, Teamtailor. Every one of those systems publishes a job feed so the career page can be embedded on the company's site. First-party, current, no login, no anti-bot.

The feeds

Greenhouse       https://boards-api.greenhouse.io/v1/boards/{company}/jobs?content=true&pay_transparency=true
Lever            https://api.lever.co/v0/postings/{company}?mode=json
Ashby            https://api.ashbyhq.com/posting-api/job-board/{company}?includeCompensation=true
SmartRecruiters  https://api.smartrecruiters.com/v1/companies/{company}/postings
Recruitee        https://{company}.recruitee.com/api/offers/
Workable         https://apply.workable.com/api/v1/widget/accounts/{company}?details=true
Personio         https://{company}.jobs.personio.de/xml
Breezy HR        https://{company}.breezy.hr/json
Pinpoint         https://{company}.pinpointhq.com/postings.json
Rippling         https://api.rippling.com/platform/api/ats/v1/board/{company}/jobs
Teamtailor       https://{career-site-host}/jobs.rss
Enter fullscreen mode Exit fullscreen mode

{company} is the slug in the career-page URL: jobs.lever.co/spotify gives spotify.

The catch: eleven shapes

The feeds agree on nothing. Greenhouse sends the description as HTML that has been entity-escaped once (<p>), pay ranges in cents, and the location as { "name": ... }. Lever sends createdAt as a millisecond string and splits the description across descriptionPlain, lists[] and additionalPlain. Ashby nests compensation three levels deep. SmartRecruiters paginates and needs one more request per posting for the text. Personio is XML, Teamtailor is RSS with its own namespace, and Rippling's feed has no descriptions at all.

Un-escaping Greenhouse, for instance, is a two-step:

import { load } from 'cheerio';

export function htmlToText(html: unknown): string | null {
    if (!html) return null;
    let h = String(html);
    if (!h.includes('<') && h.includes('&lt;')) h = load(`<x>${h}</x>`)('x').text(); // un-escape once
    h = h.replace(/<\s*br\s*\/?>/gi, '\n').replace(/<\/(p|div|li|h[1-6]|ul|ol|tr)>/gi, '\n').replace(/<li[^>]*>/gi, '');
    return load(`<x>${h}</x>`)('x').text().trim() || null;
}
Enter fullscreen mode Exit fullscreen mode

(Workday, the twelfth system, has no feed. Its career sites are allowed by robots.txt and every job page carries schema.org JobPosting as JSON-LD, so it is read the old-fashioned way: listing, then page.)

One shape out

I normalised all twelve into 26 fields: title, department, team, locations[], workplace_type (remote / hybrid / onsite), salary_min, salary_max, salary_currency, posted_at, apply_url, plain-text description, and a stable id (ats:company:job) so a daily run can be diffed in one line.

A real run on 2026-09-17, three boards (Duolingo on Greenhouse, Spotify on Lever, Ramp on Ashby): 304 jobs in 18 seconds, 224 of them (74%) with a published pay range already parsed to numbers.

Two things I deliberately leave out: recruiter names and mailboxes that some feeds carry, and any e-mail or phone number inside a description (replaced with [email removed]). It is job data, not people data.

Use it

If you would rather not maintain twelve parsers, it is on Apify Store as Career Site Jobs Scraper. Paste career-page URLs, filter by title keywords, location or "posted within N days", $1 per 1,000 jobs, callable from the API or from an AI agent over MCP:

https://apify.com/dododata/career-site-jobs-scraper

If a feed changes shape, it is fixed within 48 hours, and that promise is on the listing.

Top comments (0)