DEV Community

Dodo Data
Dodo Data

Posted on

5,966 job posts: who actually publishes pay, and who only looks like they don't

Every applicant-tracking system has a structured pay field. Greenhouse calls it pay_input_ranges, Ashby has compensation, Lever has salaryRange. If you are building anything on job data, that field is where you look for salary, and it is where most pipelines stop.

I pulled 24 public career sites, 5,966 open jobs, and counted. Then I read the descriptions of the boards that came back empty, and the number moved by 445 jobs.

The structured field

One request per board, no browser, no login. Greenhouse at boards-api.greenhouse.io/v1/boards/<board>/jobs, Ashby at api.ashbyhq.com/posting-api/job-board/<org>?includeCompensation=true, Lever at api.lever.co/v0/postings/<org>?mode=json.

System Boards Jobs With structured pay
Ashby 8 1,293 72.8%
Greenhouse 12 3,435 56.2%
Lever 4 1,238 3.2%

Lever's 3.2% is the first thing worth noticing. It is not that Lever companies pay in secret. salaryRange exists on the Lever posting API, and almost nobody fills it in. Three of the four boards I pulled had it empty on every single job.

Per company, the spread inside one system is wider than the gap between systems:

Board System Jobs Structured pay
Duolingo Greenhouse 81 100.0%
Coinbase Greenhouse 217 98.2%
Ramp Ashby 148 95.3%
Anthropic Greenhouse 611 89.4%
OpenAI Ashby 816 82.4%
Databricks Greenhouse 874 54.1%
Stripe Greenhouse 665 0.0%
Palantir Lever 313 0.0%
Notion Ashby 128 0.0%
Discord Greenhouse 47 0.0%

The zeroes are not all the same zero

Four of those 0.0% boards state a pay range in plain English in the job description. They just never filled in the field.

I ran a range extractor over the descriptions of every board that reported zero:

Board Structured Found in description text
Discord 0 of 47 44
Palantir 0 of 313 200
Notion 0 of 128 79
Affirm 0 of 199 121
Stripe 0 of 665 1
Spotify 0 of 71 0
Linear 0 of 32 0

Discord publishes pay on 94% of its jobs. Any dataset built on the structured field alone reports 0%.

That is the finding: an empty pay field is a data-entry fact, not a pay-transparency fact. Stripe and Spotify genuinely do not publish ranges. Discord, Palantir, Notion and Affirm do, in a sentence near the bottom of the post that no API returns as a number.

Across the whole sample, the structured field says 48.8% of jobs publish pay. Reading the text as well takes it to 56.3%, and the gain is concentrated entirely in the boards that looked silent.

Extracting a range from prose, without inventing numbers

The naive version of this is a regex for two dollar amounts with a dash between them, and it is wrong within about ten jobs. Here is what actually breaks.

Equity tables look exactly like salary ranges. Affirm puts this in most of its posts:

Equity Grade - 2 (awarded in US Dollars)
Equity grade: 2   New hire equity: $16,000-$24,000   Annual Refresh: $5,000
Enter fullscreen mode Exit fullscreen mode

$16,000-$24,000 is a real range next to a real dollar sign, and it is not the salary. A blocklist on the preceding text is what saves you: equity, refresh, bonus, sign-on, 401, stipend, RSU, grant, option.

On-target earnings are not base pay. Sales roles quote OTE, which is base plus commission at 100% attainment. Rolling it into a salary_min column silently inflates every sales row in the dataset. I let those go unmatched on purpose, which is most of what the extractor misses on Affirm.

A number needs a cue word in front of it, not just a currency symbol. Requiring one of base pay, pay range, salary range, compensation range, hourly rate and so on within the preceding 90 characters removes nearly all of the false positives without costing real matches.

Hourly and annual are the same shape. $22 - $30 and $220,000 - $300,000 differ by three zeros. Decide the period explicitly from the surrounding words, and refuse to guess when there is neither an hourly nor an annual cue and the top of the range is under $10,000.

The whole thing is about thirty lines:

const MONEY = String.raw`(?:\$|USD\s?|£|€)\s?(\d{1,3}(?:,\d{3})+|\d{2,3}(?:\.\d+)?\s?[kK]\b|\d{2,6}(?:\.\d{2})?)`;
const RANGE = new RegExp(`${MONEY}\\s*(?:-|to|and)\\s*${MONEY}`, 'g');
const PAY_CUE = /(base pay|pay range|base salary|salary range|compensation range|annual salary|hourly rate|pay rate|salary of|compensation of|remuneration)/i;
const NOT_PAY = /(equity|refresh|bonus|sign[- ]?on|401|stipend|budget|revenue|funding|raise[ds]|valuation|grant|rsu|option)/i;

for (const m of flat.matchAll(RANGE)) {
    const before = flat.slice(Math.max(0, m.index - 90), m.index);
    if (!PAY_CUE.test(before) || NOT_PAY.test(before)) continue;
    // ...amounts, currency from the symbol, period from hourly/annual cues
}
Enter fullscreen mode Exit fullscreen mode

It does not handle 35 000 €, the French spacing convention, and I left it that way rather than half-supporting it. A French board returns null, which is honest, instead of a number pulled out of a format the parser does not really read.

Keep the two sources apart

If you merge text-extracted ranges into the same column as the structured field and say nothing, you have made your dataset less trustworthy, not more. One came from a form the employer filled in. The other came from a regex over prose, and it can be wrong.

Carry a third column. Mine is salary_source, one of structured, description, or null, and a run over Affirm and Duolingo looks like this:

200 rows   structured: 81   description: 94   none: 25
  duolingo   structured: 81
  affirm     description: 94   none: 25
  e.g. Administrative Assistant IV        115,000 - 165,000 USD
  e.g. Affirm Bank Strategic Finance Mgr  185,000 - 245,000 USD
Enter fullscreen mode Exit fullscreen mode

Anyone doing serious analysis can filter to structured only. Anyone building a job board gets a pay range on four times as many Affirm roles. Nobody gets a number without knowing where it came from.

Use it

The extractor ships in the Dodo Data job scrapers on Apify Store, across Greenhouse, Lever, Ashby, Workday and the multi-system one. Same 26-field row shape, plus salary_source, $1 per 1,000 jobs:

https://apify.com/dododata/career-site-jobs-scraper

If you want to re-run the survey yourself, it is one request per board and the endpoints are all listed above. The interesting number is not the average. It is which of your target employers are in the "publishes pay, never filled in the field" bucket, because that one is invisible until you read the text.

Top comments (0)