DEV Community: Radar Labs

One company, one 75,500 dollar gap between role families

Radar Labs — Sun, 25 Oct 2026 23:18:47 +0000

The company is one company. The pay is not one number.

I keep a public dataset of job postings that actually state a salary range — right now 8,049 of them, deduplicated, from 266 employers' own hiring systems. Same dataset, one question: inside a single company, how far apart are the role families?

The answer is $75,500 of median difference between the top and bottom of the range table.

Data and ML: $237,500 median. Support and operations: $162,000.

Everything else sorts in between — engineering, product, sales, marketing, finance, design, people/recruiting, legal, security.

Why this gap is worth sitting with

Three things make it more than trivia.

It is bigger than the level gap. I also grouped these postings by job-title seniority. Titles containing "senior" or "staff" have a median of $185,791; titles with no level word at all sit at $193,500. Seniority words, as written in job ads, barely move the number — often they do not even point the right way. Role family moves it by $75,500. If you are optimising your career for pay, the discipline you pick does more work than the adjective in front of your title.

It is a range, not a promise. These are posted ranges: base salary bands the employer publishes to comply with pay-transparency law. They are not total compensation. They exclude equity, bonus, and everything else that quietly decides whether two people with the same title actually earn the same money. A $75,500 posted gap may be smaller or larger once equity is counted — the dataset cannot tell you, and any article claiming otherwise is making it up.

It compounds. A gap that size, held over a decade of raises computed as percentages of base, is not a one-time difference. It is the thing that decides whether you can buy a house.

Method, so you can distrust it properly

Source: employers' own ATS job boards (Greenhouse, Ashby, Lever, Workable, Recruitee and friends), fetched directly — not scraped from aggregators that republish stale listings.
Filter: only postings that publish an explicit salary range. That is the hard constraint, and it is why the dataset is 8,049 and not 19,000.
Deduplication across boards, currency normalisation, and a median computed per role family rather than an average of averages.
Known holes: US-heavy, base-salary-only, no equity, no cost-of-living adjustment, and role-family labels come from title keywords, so border cases (a "Solutions Engineer" in support? in sales?) are a judgement call.

What to do with it

If you are choosing between two offers at the same company, ask for the band of the role family you would actually be in, not the band of the title on the offer letter. If you are negotiating, the role-family median is a better anchor than the seniority word, because it is the number the company's own compensation team uses.

The dataset is open: salary-ranges-full.csv, one row per posting with employer, title, location, and the posted range. The role-family breakdown, with sample sizes printed on the page, is at the role pay page. Take it, quote it, argue with it — that is what it is for.

Remote work is not one pay discount: what 8,049 published salary ranges show

Radar Labs — Sun, 25 Oct 2026 23:08:14 +0000

"Remote work pays less" is the most repeated salary claim on the internet, and the data behind it is usually a single number that averages over a dozen different markets.

We collect published salary ranges from companies' own job board APIs (Greenhouse, Ashby, Lever, Workable) and normalise them to annual USD. The current snapshot has 8,049 postings that actually state a range, from 197 companies. Only postings with a stated range are included — nothing is imputed — and every figure below carries its sample size, because a median over 88 postings is not a median over 5,795.

Snapshot date: 2026-10-26.

The headline number, and why it misleads

	Postings	Median midpoint	p25	p75
Remote	2,254	$190,781	$145,317	$235,250
On-site	5,795	$204,000	$150,000	$261,250

Remote roles are 6.5% below office roles at the median. That is the number most people quote, and it is a blend of functions where remote workers are paid more and functions where they are paid a lot less.

Split by function, the gap inverts

Role family	Remote n	Remote median	Office n	Office median	Difference
Security	100	$216,902	231	$197,500	+9.8%
Sales	340	$175,000	899	$159,950	+9.4%
Product	166	$222,465	382	$225,425	−1.3%
Marketing	137	$180,000	382	$193,675	−7.1%
Engineering	561	$212,000	1,640	$226,000	−6.2%
Support / ops	385	$153,318	830	$168,301	−8.9%
Finance / legal	183	$181,022	548	$200,975	−9.9%
Data / ML	88	$206,000	278	$250,000	−17.6%

Two families pay a remote premium: security (+9.8%) and sales (+9.4%). Both are functions where the candidate pool is thin and geography matters least — an enterprise seller or a security engineer can be sourced anywhere, so companies compete on the number rather than the office.

Data and ML shows the steepest discount (−17.6%). Those postings cluster in a few expensive metros, and the remote-labelled version of the same role is often a level below the on-site one. That is not remote work being punished; it is two different levels carrying one label.

The rest of the dataset says the same thing about labels

Two other results from the same snapshot, both of which look like bugs and are not:

Postings whose title contains "senior" have a lower median ($185,791) than postings with no level word at all ($193,500). Unlabelled titles are full of "Member of Technical Staff" and "Forward Deployed Engineer" at labs where no adjective means a senior band by default.
Postings that list several cities have a median of $345,000 against $195,000 for single-location postings. Multi-metro listings are a marker of roles the company has struggled to fill, not a flexibility bonus.

How to use this

If you are negotiating, benchmark against your role family, not the headline. "Remote pays 6.5% less" is true of the dataset average and false for security and sales in this snapshot. And ask which level the band was written for — that explains more variance than the remote flag does.

Method, sample sizes and the raw CSV are public:

Deep dive with the full tables: https://sjdfi0wjvosdju9.github.io/topics/remote-pay.html
Dataset (CC BY 4.0, no signup): https://sjdfi0wjvosdju9.github.io/salary-data.html
What changed and when: https://sjdfi0wjvosdju9.github.io/changes.html

If you want a specific cut — a role family, a city, a company — say so in the comments and I will publish it from the next refresh.

I counted 1,726 job postings as 'other'. My classifier was lying.

Radar Labs — Sat, 24 Oct 2026 02:46:16 +0000

I counted 1,726 job postings as "other". My classifier was lying, and it moved a headline number.

I publish a free dataset of job postings where the employer states a pay range. To make it more than a CSV, I group every posting into a role family — engineering, sales, product, design, support, and so on — so readers can ask "what does this company pay a data scientist" instead of "what is the average of everything".

Last week the distribution looked wrong in a way that was easy to wave away:

family	postings	share
engineering	3,188	39.7%
other	1,726	21.5%
sales	868	10.8%
support / ops	510	6.3%
data / ml	488	6.1%

One fifth of the corpus landing in "other" is not a labelling preference. It means the classifier did not know, and every per-family median I published was computed on a corpus with a hole in it.

What was actually wrong

The function was one flat list of rules, evaluated in order, with engineering first:

RULES = [
    ("engineering", ("engineer", "developer", "swe", ...)),
    ("data / ml",   ("data", "ml", "machine learning", ...)),
    ("sales",       ("sales", "account executive", ...)),
    ...
]

Two things follow from that shape, and both are silent:

engineer wins before anything else can be considered. Sales Engineer → engineering. Solutions Engineer → engineering. Support Engineer → engineering. Three different jobs, one bucket, no error message.
Short, common substrings swallow rare ones. data is inside Data Center Technician, so a rack-and-stack role became a data scientist.

And other was not "jobs we could not classify". It was jobs whose title did not contain any of my keywords — technical program managers, executive assistants, people partners, compliance, fraud. A bucket of 1,726 postings that all mean something different, sitting in the same column.

The fix is boring, and that is the point

Four passes instead of one, ordered by specificity:

disambiguation — combinations that a single keyword gets wrong: technical account manager and customer success manager are support, not sales.
specific rules — multi-word, unambiguous: account executive, data scientist, platform engineer.
generic rules — single strong words: engineer, designer.
broad rules — weak signals, tried last: manager, lead.

Ordering is the whole mechanism. engineer only runs after the specific pass has had its chance.

The test caught me, immediately

I wrote 21 assertions before trusting the change, and the third one failed:

AssertionError: 'sales' != 'support / ops'

Technical Account Manager was still landing in sales, because account manager sits in the sales rules and runs first. That is the same class of bug as the original one, one layer down. It is the reason the disambiguation pass exists at all — not because I thought of it, but because a test I wrote to describe my intent disagreed with my code.

What changed in the numbers

Same CSV, only the classifier re-run:

family	before	after
engineering	3,188 (39.7%)	2,200 (27.4%)
other	1,726 (21.5%)	680 (8.5%)
sales	868	1,238
support / ops	510	1,219
finance / legal	268	728
product	319	547

The other bucket did not shrink because I invented categories for it. It shrank because 1,200 postings had been sorted into the wrong family all along — mostly sales engineers, account managers, TPMs and support staff.

The part I did not enjoy: a published claim moved

Three weeks ago I published that, within the same role family, remote roles advertise about 11.5% less pay than on-site ones. That number was computed on top of the broken classifier, with sales engineers counted as engineers.

Re-run on the corrected families, it is −6.2% (remote $212,000 vs on-site $226,000, n=563 vs 1,638).

The direction survived; the size did not. I appended a dated revision note to the original article and wrote the change into the public change log, because a dataset site whose numbers silently drift is worse than one that never published them.

Three checks I now run before publishing any derived column

Does the residual bucket have a story? If 20% of rows land in other, that is not a category, it is a bug list.
Can two rules both claim the same row? If yes, ordering is load-bearing, and ordering needs a test.
Does a change in this column move a headline number? If it can, the change needs a dated note on the article that used it.

The dataset is free and re-generated weekly: salary-data.html, every number that moved is at changes.html, and the code that does the classification is in the same repo.

If you spot a title in the other bucket that clearly belongs somewhere, tell me which family — that is a one-line fix and a test case, and it is exactly the kind of thing that is easier from outside than from inside.

11 companies with 40+ open roles publish a salary on none of them

Radar Labs — Sat, 24 Oct 2026 00:25:25 +0000

11 companies with 40+ open roles publish a salary on none of them

I measured pay transparency the way it is usually measured wrong.

Most "salary transparency" numbers you see are computed over postings that have a salary. That tells you nothing about how often companies disclose, because the ones that disclose nothing never enter the sample. So I built the denominator first: read every posting from the public job feeds of 266 companies (18,668 after de-duplication), then counted how many of those carry a pay range.

The result splits two ways.

Same board, opposite behaviour

Board-level rates hide the interesting part. Here is the range within Ashby and Greenhouse, restricted to companies with at least 25 de-duplicated postings:

Company	Board	With pay / postings	Rate
sentry	ashby	42/42	100%
rain	ashby	37/37	100%
algolia	greenhouse	28/28	100%
klaviyo	greenhouse	150/154	97.4%
ramp	ashby	160/167	95.8%
plaid	ashby	129/135	95.6%

And the other end, still publishing something:

Company	Board	With pay / postings	Rate
supabase	ashby	1/59	1.7%
nubank	ashby	1/97	1.0%
sumup	greenhouse	3/344	0.9%
elevenlabs	ashby	1/161	0.6%
gopuff	lever	1/784	0.1%

Both lists contain companies that are healthy, well-funded and hiring hard. They simply made different decisions about the same field.

Eleven companies publish nothing at all

40+ open postings, zero pay ranges, and this is not a scraping artefact — the field is absent from the feed, which I verified by dumping the raw JSON keys:

Company	Board	Postings
datadog	greenhouse	418
betterhelp	greenhouse	154
notion	ashby	140
langchain	ashby	104
spotify	lever	89
getyourguide	greenhouse	74
trading212	ashby	53
mollie	ashby	51

A board can be perfectly transparent and a company on it can be perfectly opaque. Board-level rates are a property of the company mix, not of the board. That is the single most useful correction to come out of this dataset.

Why the denominator discipline matters

Two numbers from the same crawl:

Postings that state a range: 8,039
Postings read: 18,668

If you compute an average over the first number, you are describing the subset that already chose to be transparent. Any comparison built that way — "remote pays 12% less", "engineers at company X earn Y" — inherits the selection. The honest version pins the denominator to everything read, which is why every rate in this post is with pay / all postings and never with pay / 8,039.

The two failure modes I hit while building this, both of which produce plausible-looking numbers:

Mixing bases. I first computed rates as CSV rows / board postings, where the numerator was de-duplicated and the denominator was not. One company came out at 105.9% — a value that cannot exist, and which is the only reason I caught it. Same base on both sides, always.
Unit words anywhere in the text. A job description that mentions "hourly" anywhere turned a $210,000 salary into $436,800,000. I now only read the 40 characters after the last number for a per-hour/per-month hint.

The data

The dataset is free and CC BY 4.0: 8,039 postings that state a pay range, from 266 companies across Greenhouse, Ashby, Lever and Workable, normalised to annual USD, with the original local-currency range kept alongside.

If you want the same rows as JSON with salary filters and an incremental "only jobs I have not seen" mode, that runs as a hosted Actor at $0.0015 per row. The CSV above is the free version and always will be.

If you take one thing from this: before you trust any salary statistic, ask what the denominator was. In pay transparency, the denominator is the whole story.

My site answered 200 on all 163 URLs. Crawlers never saw a single page.

Radar Labs — Sat, 24 Oct 2026 00:10:04 +0000

My site answered 200 on all 163 URLs. Crawlers never saw a single page.

For two months I ran a small data site: 163 URLs, 152 generated company pages, a 1.7 MB CSV, valid sitemap.xml, and an IndexNow submission that returned 200 every single week.

Every check I had was green. And every search engine had zero of my pages.

This is the story of what was wrong, because I think it is a common failure and the symptom looks exactly like success.

The clue was one request away

On a hunch I fetched the same file twice:

curl -s https://example.site/robots.txt | wc -c
# 26

curl -s "https://example.site/robots.txt?probe=1" | wc -c
# 162

Two different files from the same path. The 26-byte one was:

User-agent: *
Disallow: /

The 162-byte one was mine, the one I had deployed:

User-agent: *
Allow: /
Sitemap: https://example.site/sitemap.xml

The host was rewriting the bare path. ?probe=1 bypassed whatever was doing it, so every "is my robots.txt right?" check I had ever run against the bare URL was reading the host's stub, not my file. Crawlers read the bare path. So for two months I was politely asking every crawler in the world to go away, while my dashboard said the site was up.

Three checks confirmed it was the platform, not a cache:

The injected response had no charset, no Cache-Control, no ETag; mine had all three.
Deploying the identical build to a brand-new subdomain produced the same 26 bytes — so it was not a stale CDN object.
The response headers of the injected copy came from the platform's edge, not from my deployment.

The second bug, in the same audit

While auditing I submitted my sitemap to IndexNow and got:

HTTP 422 {"errorCode":"InvalidRequestParameters",
          "message":"One or more URLs are not related to your verified domain."}

Seven URLs in a 163-URL sitemap still pointed at the old hostname. The sitemap generator only rewrote the section it owned and left a hand-written header block from weeks earlier intact.

And here is the embarrassing part: my own residue check had already reported "0 occurrences". I had written:

Select-String -Path site\*.html -Pattern 'old-host\.com' -SimpleMatch

-SimpleMatch treats the pattern as a literal string — backslashes included. So I was searching for old-host\.com with an actual backslash in it, which matches nothing, ever. The check was a no-op wearing a green badge.

What I changed

The real fix was not the migration. It was making the failure impossible to miss. Every deploy now runs a gate before publishing:

# robots.txt must not block the site, and the bare path must match ?probe=1
if re.search(r"(?im)^\s*disallow:\s*/\s*$", robots):   problems.append("robots.txt blocks the whole site")
if not re.search(r"(?im)^\s*allow:\s*/\s*$", robots):   problems.append("no site-wide Allow: /")
_, fresh = fetch(robots_url + "?probe=1")
if fresh != raw:                                        problems.append("bare URL differs from ?probe=1 (crawlers read the stale one)")

# sitemap must be single-host, and sampled pages must not be noindex
hosts = {u.split("/")[2] for u in locs if "://" in u}
if hosts != {site_host}:                                problems.append(f"foreign host in sitemap: {hosts}")

That gate is step 8 of 10 in the weekly pipeline now, and it is allowed to fail the run.

I moved to a host whose robots.txt I actually control end to end (a GitHub Pages user site, served at the domain root — a project site under /repo-name/ would have broken every root-absolute link I have). Same bytes, different owner:

$ curl -s https://my-site/robots.txt
User-agent: *
Allow: /
Sitemap: https://my-site/sitemap.xml

Four things I would tell my past self

1. 200 OK is not a statement about crawlability. It is a statement about HTTP. Everything that decides whether a page enters an index — robots.txt, noindex, canonical tags, sitemap host consistency, whether the CDN serves a different body on the bare path — lives beside the status code, not in it.

2. Verify what machines read, not what your browser showed you. A browser hides this class of bug completely, because you looked at ?probe=1 when you debugged, and the crawler looks at /.

3. Compare the bare URL with a query-string URL. It costs one request and catches platform-injected responses, stale edge objects, and rewrite rules. I now do it on robots.txt on every deploy.

4. A dashboard that is green in every cell can hide a total failure. My "163/163 URLs 200" metric was true and useless. The metric that would have caught this is "does the file a crawler fetches contain what I deployed".

The upside of finding it

The audit also forced two fixes I would not have made otherwise: a sitemap that can never drift to a foreign host, and a robots.txt with no BOM (a UTF-8 BOM in robots.txt silently breaks the first directive, which is its own quiet version of this bug).

The dataset I was trying to get indexed is public: 8,040 job postings that state a salary, from 266 companies, normalised to annual USD — dataset page, raw CSV on GitHub, CC BY 4.0.

If you are running a content site and your traffic is zero while every check is green, spend one request on robots.txt?probe=1. It is the cheapest test in this post and it took me two months to run it.

1,170 job postings looked like they had no salary. My currency table was deleting them.

Radar Labs — Fri, 23 Oct 2026 21:47:01 +0000

My dashboard had a line I almost shipped as a footnote: 0.1% of salary rows can't be converted to USD.

That reads like a rounding error. It wasn't. It was my filter deleting 1,170 real job postings.

The setup

I run ATS Job Feed, which pulls open roles from Greenhouse, Lever, Ashby, Workable, Personio and Workday, normalises pay to yearly USD, and de-duplicates across boards. One of the switches, salaryOnly, drops postings that don't state a number — people who only care about jobs with published pay.

Under the hood the filter is one line:

if salary_only and not (job.get("salaryLow") or job.get("salaryHigh")):
    continue

And the conversion was a dict:

TO_USD = {"USD": 1.0, "EUR": 1.08, "GBP": 1.27, "CNY": 0.14, ...}  # ten currencies

Anything not in that dict returned None. Which meant: a posting quoted in PLN, AED, MXN, RON, BRL or MYR got salaryUsd* = None, and salaryOnly then treated it as "no salary published" and dropped it.

The posting was real. The pay was real. The number was on the page. My pipeline just couldn't convert it, and the filter read "can't convert" as "nothing to see".

The failure mode worth remembering

A missing conversion is indistinguishable from missing data.

Once a derived field feeds a filter, every failure of the derivation becomes a confident "no" instead of an honest "unknown". Nobody logs it, because None is a legal value. My coverage stat even printed it as a tiny percentage instead of an error, because I'd written the stat to measure "rows without USD" — which is exactly the symptom, not the cause.

If I had shipped that footnote, a customer filtering to salaried-only roles would have silently lost 1,170 postings in six currencies and had no way to notice. That's the bad kind of bug: not a crash, a quiet omission.

The fix

Fetch a dated rate table once per run. open.er-api.com needs no key, covers 166 currencies, and returns its own update timestamp.
Cache it for 24 hours so 259 company fetches don't become 259 HTTP calls, and a network blip doesn't take out the run — on failure it falls back to a static table and logs a warning.
Invert honestly. The API says "how much local currency per 1 USD". I need "what do I multiply by". That's 1 / rate, and it deserves its own test.
Never guess. Unknown currency → null, not 1.0, not the previous row's rate.
Publish the provenance. Every run reports fxDate in its summary: Fri, 23 Oct 2026 00:02:31 +0000. A converted number you can't trace to a date is a rumour.
Keep the original numbers. Output carries salaryCurrency plus the local low/high, so a GBP posting shows both 375,000–640,000 GBP and 502,236–857,149 USD.

What it actually moved

	before	after
rows with a USD midpoint	7,994	8,004
rows where USD midpoint is missing	10	0
non-USD rows that convert	partial	777
`salaryOnly` pass-through, 259-company pool	6,837	8,005 (+17%)

Sample: anthropic, GBP 375,000–640,000 → USD 502,236–857,149. Currency mix across the paid rows: USD 7,227 / EUR 405 / CAD 169 / GBP 155 / SGD 10 / MXN 9 / AUD 6 / INR 6 / AED 5 / CNY 4 / BRL 3 / PLN 2 / RON 2 / MYR 1.

Three checks you can copy

Split "unknown" from "no". If a derived field gates a filter, log derived is None separately from field is absent. They look identical downstream and mean opposite things.
Make derivations self-describing. Rate date, timezone, source version — print it in the artefact, not just in a log line you'll never read.
Test the unknown path. One test per branch: static rate works, live rate overrides it, unknown currency stays null. That third test is the one that keeps the bug from coming back.

The feed is on Apify — $0.0015 per de-duplicated posting with a normalised annual salary, no charge for rows that get filtered out. The same data is free as a CSV (8,004 rows, CC BY 4.0), and you can look up a single company on this page.

If you're building something similar and you've got a filter sitting downstream of a conversion step — go count the Nones. They may not be as small as they look.

I was deleting 17% of real job postings with one line of dedupe

Radar Labs — Fri, 23 Oct 2026 21:11:35 +0000

A one-line dedupe rule was quietly deleting one in six real job postings from my dataset. Here is how it showed up, and the field I forgot to put in the key.

The symptom was a number I was proud of

After cleaning up a date bug, I looked at my pipeline's own funnel:

18,833 postings scraped  →  15,548 after dedupe

A 17.4% duplicate rate. I had been treating that as proof the sources were messy and my dedupe was doing heroic work. It is not. On real job-board data, cross-source duplicates sit around 1–3%. Anything much above that is usually not the data being dirty — it is your key being too coarse.

So I stopped celebrating and measured what was actually colliding.

Measuring the collision

I took seven companies (stripe, gitlab, anthropic, figma, brex, ramp, openai), 2,835 postings, grouped them by my dedupe key, and then asked a much more useful question: inside each collision group, how many distinct locations are there?

The key was:

key = (job["company"].lower(), title_key(job["title"]))

The answer:

194 collision groups
188 of them (97%) contained more than one location
353 rows merged away, 347 of which were distinct jobs

Real examples, straight from the run:

Company	Title	Locations collapsed into one row
stripe	abuse investigator	`Dublin` and `Seattle, San Francisco, New York City`
stripe	account executive commercial hunter	`Chicago` and `Singapore`
stripe	account executive platforms hunter	`Chicago`, `Chicago, Illinois`, `London`

Those are not duplicates. They are separate requisitions, with separate links and usually separate pay ranges. My "clean" dataset was throwing them away.

The fix is one field

def location_key(location):
    text = clean_text(location or "").lower()
    text = re.sub(r"[,/|;]+", " ", text)
    text = re.sub(r"[-–—]+", " ", text)      # US-Chicago == us chicago
    return re.sub(r"\s+", " ", text).strip()

key = (job["company"].lower(), title_key(job["title"]), location_key(job.get("location")))

Two details that bit me:

Normalise the hyphen. Without it, US-Chicago and us chicago are different keys. My first test failure was exactly this.
Fix the fingerprint too. If you build "only return jobs I have not seen" on an id derived from company+title, a second city for the same title looks like an already-seen job and never gets delivered.

What it changed

Measure	Title-only key	With location
7 companies: scraped → deduped	2,835 → 2,482 (12.5% lost)	2,835 → 2,824 (0.4% lost)
Full pool: scraped → deduped	18,833 → 15,548 (17.4% lost)	18,833 → 18,567 (1.4% lost)
Postings that state a salary	6,833	7,984 (+16.9%)

The recovered rows are mostly mid-to-low pay, so the median moved down slightly: $204,750 → $200,000. That is also a lesson — the bug was biasing the dataset upward, because the rows it deleted were disproportionately the many-city, lower-level sales and support reqs. If you only ever look at the median, the bias is invisible.

The rule I now follow

Treat the dedupe rate as an instrument, not a trophy. If it is above a few percent, print the collision groups and look at what differs inside them — location, department, employment type, seniority. The first differing field is almost always the one you forgot in the key.

And the inverse check is just as useful: if your dedupe rate is near zero, you are probably not deduplicating anything at all, and your cross-source duplicates are still in there.

The corrected numbers are live at sjdfi0wjvosdju9.github.io/salary-data.html — 18,567 postings, 7,984 with published pay, 259 companies, free CC BY 4.0 CSV. The age distribution is on a separate page if you came here for the earlier date-field bug. If you would rather not maintain the pipeline, the same data runs as a paid Apify Actor — but read the free CSV first and decide whether it is worth $0.0015 a row.

0% of Greenhouse jobs were over 6 months old. That was my bug, not their data

Radar Labs — Fri, 23 Oct 2026 20:55:58 +0000

I spent a week assuming my scraper was fine. Then a sanity check told me 0% of Greenhouse postings were older than six months, which is not a thing that happens on a real job board.

The culprit was one field. And it is not exotic — if you have ever scraped a job feed, you have probably made the same mistake.

The mistake is `updated_at`

Our Greenhouse connector read:

postedAt = row.get("updated_at") or row.get("created_at")

That reads fine. It is wrong twice over. First, the Greenhouse board API does not return created_at for a job, so the fallback never fires. Second — and this is the part that matters — updated_at is when a recruiter last touched the requisition, not when the job went live.

Here is what that does to real data. Account Executive, AI Native came back with a recent updated_at. Its actual first_published is 2024-12-20. A req opened in 2024 and edited yesterday looks one day old.

The fix is one line:

postedAt = (row.get("first_published") or row.get("created_at")
            or row.get("updated_at"))

first_published exists. It was just not the field we reached for.

What it changed

Same 259 companies, same day, only the date field changed:

	with `updated_at`	with `first_published`
median age	22 days	51 days
older than 180 days	5.5%	14.2%
Greenhouse older than 180 days	0.0%	14.5%

The Greenhouse number is the tell. No board has zero stale reqs. 0.0% was not a finding, it was a bug report: the freshness signal had been pinned to "last edited", so every edit reset the clock.

If you are filtering scraped jobs by age, or building an "only new since last run" delta on top of posted_at, this bug is silent and total. Every stale req comes back as new, forever.

The full age distribution (6,833 postings that publish a salary)

Measured 2026-10-24 from each board's own first-published field:

p10: 8 days
median: 51 days
p90: 236 days
oldest: 2,628 days (just over seven years)
older than 30 days: 65.9%
older than 90 days: 30.1%
older than 180 days: 14.2%
older than a year: 4.9%

So the median "open" role has been open for close to two months. Nearly a third are past a quarter, and one in seven is past half a year.

Boards differ, and the difference is the date semantics

Board	Postings	Median age	p90	Oldest	>180d	Date field
Greenhouse	4,063	52d	236d	2,628d	14.5%	`first_published`
Ashby	2,615	46d	215d	1,325d	12.7%	`createdAt`
Lever	155	77d	527d	2,230d	30.3%	`publishedAt`

Lever postings are older on median and much older in the tail. That is either a different hiring style or a board that keeps reqs visible after the hire — I cannot tell which from the data, so I am not going to dress it up as a conclusion.

Companies with the longest-running open reqs (minimum five dated postings): Secureframe median 442 days, Zilliz 430, Character 310, Delfina 299. A stale listing is not proof of a fake job either. Plenty of teams keep a rolling req open for a role they hire into continuously. But if you are a candidate sorting by "newest", a rolling req sorted by updated_at will follow you around forever.

The general rule

Before you trust any posted_at, answer two questions about it:

Which field is it? first_published / createdAt / publishedAt are posting dates. updated_at / modified_at are edit dates. Mixing them is the bug above.
Does the board even have the first kind? If it only exposes last-modified, your ages are understated and no amount of downstream maths will fix it — report it separately rather than averaging it in with boards that do it right.

A cheap sanity test that would have caught this in minutes: compute the share of postings older than six months, per board. If any real board comes back at 0%, you are not measuring what you think you are measuring.

The age numbers and the per-company breakdown live at sjdfi0wjvosdju9.github.io/freshness.html, regenerated weekly from the companies' own public feeds. The underlying postings, with dates and salary ranges, are a free CC BY 4.0 CSV — 6,833 rows across 259 companies. If you would rather not run the scraping yourself, the same data is available as a paid Apify Actor, and I would rather you read the free CSV first and decide whether it is worth $0.0015 a row.

Getting 6,826 job postings with salary as JSON in four minutes (and what it actually costs)

Radar Labs — Fri, 23 Oct 2026 19:48:05 +0000

Most "job API" tutorials show you a toy call and stop before the part where you need 20,000 rows. This is the other version: the exact request, the exact response, the exact bill, and the four ways it will fail on you.

What you get

One call returns jobs from 259 companies across Greenhouse, Ashby, Lever, Workable, Personio and Workday. Strip the ones without pay and you are left with about 6,826 postings that state an annual salary range, normalised to USD.

Each record looks like this:

{
  "company": "openai",
  "ats": "ashby",
  "title": "Research Scientist / Research Engineer - Foundations, Search",
  "location": "San Francisco, CA",
  "remote": false,
  "salaryLow": 380000,
  "salaryHigh": 850000,
  "salaryCurrency": "USD",
  "salaryUsdLow": 380000,
  "salaryUsdHigh": 850000,
  "postedAt": "2025-06-16T22:35:00.347000+00:00",
  "url": "https://jobs.ashbyhq.com/openai/…"
}

salaryUsdLow/High are the interesting fields: they exist because I convert hourly and monthly bands into a year and convert currencies, and because I keep the original numbers next to the normalised ones so you can second-guess me.

The call

No SDK, no pagination, no auth dance beyond an Apify token:

curl -X POST "https://api.apify.com/v2/acts/contributory_garnet_hyz~ats-job-feed/runs?token=$APIFY_TOKEN&waitForFinish=600" \
  -H "Content-Type: application/json" \
  -d '{
        "useVerifiedPool": true,
        "salaryOnly": true,
        "minSalaryUsd": 150000,
        "maxPerCompany": 3,
        "maxJobs": 100
      }'

That run takes about four minutes and returns 100 jobs. The request-level knobs:

input	what it does	default
`useVerifiedPool`	read the bundled 259-company list instead of the `companies` you pass	`false`
`companies`	your own slugs or careers URLs (Workday needs the full URL)	13 well-known slugs
`salaryOnly`	drop postings without a published range	`false`
`minSalaryUsd`	drop below this annual midpoint	`0`
`keywords`	substring match on title	none
`maxPerCompany`	stop one company eating the list	`20`
`maxJobs`	hard cap on returned rows	`200`
`maxWorkers`	parallel board requests (max 10)	`6`
`includeDescriptions`	read Greenhouse job bodies (needed for their salaries)	`true`
`onlyNew`	only return postings you have not seen in previous runs	`false`
`remoteOnly`, `excludeSenior`, `excludeJunior`, `excludeNonTech`, `maxAgeDays`	obvious filters	off

The bill, itemised

This is the part nobody publishes. Real numbers from my own runs:

$0.0005 per run start
$0.0015 per job returned
platform usage: about $0.006 with job descriptions on, about $0.0008 with them off

So: 100 jobs with descriptions ≈ $0.006 + $0.0005 + $0.15 = $0.157. 50 jobs ≈ $0.08. A daily diff of ~20 new postings ≈ $0.04/day.

That platform-usage line is 8x higher when includeDescriptions is on, because Greenhouse descriptions are a 144 MB download. If you do not need salary for Greenhouse companies — or you are happy with the ~18% coverage that structured fields give you — turn it off and the run is twice as fast and an order of magnitude cheaper.

But note the catch: salaryOnly + includeDescriptions: false will mostly return Ashby jobs, because Greenhouse publishes pay in prose, not in metadata. You are choosing between cheap and complete.

Four ways it breaks

Slugs rot. 527 candidate company slugs tested → 259 answered. Next month some of the 259 will 404. Every run returns failedCompanies with the reason per board, e.g. "greenhouse: empty; lever: HTTP 404; ashby: HTTP 404". It does not silently return fewer rows.
Workable is a shell. Its public endpoint answers for 4 companies and returns an empty list with a 200. Treated as a failure, not as "this company is not hiring".
Workday cannot be guessed. Its board host is {tenant}.wd{N}.myworkdayjobs.com and the root path returns HTTP 406, so there is no redirect to discover the site name. I tested 24 large companies × 2 host numbers × 4 site names = 192 guesses and got exactly one hit. That is why Workday needs a URL. Its list API also has no salary field at all, so a range can only come from the job description.
Greenhouse salaries are prose. "The base salary range is 196,000 USD - 310,500 USD" — currency after each number, plus postings that list Level 4 and Level 5 bands in one sentence. My first parser turned that into 196000–196000. If you build your own, test on NVIDIA's postings.

If you only want the snapshot

You do not need an account for the frozen version — 1,459 rows of it, CC BY 4.0:

CSV: https://sjdfi0wjvosdju9.github.io/data/salary-ranges.csv
Method, per-company medians, publication rates per board: https://sjdfi0wjvosdju9.github.io/salary-data.html

The live actor is https://apify.com/contributory_garnet_hyz/ats-job-feed — $0.0015 per job, and it is a one-person project, so if the parsing looks wrong somewhere I do want to hear about it.

Two honest caveats: this is published pay, not actual pay, and roughly 44% of postings publish nothing at all. Also, some companies post evergreen roles that stay up for a year — maxAgeDays will not save you from those, because their posting date is genuinely old and they are genuinely still open.

Numbers revision (2026-10-24): the pipeline was merging distinct city-level requisitions under one title, so row counts here are lower than the current dataset. After fixing the dedupe key the same 259 companies yield **18,567 postings / 7,984 with published pay* (median $200,000). The current figures live on the dataset page; the bug is written up here.*

Across 15,534 postings, remote engineering roles publish 11.5% lower pay than on-site ones

Radar Labs — Fri, 23 Oct 2026 19:33:43 +0000

Last time I published what 259 companies' job boards say about pay. Two questions came out of the comments that I could actually answer with the same dataset, so I pulled a fresh snapshot (15,534 de-duplicated postings, 6,826 with a stated annual range) and looked at them.

Short version: on-site engineering roles publish an 11.5% higher midpoint than remote engineering roles at the same companies, and Lever barely publishes pay at all — 9.5% of its postings, versus 53.5% on Ashby.

First: which boards tell you the salary

This is the part I had wrong in my head before measuring. I assumed Ashby was the pay-transparent board because it returns structured salary objects. It is, but only slightly more so than Greenhouse once you read Greenhouse's job descriptions:

Board	Postings read	With a stated range	Rate
Ashby	4,879	2,612	53.5%
Greenhouse	8,933	4,059	45.4%
Lever	1,640	156	9.5%
Workable	83	0	0%

The denominator matters: that is every posting read from the board, not just the ones with pay. So this is a publication rate, not a composition effect.

Two practical consequences:

Ashby gives you structured numbers (minValue, maxValue, currencyCode, interval) for half its postings. Cheap and exact.
Greenhouse gives you almost as much coverage but as free text inside a 144 MB download of job descriptions. You have to find "the base salary range is 210,000 USD - 310,500 USD" in a wall of marketing copy, and that is where I generated three spectacular parsing bugs last time.
Lever is where salary data goes to die. Clean schema, almost nobody uses it.

If you are building a pay dataset and you have to pick one board, pick Ashby. If you need volume, Greenhouse — and budget for the parsing.

Second: the remote gap

Overall, remote postings in this sample have a median midpoint of $191,000 against $211,500 for on-site. That is a 9.7% gap, and the obvious objection is composition: maybe remote postings are concentrated in lower-paying functions.

So I compared remote and on-site within the same role family, requiring at least 60 postings on each side:

Role family	Remote median	On-site median	Gap
other	$161,968	$198,000	−18.2%
support / ops	$140,600	$170,260	−17.4%
engineering	$207,094	$234,000	−11.5%
data / ml	$208,437	$226,000	−7.8%
marketing	$169,025	$182,750	−7.5%
product	$220,000	$230,000	−4.3%
sales	$166,500	$156,875	+6.1%

The gap survives controlling for function. It is biggest in support/ops and in the catch-all bucket, solid in engineering, and it reverses for sales — remote sales roles publish 6.1% more, which makes sense if you think of remote sales as covering bigger or more scattered territories.

What I cannot tell you

I can't separate two explanations, and I want to be blunt about it:

The same job pays less when it is remote. Companies discount for location independence.
Remote roles are open to cheaper geographies, so the published range reflects a national or global band rather than a San Francisco one.

Both produce exactly the pattern above. The dataset has a location column, and you could partly untangle it by splitting "remote (US)" from "remote (anywhere)" — but a large share of postings just say "Remote" with no country, so any split I do here would be me assigning geography by vibes. Not doing that.

There is also a smaller caveat: medians mix seniority. A company posting mostly staff-level on-site roles and mostly mid-level remote ones would show this gap without any remote discount existing. I have title, so a title-based seniority regex is the obvious next step, and it is on the list.

The part that is actually useful

If you are negotiating a remote offer, the interesting number is not "what do remote roles pay" — it is what does this company publish for the on-site version of my job. That is a concrete anchor, it comes from the same salary band, and it is a legitimate question: "I noticed the on-site posting for this role has a higher band; is the remote band adjusted for location, and if so, to what?"

In engineering the median difference at these companies is $26,906. That is worth one email.

Get the data

Same free CSV as before, refreshed: 1,459 rows, stratified so all 259 companies appear, CC BY 4.0, no signup:

CSV: https://sjdfi0wjvosdju9.github.io/data/salary-ranges.csv
Page with the method, per-company medians and the full tables: https://sjdfi0wjvosdju9.github.io/salary-data.html
Stats as JSON, if you want to check my arithmetic: https://sjdfi0wjvosdju9.github.io/data/salary-stats.json

The remote column is yes/no straight from the board, and role_family is my own regex over the title — so treat the family table as indicative, not as a classification anyone agreed on.

If your own company contradicts the engineering number, I would like to hear it. One counter-example with a real band is worth more than my median.

Numbers revision (2026-10-24, role classifier fix)

The role-family buckets in the earlier numbers came from a single-tier classifier
that matched engineer before anything else, so Sales Engineer, Solutions
Engineer and Support Engineer were all counted as engineering, and the word
data pulled data-center technicians into data/ML. That is also why other
reached 21.5% of rows.

Rebuilt with a four-tier classifier (disambiguation -> specific -> generic ->
broad) and 21 unit tests. Same 8,041 rows; only the buckets moved:
engineering 39.7% -> 27.4%, sales 10.8% -> 15.4%, support / ops 6.3% -> 15.2%,
finance / legal 3.3% -> 9.1%, other 21.5% -> 8.5%.

Consequence for the remote-vs-on-site gap: engineering was -11.5%; after the
fix it is -6.2% (remote median $212,000 vs on-site $226,000, n=563 / 1,638).
The direction holds, the magnitude shrinks. Current numbers:
https://sjdfi0wjvosdju9.github.io/salary-data.html

What 259 companies actually publish as pay: 6,823 job postings, normalised to USD

Radar Labs — Fri, 23 Oct 2026 19:24:29 +0000

I scraped 259 companies' job boards this week: 15,532 live postings after de-duplication. 6,823 of them state an annual salary range — 43.9%. I normalised all of them to USD and put the whole thing in a free CSV, because the numbers turned out more interesting than the tool I built to collect them.

Here is what they say.

The distribution

	Midpoint of the published range
p25	$154,700
median	$204,750
p75	$260,630
p90	$335,000

29.9% of the salary-bearing postings are remote.

A warning before anyone quotes that median: this is a census of published ranges, not a survey of what everyone earns. Companies that publish pay skew towards US tech, and towards senior roles. The 259-company pool was itself built by testing 527 slugs and keeping the 259 that answered, so it over-represents companies with a modern ATS and an explicit pay-transparency policy. Treat $204,750 as "the middle of what these 259 companies chose to write down", not "the middle of the market".

Who publishes, and who does not

The four boards I read behave very differently:

Ashby returns salary as structured numbers (minValue / maxValue / currencyCode / interval). About 52% of its postings carry one. Nothing to guess.
Greenhouse has a metadata field, but with content=false it comes back empty for every single posting. The salary is loose text inside the job body instead — and roughly 44% of postings have one. You have to read the description and find the range.
Lever has a clean salaryRange object, and 9% of postings use it.
Workable answered for 4 companies and published no pay at all.

So of the 6,823 paid postings: 4,058 came from Greenhouse, 2,609 from Ashby, 156 from Lever.

That matters if you are choosing where to look. If you want money data, Ashby is the cheapest source per row. If you want volume, Greenhouse wins on postings but you pay for it in parsing.

Highest published medians (companies with ≥3 paid postings)

161 companies cleared the three-posting bar. The top of the list is not subtle:

Company	Postings	p25	Median	p75
anthropic	568	$287,500	$347,500	$402,500
openai	631	$248,000	$307,500	$366,000
cohere	77	$204,400	$259,150	$286,525
perplexity	106	$195,000	$247,500	$312,500
vercel	71	$195,500	$245,000	$260,000
render	36	$230,000	$242,000	$259,000
figma	81	$207,000	$236,000	$264,500
modal	40	$200,000	$231,250	$275,000
vanta	47	$212,000	$229,000	$264,500

These medians mix seniority levels. A company hiring mostly staff engineers looks expensive; a company hiring support staff looks cheap. It is a composition effect as much as a pay effect. The first two rows are also the two with the most postings, which is a coincidence I would not read too much into — but it does tell you where the AI labs are setting the ceiling right now.

By role family, with at least 20 postings each:

Family	Postings	Median midpoint
product	255	$225,500
engineering	2,677	$225,000
data / ml	430	$221,337
finance / legal	229	$202,500
security	109	$195,000
design	142	$185,000
marketing	335	$179,000
sales	697	$160,000
support / ops	430	$158,875

Product managers and engineers land within $500 of each other, which surprised me. Sales spreads widest in absolute terms, because commission-heavy roles publish base ranges that mean very little on their own.

The parsing is where the lies live

Every one of these numbers went through a normaliser, and the normaliser produced three spectacular wrong answers before I made it stricter. All three are real examples from this dataset:

A monthly range labelled as USD. One Lever posting had {min: 128000, max: 176000, currency: "USD", interval: "per-month-salary"}. Multiply by twelve and it becomes $1,536,000 – $2,112,000 for a DevOps engineer. That number went to the top of my leaderboard and I nearly published it.
The word "hourly" anywhere in the description. A Greenhouse job description said "the base salary range is 210,000 USD - 310,000 USD" in one paragraph and mentioned "hourly employees" in another. My annualiser saw the word and multiplied by 2,080: $436,800,000.
A genuinely hourly contract. $20.67 – $21.88 went in as an annual salary, because the guard at the time only rejected values below $100.

The fix that finally held was unglamorous: read the currency from the end of each number, not the start, only apply an hourly multiplier when the range actually looks hourly (floor under $500), and then throw away anything outside $5,000 – $5,000,000 after normalisation instead of trying to repair it.

The last rule costs me rows. GoCardless and Coinbase both published monthly figures I could not prove were monthly, so those rows simply have no salary now. Empty beats confidently wrong. If someone makes a career decision off this CSV, the only acceptable failure mode is a blank cell.

What is in the file

salary-ranges.csv — 1,460 rows, stratified so every company in the pool appears at least once. One row per posting:

company, ats, title, location, remote, role_family,
salary_usd_low, salary_usd_high, salary_usd_mid, posted_at, url

CC BY 4.0. No signup, no email gate:

CSV: https://sjdfi0wjvosdju9.github.io/data/salary-ranges.csv
The page with the method and the full tables: https://sjdfi0wjvosdju9.github.io/salary-data.html
How I picked the 259 companies (55 niches measured, several of them rejected with numbers): https://sjdfi0wjvosdju9.github.io/how-we-choose.html

Captured 2026-10-24. Job boards rotate slugs constantly, so expect a handful of companies to 404 next week — the tool reports those in failedCompanies rather than silently returning less.

Why I am giving it away

I built a paid actor that returns this data live (https://apify.com/contributory_garnet_hyz/ats-job-feed, $0.0015 per job), and the honest situation is that nobody knows it exists yet. A CSV with a real number in it is a better introduction than another landing page.

If you want the refreshed version rather than the snapshot: set useVerifiedPool: true and salaryOnly: true on that actor and it returns the same schema for the current day, 259 companies, 259 HTTP requests, about four minutes.

Two things I would genuinely like feedback on:

Is the midpoint the right summary? Right now I rank by it, but a $150k–$350k range and a $240k–$260k range have the same midpoint and are not the same offer.
Should I keep rows where the range came from free text rather than a structured field? It costs a 144 MB download to read Greenhouse descriptions and it is where most of the garbage lived.

If you spot a salary in that CSV that is obviously wrong, tell me — I would rather fix the parser than defend it.

Numbers revision (2026-10-24, role classifier fix)

Six APIs, zero keys: what 18,825 real job postings contain

Radar Labs — Thu, 22 Oct 2026 21:27:25 +0000

I wanted a job feed I could filter myself, so I tried to fetch one from the source instead of the aggregators. It turns out that most company career pages are thin wrappers around a handful of applicant tracking systems (ATS), and those systems expose JSON endpoints without keys or login.

Six of them answered. I guessed 527 plausible company slugs, 259 responded, and that produced 18,825 real job postings. Here is what I found in them — including two bugs that silently published numbers that were off by two orders of magnitude.

The six endpoints

ATS	Endpoint	Auth
Greenhouse	`boards-api.greenhouse.io/v1/boards/{slug}/jobs`	none
Lever	`api.lever.co/v0/postings/{slug}?mode=json`	none
Ashby	`api.ashbyhq.com/posting-api/job-board/{slug}`	none (add `?includeCompensation=true` for pay)
Workable	`apply.workable.com/api/v1/widget/accounts/{slug}`	none
Personio	`{slug}.jobs.personio.de/xml`	none
Workday	`POST {tenant}.wd{N}.myworkdayjobs.com/wday/cxs/{tenant}/{site}/jobs`	none

All public, all read-only, none of them need an account. Some have changed shape or disappeared: SmartRecruiters returned 404 on both endpoints I tried, and Teamtailor customers usually sit behind their own domain so the documented JSON path just 404s.

Workday is the awkward one. It needs a tenant, a cluster number and a site name before you can address a board at all, and guessing is close to hopeless: 24 well-known companies × 2 clusters × 4 site names = 192 probes, exactly one hit. Everything else had to be supplied by hand.

What a sweep returns

18,825 postings from 259 companies (49% of guessed slugs)
8,071 with a published salary (43%) — but that number only exists because of the mistake below, which is the most interesting thing I learned

Finding 1: I was wrong about Greenhouse

My first sweep looked like this:

ATS	Companies	Postings	With salary
Greenhouse	135	11,608	50
Ashby	98	5,274	2,734
Lever	23	1,810	160
Workable	4	121	0

93% of every salary-bearing row came from Ashby. Greenhouse had the volume and essentially no pay, so the obvious conclusion was "Greenhouse doesn't publish salaries".

That conclusion was wrong, and the reason is that I was asking for the wrong payload. ?content=false returns postings with the metadata array set to null for all of them. Ask for ?content=true and the pay range is sitting right there in the description text — 44% of Greenhouse postings have one.

ATS	Companies	Postings	With salary
Greenhouse	134	11,619	5,164 (44%)
Ashby	98	5,283	2,739 (52%)
Lever	23	1,802	168 (9%)
Workable	4	121	0

The trap is that Greenhouse's content field is double-escaped HTML: the description literally contains <div class="content-intro">. One html.unescape is not enough. If you unescape once and search for $, you find nothing and conclude there is no salary. You need to unescape until the string stops changing (I cap it at three rounds).

Finding 2: Ashby hides its compensation behind one flag

Ashby returns beautiful structured compensation — compensationTiers, summaryComponents with minValue, maxValue, currencyCode, interval — but only when you pass ?includeCompensation=true. Without the flag you get a gorgeous list of jobs and zero numbers. Same failure mode as Greenhouse, different cause: the API is right, the request is wrong.

Finding 3: the $1,536,000 ghost

Lever returns structured ranges:

{"min": 128000, "max": 176000, "currency": "USD", "interval": "per-month-salary"}

That is a monthly salary — in a Latin American band — labeled USD. Annualise it and you get $1,536,000 for a DevOps engineer. It briefly topped the entire scan and looked completely legitimate, because it was a well-formed number in a well-formed field.

Then this one, from a long Greenhouse description:

The base salary range is 210,000 USD - 310,000 USD. Some hourly employees exist.

The word "hourly" appears somewhere in the text, so my annualiser multiplied an annual range by 2,080 and produced $436,800,000.

Both bugs share a shape: the output was plausible. A range that is off by 12× or 2,080× does not look broken; it looks like a senior role at a rich company.

I stopped trying to repair these and started deleting them. Anything that annualises below $5,000, above $5,000,000, or above $1,500,000 in USD loses its salary fields entirely while the posting stays. And the unit words are only read from the ~40 characters after the last number in the sentence, not from the whole description:

def _unit_tail(window, limit=40):
    # "…210,000 USD - 310,000 USD. Some hourly employees exist."
    # -> only look at " USD." , so "hourly" later in the text cannot fire

Since then the maximum salary in a full sweep is $850,000, which belongs to the posting it is attached to.

Lesson: in a data product, one wrong order of magnitude is worse than one missing field.

Finding 4: the pay range is not always a number

American postings often write the currency after each number:

The base salary range is 196,000 USD - 310,500 USD.

My regex was built for $196,000 - $310,500, so it matched the first number plus "USD", stopped, and produced 196000 - 196000. The output looked perfect — a single plausible number — which is exactly the kind of bug that ships. You need a separate pattern for number currency - number currency, and you need to decide what to do about postings that list several bands (… for Level 4, and … for Level 5); I take the first band and say so in the docs.

Finding 5: dedupe and "only new" are the real features

Large companies post the same role on several boards, so a raw sweep duplicates heavily: 18,825 rows collapsed to 15,528. And if you (or your users) pay per row, the feature everyone actually wants is "only the postings I have not seen yet" — which needs a stable fingerprint per posting plus somewhere to persist the seen set. Mine is company + normalised title + url, hashed, stored in a key-value store, capped at 50k entries, FIFO.

The normalisation detail that matters: strip punctuation and case, but do not strip location qualifiers. Engineer (Remote) and Engineer (Berlin) are different jobs.

If you want to try this yourself

Do not trust labels, tags or category counts — measure the payload.
Ask for compensation explicitly (Ashby), and fall back to reading the description (Greenhouse, Workday).
Guard every annualisation with a plausibility bound, and drop rather than repair.
Deduplicate on company + normalised title + url, never on title alone.
Store a fingerprint per posting if you want incremental results.

Honest limits

Slug guessing worked 49% of the time. Workable boards often answer with an empty shell. Some postings are evergreen and never close, so "new" is fuzzy at the edges. Salary only exists when a company chooses to publish it — 43% of rows here, not 100%. And a wrong number is worse than a missing one, so I drop aggressively and tell users when a company returned nothing.

I packaged this as a small tool that scans the six APIs, normalises salaries to annual USD, dedupes across boards and can return only new postings: ATS Job Feed. It ships the list of 259 companies that were verified live, so you do not have to guess slugs, and it can return only postings that actually state a salary.

If you pull job data for a living: which of these six APIs has bitten you the hardest? I would like to know which shape I should handle next.

Inspect the actual rows before trusting any of these numbers. Both sample outputs are mirrored as static tables — job feed: https://sjdfi0wjvosdju9.github.io/samples/ats-jobs.html, lead radar: https://sjdfi0wjvosdju9.github.io/samples/freelance-leads.html. No account needed. The selection notes (including the markets I rejected) are at https://sjdfi0wjvosdju9.github.io/how-we-choose.html.

DEV Community: Radar Labs

One company, one 75,500 dollar gap between role families

Why this gap is worth sitting with

Method, so you can distrust it properly

What to do with it

Remote work is not one pay discount: what 8,049 published salary ranges show

The headline number, and why it misleads

Split by function, the gap inverts

The rest of the dataset says the same thing about labels

How to use this

I counted 1,726 job postings as 'other'. My classifier was lying.

What was actually wrong

The fix is boring, and that is the point

The test caught me, immediately

What changed in the numbers

The part I did not enjoy: a published claim moved

Three checks I now run before publishing any derived column

11 companies with 40+ open roles publish a salary on none of them

11 companies with 40+ open roles publish a salary on none of them

Same board, opposite behaviour

Eleven companies publish nothing at all

Why the denominator discipline matters

The data

My site answered 200 on all 163 URLs. Crawlers never saw a single page.

My site answered 200 on all 163 URLs. Crawlers never saw a single page.

The clue was one request away

The second bug, in the same audit

What I changed

Four things I would tell my past self

The upside of finding it

1,170 job postings looked like they had no salary. My currency table was deleting them.

The setup

The failure mode worth remembering

The fix

What it actually moved

Three checks you can copy

I was deleting 17% of real job postings with one line of dedupe

The symptom was a number I was proud of

Measuring the collision

The fix is one field

What it changed

The rule I now follow

0% of Greenhouse jobs were over 6 months old. That was my bug, not their data

The mistake is updated_at

What it changed

The full age distribution (6,833 postings that publish a salary)

Boards differ, and the difference is the date semantics

The general rule

Getting 6,826 job postings with salary as JSON in four minutes (and what it actually costs)

What you get

The call

The bill, itemised

Four ways it breaks

If you only want the snapshot

Across 15,534 postings, remote engineering roles publish 11.5% lower pay than on-site ones

First: which boards tell you the salary

Second: the remote gap

What I cannot tell you

The part that is actually useful

Get the data

Numbers revision (2026-10-24, role classifier fix)

What 259 companies actually publish as pay: 6,823 job postings, normalised to USD

The distribution

Who publishes, and who does not

Highest published medians (companies with ≥3 paid postings)

The parsing is where the lies live

What is in the file

Why I am giving it away

Numbers revision (2026-10-24, role classifier fix)

Six APIs, zero keys: what 18,825 real job postings contain

The six endpoints

What a sweep returns

Finding 1: I was wrong about Greenhouse

Finding 2: Ashby hides its compensation behind one flag

Finding 3: the $1,536,000 ghost

Finding 4: the pay range is not always a number

Finding 5: dedupe and "only new" are the real features

If you want to try this yourself

Honest limits

The mistake is `updated_at`