I scraped 89 job postings to find out how my salary parser was lying to me

I shipped a LinkedIn jobs scraper that returns salary as structured numbers — salaryMin, salaryMax, currency, and a yearly figure so an hourly warehouse job and a director role sit on the same axis. The whole point was to make "show me jobs paying over $150k" a question you can actually ask.

Then I ran it against 89 real postings and found six separate ways it was wrong.

None of them threw an error. Every single one produced a confident-looking number. That is the part worth writing about: a salary parser doesn't fail loudly, it fails plausibly, and the only way to find out is to sit down with real postings and check them one by one.

Here's what I found, in the order I found it, including the one where the platform had already solved my problem and I broke it by "fixing" it.

Why job descriptions are hostile to salary parsing

My first assumption was that the salary would live in a structured field. It doesn't.

I pulled 64 postings across eight roles and countries — software engineer in the US, registered nurse in California, retail associate in Texas, truck driver, marketing manager in New York, software developer in the UK, data analyst in Canada, accountant in the US — and looked for where the money actually was:

• The top-card salary box: 3 out of 64
• JSON-LD baseSalary: 0 out of 64

So the salary is in the free-text description, mixed in with every other dollar figure a company might mention. And companies mention a lot of dollar figures:

"We're a Series A company with $20M+ raised from Headline, Village Global"
"serving an $800 billion market"
"New Hire Bonus: $3,000 after 90 days"
"[$5,000] signing bonus available to external hires"
"Annual maximum out of pocket $3,000 per individual / $7,500 per family"
"we have grown from $1 million in sales in 1983 to more than $500 million today"

Not one of those is a salary. A naive regex that grabs $[\d,]+ returns all of them.

My first version had a plausibility gate for exactly this reason: normalize any candidate to a yearly figure first, and if it lands outside $8,000–$2,000,000, it isn't a wage. That one rule kills $20M raised, $800 billion market, and $1 million in sales instantly, because nobody is paid $800 billion a year.

It also killed a real salary, which is where the trouble started.

Bug 1: the bonus in the next sentence poisoned the salary

Here's a real posting:

Salary Range: $46.00 - $50 per hour   New Hire Bonus: $3,000

My parser rejected it. Not the bonus — the salary.

The reason: I was testing my noise words (bonus, signing, raised, market) against a window of context around the candidate. $46.00 - $50 per hour had "New Hire Bonus" sitting 8 characters to its right, so the whole match got thrown out.

The fix was to stop treating context as a blob and start treating it as directional. A number's label is what comes before it. What comes after belongs to the next number:

// Drop it only if this number's own label is noise.
// Whatever sits on the right belongs to the next number, not this one.
if (NOISE_BEFORE.test(left)) return null;

NOISE_BEFORE is anchored to the end of the left context:

const NOISE_BEFORE = /(bonus|signing|sign[-\s]?on|relocation|referral|stipend|allowance|raised|funding|valuation|revenue|market|worth|in\s+(sales|arr)|series\s+[a-e])\b[\s:_*\-]*$/i;

The $ anchor is the whole trick. "Bonus:" immediately before a number means that number is a bonus. "Bonus" thirty characters later means nothing about this number at all.

Once I made that change, the pay range came back and the $3,000 bonus stayed rejected — which is exactly right.

Bug 2: a following word's first letter became a magnitude suffix

This one was invisible until I checked a number I had no reason to suspect.

1st year compensation plans range from $55,000-$95,000 Medical, dental, life

Parser said: no salary found.

I instrumented the candidate loop and printed what it actually matched:

raw=|$55,000-$95,000 M|

There it is. My money regex ended with [kKmM]? to catch $150K and $20M. The word after the range was Medical. The regex ate its first letter as a magnitude suffix, read $95,000 M as ninety-five billion, and the plausibility gate — correctly, given what it was handed — threw the whole thing away.

A parser that turns $95,000 into $95,000,000,000 and then silently drops the job is worse than one that crashes. The job just quietly isn't in your results.

The fix is a negative lookahead: a k or m only counts as a suffix if a letter doesn't follow it.

// k/m is a magnitude suffix only when it follows the digits AND no letter follows it,
// so "95,000 Medical" keeps its M.
const NUM_RE = String.raw`\d[\d,]*(?:\.\d+)?\s?[kKmM]?(?![A-Za-z])`;

Regression test, because I never want to relearn this:

test('regression: does not eat a following word initial M/K as a suffix ($95,000 Medical)', () => {
  const p = ok('compensation plans range from $55,000-$95,000 Medical, dental');
  assert.equal(p.salaryFound, true);
  assert.equal(p.salaryMax, 95000); // must not read "$95,000 M" as 95,000,000,000
  const q = ok('base pay $80,000 Monthly meetings and $18/hr overtime');
  assert.equal(q.salaryFound, true); // does not eat the M of "Monthly"
});

Bugs 3 and 4: ranges that collapsed to a single number

Two postings looked fine in the output. They weren't.

Salary Range: $81,000 USD - $105,000 USD    →  min 81000, max 81000
expected hiring range is $36.79/hr - $58.50/hr  →  min 36.79, max 36.79

Both reported the bottom of the range as if it were the whole range. If you filter on "pays at least $100k", the first job is wrongly excluded. Quiet, plausible, wrong.

The cause was the same in both cases: my range pattern was positional, NUMBER separator NUMBER, and both postings put something between the first number and the separator — a currency word in one, a unit in the other. The pattern stopped at the first number and treated it as a lone value.

I could have kept patching the pattern with more optional groups. Instead I stopped matching positions and started counting numbers:

// Take at most two numbers out of raw. A positional range regex breaks as soon as a unit
// or currency word sits between them ("$X/hr - $Y/hr", "$X USD - $Y"), so extract the
// numbers instead and let the count decide.
const nums = [...raw.matchAll(/(\d[\d,]*(?:\.\d+)?)\s?([kKmM])?(?![A-Za-z])/g)]
  .map((m) => toNumber(m[1], m[2])).filter((v) => v != null);

Two numbers inside one money token means a range. One means a single value. It stopped mattering what was sitting between them.

That refactor immediately exposed bug 4: C$90,000 - C$110,000 also collapsed, because a leftover separator check didn't recognise C$ after the dash. Deleting the check fixed it — the tokenizer already guaranteed what the check was testing for.

Bug 5: the insurance benefit that looked like a director's salary

An electrician job came back at $200,000 a year.

Electricians in that market make about $80k. The parser had found this:

Annual maximum out of pocket $3,000 per individual / $7,500 per family
... Health Insurance (In-network) $1,000 individual deductible
... Life Insurance plan with coverage up to $200,000

The word Annual was doing the damage. My period detector saw it, concluded "yearly", and the plausibility gate happily accepted $200,000 as a yearly wage. It is a plausible salary. It is not this job's salary — it's an insurance payout ceiling.

The fix was another directional rule, this time for benefits vocabulary:

// Benefits and insurance words disqualify a number only when they sit on the LEFT,
// the label side. "out of pocket $200,000" -> drop it.
// "salary $120,000 plus health insurance" -> the label is salary, so keep it.
if (BENEFIT_NEAR.test(left)) return null;

I wrote it as a both-sides check first and caught myself: base salary $120,000 plus health insurance and 401k match is an extremely normal sentence, and a both-sides rule throws that real salary away. Left-side only keeps it.

Bug 6: "40 hours per week" is not a pay period

A dental hygienist posting listed both a full-time salary and a part-time rate:

Full time hours-40 hours per week salary-$60,428.52.
Part-time-work up to 29 hours per week, $45/hour.

The parser returned the part-time hourly rate and ignored the full-time salary entirely.

Why: $60,428.52 had "hours per week" sitting in its left context. My period detector read per week, called it a weekly wage, multiplied by 52, got $3.1M, and the plausibility gate threw it out. The part-time rate survived, so the output looked complete.

Schedule language and pay-period language use the same words. So I strip schedule phrases before reading the period:

// A work schedule ("40 hours per week", "5 days a week") is not a pay period, so remove it
// from the context first. Leaving it in makes the $60,428 in "hours per week salary-$60,428"
// look like a weekly wage.
context = String(context).replace(/\b(hours?|hrs?|days?|shifts?)\s*(?:\/|per\s|a\s|an\s)\s*(?:week|day|month|hour|year)/gi, ' ');

With that, the full-time salary came back at $60,429/year — and Base pay $800 per week, a genuine weekly wage, still parses as weekly. Both are in the test suite.

What I got wrong about being wrong

I want to be honest about a seventh thing, because it wasn't a parser bug — it was me.

Separately from the salary work, I was making my Actors usable as tools for AI agents through Apify's MCP server. I compared my input schemas against what the MCP tools/list response exposed, and noticed that required didn't appear in the tool definitions. Three Actors, same result. I concluded the agent never learns which fields are mandatory, edited six Actors to prepend "Required." to those field descriptions, and deployed all six.

Then I looked at the description text an agent actually receives:

**REQUIRED** LinkedIn job URLs (https://www.linkedin.com/jobs/view/...)

Apify's MCP server already injects **REQUIRED** into the description of required fields. The information was reaching the agent the whole time. My "fix" produced **REQUIRED** Required. LinkedIn job URLs... on three live Actors before I caught it. I reverted all six and redeployed.

The mistake wasn't the edit. It was that I checked the field I expected to carry the information instead of the payload the consumer actually receives. Which is the same mistake as bug 5, one level up: I trusted a signal (Annual, required) without checking what it meant in context.

What the parser refuses to do

The finished version outputs a number only when it can defend it:

• Normalize first, judge second. Everything becomes a yearly figure before the plausibility gate. Hourly × 2080, monthly × 12. $800B market never reaches the output because $800B/year is not a wage.
• A number needs its own label. A bare $600,000 in a sentence about project value is not pay. Something must mark it: a unit (per hour, /yr), a pay word before it (salary, pay range, compensation), or a period word in context.
• Direction matters. Labels sit to the left. Words to the right belong to the next number.
• When it can't tell, it says so. salaryFound: false and no invented figure.

And every run reports the fill rate, because the honest answer to "why did I only get 74 jobs" is usually "because 6 in 10 US postings state pay, and the rest didn't write a number down":

examined: 12   withSalary: 8 (67%)   returned: 7
droppedBelowMin: 1   droppedNoSalary: 4
annualSalaryMedian: 250000   range: 135000-350000

That 67% matched my earlier measurement of 63% across a different sample, which was the first thing that made me trust the pipeline.

If you're building something similar

Three things I'd do from the start next time:

Build the corpus before the parser. I spent an afternoon collecting money phrases with their surrounding context from eight different job types. Nurses and retail workers are quoted hourly, engineers yearly, accountants monthly, and each format breaks a different assumption. If I had only tested on software jobs I would have shipped four of these six bugs.

Print what your regex actually matched. Bug 2 was undiagnosable from the output — the field was just empty. One line dumping raw=|$55,000-$95,000 M| made it obvious in seconds.

Write the regression test in the same commit as the fix. All six are in the suite. When I later replaced the range logic entirely, those tests caught a fifth bug I'd introduced in the process, before it shipped.

The parser and every regression test in this post are at northbell-dev/honest-scraping — no dependencies, npm test runs the 40 tests with nothing to install.

The Actor is LinkedIn Jobs Salary Data. It reads only public pages — no login, no cookies, and the request headers are a frozen object that cannot carry one.

---

northbell builds honest web scrapers on Apify. If a derived number has an error margin, the margin ships with it.