DEV Community: northbell

I measured what my 11 Actors cost to run. The 96x spread was mostly one config field.

northbell — Sat, 05 Sep 2026 23:23:04 +0000

I wrote this article twice. The first version was finished, proofread, and queued to publish. Then someone asked a question about one number in it, and the thesis came apart.

I am publishing the second version, along with the part where I was wrong, because the mistake turned out to be more useful than the article I meant to write.

What I set out to do

Pay-per-event pricing asks you to name a price per unit of value. I had priced eleven Actors that way without knowing what a unit costs me to produce.

So I measured. Twenty-two runs, every Actor I have published, cost taken from the platform's own accounting rather than from an estimate. Every finished run carries usageTotalUsd, every dataset carries itemCount, and cost per row is the first divided by the second:

const run = await api(`/actor-runs/${runId}`);
const ds  = await api(`/datasets/${run.defaultDatasetId}`);
const costPerThousand = (run.usageTotalUsd / ds.itemCount) * 1000;

Two things to know before you copy it. On the free plan, runs and their datasets are kept for a limited window and then removed — I went looking for figures from two weeks earlier and they were simply gone, so save whatever you measure outside the platform. And one run is not a measurement: per-run overhead lands entirely on however many rows that run happened to produce.

The table I almost published

Actor	cost / 1,000 rows	rows per second
bulk job listings	$0.008	5.8
Shopify app reviews	$0.010	5.7
App Store ranks	$0.023	7.7
all jobs at one company	$0.035	1.8
company posts	$0.059	1.9
jobs + applicant counts	$0.105	0.6
company headcount	$0.254	0.3
jobs + salary parsing	$0.417	0.5
jobs + working filters	$0.427	0.5
Google Play ratings	$0.508	0.7
job details from a URL (free)	$0.730	0.3

$0.008 to $0.730. A 96× spread, and the most expensive one was the Actor I give away for free.

The story I told myself

Requests per row, I wrote. That's the whole story.

The cheapest Actor and one of the most expensive read the same public endpoint, on the same site, with the same library. The cheap one takes ten rows out of every request it makes. The expensive one makes one request per row, because the thing it needs — the salary text, the applicant count — only exists on the individual posting.

It is a good story, and everything in the table lined up behind it. Cheap Actors ran at 5–8 rows per second, expensive ones at 0.3–0.7. Cost and throughput, the same measurement wearing different clothes, because you are billed for the time a machine is held open.

I wrote twelve hundred words on that and finished with the free Actor's $0.730 as the closing irony.

The question that broke it

Then someone read the draft and asked the least sophisticated question available:

"That $0.730 — is that actually fine as it is?"

I had treated the number as a finding. It was a bug report, and I had filed it as an anecdote.

What I found

I set out to check whether $0.730 was affordable. I ended up checking the run settings of the four most expensive Actors instead.

All four were configured for 4096 MB of memory. The other seven were at 512.

The four most expensive Actors in my table were exactly the four with eight times the RAM. Not approximately — exactly.

Apify bills memory × time. A container with 8× the memory costs 8× per second whether or not anything uses it. And none of these four needs it: no browser, no Playwright, one dependency (apify), plain fetch, and the detail lookups run in a sequential rate-limited loop. Four gigabytes to hold one HTTP response at a time.

I have no memory of choosing 4096. It is the kind of value that lands in a template and then in every Actor you scaffold from it.

The controlled test

Same Actor, same input, three memory settings:

memory	cost	rows	seconds
4096 MB	$0.000746	1	3
1024 MB	$0.000224	1	3
512 MB	$0.000111	1	3

Roughly linear in memory, flat in time. And on a real workload rather than a one-row sample — 51 rows, 108 seconds:

Actor	at 4096 MB	at 512 MB	throughput
jobs + working filters	$0.427 / 1,000	$0.093 / 1,000	0.5 → 0.5 rows/s
jobs + salary parsing	$0.417 / 1,000	$0.113 / 1,000	0.5 → 0.5 rows/s

Identical throughput. The RAM was never doing anything.

The corrected table

Actor	cost / 1,000	was	rows/s
bulk job listings	$0.008	—	5.8
Shopify app reviews	$0.010	—	5.7
App Store ranks	$0.023	—	7.7
all jobs at one company	$0.035	—	1.8
company posts	$0.059	—	1.9
jobs + working filters	$0.093	$0.427	0.5
jobs + applicant counts	$0.105	—	0.6
job details from a URL (free)	$0.111	$0.730	0.3
jobs + salary parsing	$0.113	$0.417	0.5
Google Play ratings	$0.139	$0.508	0.7
company headcount	$0.254	—	0.3

The spread is 33×, not 96×. The free Actor is no longer the most expensive; it sits in the middle. The Actor now at the top is a genuinely one-request-per-row job, which is the only kind that belongs there.

What survives, and what doesn't

Requests per row is real. It is still the largest architectural driver in the table, and it accounts for the entire 33× that remains. That part of the first draft was right.

But it was not "the whole story," and the ranking I built the story on was wrong. Four Actors sat at the top of my table for a reason that had nothing to do with their code — and I explained their position with an argument about their code, an argument I could support for each of them, in a paragraph.

That is the uncomfortable part. The story was not weakly supported. It was well supported and wrong.

A derived number does two jobs: it gives you a value, and it gives you a ranking. If one input is misconfigured, you do not get a slightly wrong value. You get a confidently wrong ranking — and the ranking is the part you reason about.

And check who actually pays

One more thing the second pass turned up.

My free Actor is not monetized, which on Apify means pay-per-usage: "Users can run the Actor without any additional charges beyond the platform usage costs generated by the Actor." The person who runs it pays, from their own account.

So $0.730 was never my bill. It was the bill I was handing to whoever tried my free sample first — the one Actor in that table whose cost lands on a stranger, and the one I had written up as a personal irony.

It is now about $0.11, and it is still free.

Three things I would check on day one

The run settings, before the code. Memory and timeout are two fields. They are set once, never reviewed, and one of them is a direct multiplier on every bill you generate. Mine was wrong on four Actors and I never looked.

Whether your number ranks or only measures. If you are going to reason about which item is worst, confirm the inputs are comparable before you interpret the order.

Who the cost lands on. Free does not mean free. It means someone else.

Reproduce it

List your Actors, take the recent successful runs, divide usageTotalUsd by the dataset's itemCount, multiply by a thousand — and print defaultRunOptions.memoryMbytes as a column in the same table. That last column is the one I would add first.

The free Actor is Free LinkedIn Job Scraper — paste a job URL, get JSON back, no login and no paid plan. It is about 4× cheaper to run than it was this morning, and that took no code change at all.

northbell builds honest web scrapers on Apify. Including, evidently, honest corrections.

My 11 Actors were invisible in Apify Store for a week. One query parameter proved it.

northbell — Thu, 03 Sep 2026 01:58:19 +0000

I published eleven Actors on Apify at the end of August. A week later they had, between them, zero users who were not me or Apify's own daily QA run.

I did the things you do. I rewrote the store descriptions around the words buyers type. I published two articles under the Apify organisation on DEV. I posted in the Discord. Nothing moved, and I kept telling myself it was a discovery problem — the Actors were fine, nobody knew they existed yet.

That was true. It was just not true in the way I thought. Nobody knew they existed because, for anyone searching Apify Store, they did not.

This post is the diagnosis, the one API call that made it certain, the fix, and a warning for anyone publishing from outside the US or EU. It took about four hours to resolve once I knew what to ask. It took a week to know what to ask.

The symptom that finally looked wrong

Store search ranks by usage, so a new Actor being buried under established ones is normal. I did not expect to see mine on page one for "linkedin jobs". I did expect to see them somewhere.

The check that broke the "just buried" story was searching for one of my own Actor slugs, exactly:

GET https://api.apify.com/v2/store?search=linkedin-company-jobs-scraper
→ 12 results. None of them mine.

To make sure that was not simply what happens to small Actors, I searched the exact slug of another creator's Actor that had four users in total. It came back first, alone. So an Actor with four users is findable by its own name, and mine — public, passing QA, 98.8% run success — were not.

Two more checks, both negative:

sortBy=lastUpdate did not include my Actors minutes after I rebuilt two of them. Whatever was hiding them was not a ranking problem.
The web Store search behaved identically to the API. My profile page listed all eleven as public. The Actor pages loaded fine by direct URL.

Public, reachable, working, and absent from every list. That is not "buried". That is a filter.

The parameter

The Store endpoint has a parameter I had never read past:

includeUnrunnableActors — By default, search results exclude Actors that are not safe to run automatically (e.g. Actors from developers who haven't passed KYC, or full-permission Actors without a large user base). Set to true to bypass this safety filtering and include all Actors in the results.

So I ran the pair that settles it:

GET /v2/store?username=northbell
→ 0 Actors

GET /v2/store?username=northbell&includeUnrunnableActors=true
→ 11 Actors

The same eleven Actors appear and disappear on one flag. Nothing about indexing, nothing about ranking. The Store had them; it was choosing not to show them.

The parameter's own description names two causes. All eleven of my Actors run with LIMITED_PERMISSIONS, so it was not the second one. That leaves the first: I had not completed identity verification.

If you take one thing from this post, take the pair of requests above. It is a ten-second test, it returns a yes or a no, and it tells you whether any amount of work on titles, READMEs, categories or promotion can help you at all. For a week, in my case, it could not have.

Where the verification actually lives

I went looking for it in Settings. /settings/payouts and /settings/monetization both redirect to the account page, which is where I concluded the feature was not enabled for my account and stopped.

It is under Actors → Insights → Payouts, as a banner that says Identity verification required with a Verify Identity button. I only found it after a support reply mentioned "Insights – Payouts", and I had been looking at that sidebar entry for a week without opening it, because "Insights" did not sound like a place where a gate would be.

Why my documents failed, and why yours might

The verification portal accepts a passport, a driving licence, or a national ID card. I am in Japan. I tried my driving licence, then my national ID card, several times each, in good light, flat, all four corners in frame. Every attempt came back rejected, with a reviewer note that the provider "does not support verification through this type of document" and a banner reading Mismatch between the data you declared and the data from the verification.

The two messages together are the explanation. Japanese driving licences and national ID cards print the holder's name in Japanese characters only — there is no romanised name anywhere on the card. My account name is in Latin script. The automated check had nothing on the document to match it against, and reported a mismatch. In Japan the passport is essentially the only widely held ID that carries a romanised name, and I do not have one.

So this was never a photography problem, and retrying with a better photo was never going to work. If your country's ID does not carry your name in the script your account uses, expect the same, and go straight to support rather than re-submitting.

What support did

I wrote to support with the structural explanation above — not "it failed", but why it could not succeed. The first three replies were from an AI assistant that could not answer a policy question, and said so. When I asked it to pass the case to a person, it did. A support engineer acknowledged within half an hour, asked the verification team, and about four hours after my first email I had this in the console:

Identity verified. Your identity has been successfully verified. You're all set to receive payouts.

I re-ran the pair of requests. ?username=northbell returned eleven.

What visibility bought, measured

Being listed is necessary. I want to be precise about how far it is from sufficient, because the next trap looks a lot like success.

An hour after verification, my Actors ranked in Store search like this:

query	rank
`google play rating`	1
`linkedin applicants`	1
`linkedin headcount`	2
`shopify app reviews`	5
`linkedin company jobs`	10

First place, with zero users, on the first day. It would have been easy to post that table and stop.

But rank on a query only matters if people type the query. So for each one I added up the 30-day users of the ten Actors that rank for it — a rough measure of how many people actually arrive through that phrase:

query	my rank	30-day users, top 10 combined
`google play rating`	1	18
`linkedin applicants`	1	22
`linkedin jobs salary`	6	51
`linkedin jobs`	not in top 25	31,870
`linkedin scraper`	not in top 25	37,833

I am first on phrases that a few dozen people a month use, and absent from the phrases that thirty thousand use. The phrases I win are, more or less, my own product descriptions rephrased. The phrases with traffic are held by Actors with fifteen thousand monthly users, and Store ranking weights usage heavily enough that no wording gets a new Actor past them.

That is not a complaint. It is the shape of the problem now that the filter is gone: the Store shows you, but it does not bring people to you. What does is outside the Store — a Google result for a specific need, an article, a link someone shares. The Store is where they land, not where they start.

The checklist I wish I had on day one

Run the two requests. If includeUnrunnableActors=true is the only way your Actors appear, stop everything else and go to Actors → Insights → Payouts.
If your ID does not carry your name in the script your Apify account uses, do not retry the portal. Write to support with the reason, ask for a human, and ask for manual verification.
Once you are listed, do not read a first-place rank as demand. Add up the 30-day users of what ranks alongside you. If the total is two digits, you have won an empty room.
Record the day you became visible. Everything you measured before it was measured against a wall.

My eleven Actors are at apify.com/northbell. As of the day this was written, their combined 30-day users are ten — nearly all of it Apify's QA. That number is the baseline, and it is the first one I have been able to trust.

northbell builds honest web scrapers on Apify. If a derived number has an error margin, the margin ships with it.

My scraper returned 660 jobs. There were 880. Nothing in the output said so.

northbell — Tue, 01 Sep 2026 11:36:14 +0000

The first version of my LinkedIn company-jobs Actor looked like it worked. Point it at a company, get back every open role. The dataset had hundreds of rows, the fields were populated, the run finished green.

It was returning about three quarters of the jobs, and a different three quarters each time.

Worse, I had built change tracking on top of it — "these 12 roles opened since your last run, these 5 closed" — and that feature was manufacturing closures out of nothing. One test run reported 113 jobs closed and 117 opened, 75 seconds apart.

Nothing in the output distinguished this from a correct answer. That is the specific failure I want to write about, because "did my scrape actually finish?" turns out to be a question you can measure, not just hope about — using a method that ecologists use to count fish.

LinkedIn's public job search does not paginate

The public endpoint takes a start offset. I assumed start=0 gives you jobs 1–10, start=10 gives you 11–20, and so on.

It doesn't. It gives you ten jobs. Which ten is not stable.

I ran the same query twice and compared the returned job IDs:

3 pages deep: 33% of the IDs were different
10 pages deep: 61% of the IDs were different

So a straightforward loop — walk start from 0 upward until a page comes back short — does not collect a company's roster. It collects a sample of it. On a small company the sample happens to be everything. On a large one it isn't, and nothing tells you which case you're in.

Two things were broken by this, and only one of them was obvious.

The obvious one: change tracking. If run A samples 660 of 880 and run B samples a different 660, roughly 220 IDs are in A and not in B. Those get reported as closed. They were never closed. They were never absent — they just weren't in this draw.

The one I nearly missed: the basic listing was wrong too. Not the fancy feature — the core product. I was selling "every open role at this company" and shipping a list missing a quarter of it, with no indication that anything was missing.

Two stopping rules that don't work

My first attempt at "am I done" was the obvious heuristic: stop when four consecutive requests add nothing new.

Here's why that's not a measurement. Say the company has 700 roles and I currently hold 650. Each request returns 10 roles drawn from the pool. The chance that all ten are ones I already have:

// P(a page adds nothing) with 650 of 700 held, 10 drawn:
(650/700) ** 10   // ≈ 0.478

Nearly half. Four in a row happens about 5% of the time at that exact point — and more often as you get closer. So the rule does fire eventually. It just fires at a completeness level that depends on the fraction you already hold, which is precisely the unknown you were trying to determine. The stopping rule is circular: it tells you you're done by assuming you're nearly done.

My second attempt went the other way: collect the roster twice and only accept it if the two passes match exactly. Rigorous, and useless. Two random samples of a large pool are essentially never identical. The condition never fired, so the feature never produced a number, so I had built an elaborate way of saying "I don't know."

The answer was in the data I already had

The thing that unstuck me was reframing what those repeated draws are. They aren't retries. They are samples.

Ecologists count fish in a pond by catching some, tagging them, releasing them, then catching a second batch and seeing how many carry tags. Few tags means a big pond. That's Lincoln–Petersen:

N ≈ (size of sample A × size of sample B) / (number in both)

I already had two samples — I just had to stop merging them into one bucket. So the Actor now splits its own requests into two sets, tracks them separately, and estimates the total from their overlap:

export function estimateTotal(sampleA, sampleB) {
  const a = sampleA.size, b = sampleB.size;
  let overlap = 0;
  for (const id of sampleA) if (sampleB.has(id)) overlap += 1;
  return { total: Math.round((a * b) / overlap), a, b, overlap };
}

Then coverage is just what I hold divided by what I estimate exists, and "how many am I still missing" falls out of the same number.

Here is a real run against a company page, 800 requests, recording the unique jobs held at each point:

requests	unique jobs
40	345
80	617
160	795
240	844
320	867
400	878
520	880
520–800	880 (zero new in 280 requests)

Estimated total from the overlap: 880. Actual point of exhaustion: 880.

The pool wasn't unreachable. I had been stopping at 93 requests.

Bug 1: I split the samples along the wrong axis

The first version of the split used request parity — odd-numbered requests into sample A, even into sample B. The estimate came back at 1,334 for a company with 880 roles. Fifty percent too high, and confidently so.

The reason took me longer than it should have. The offset also advances once per request. So odd requests were looking at odd pages, and even requests at even pages. The two samples were drawing from different parts of the pool, which makes their overlap unnaturally small, which inflates N. Lincoln–Petersen assumes both samples come from the same population; mine didn't.

The fix is to split by sweep — one full pass over all offsets — rather than by request:

const sweep = Math.floor(r / pages);
const start = (r % pages) * PAGE_SIZE;
...
(sweep % 2 === 0 ? evenSample : oddSample).add(j.jobId);

Now both samples cover every offset. The estimate landed on 880.

Bug 2: the formula collapses, and it collapses quietly

This one reached production before I caught it, and it produced the worst kind of output: a confident, plausible, wrong number.

The two samples were 20 and 681 items, with an overlap of 20. Run the formula:

N = 20 × 681 / 20 = 681

The estimate equals the count I already had. Coverage: 100%. The Actor concluded it had collected everything, stopped, compared against the previous run, and reported 199 jobs closed. None had closed.

When one sample is entirely contained in the other, the formula returns the larger sample. It isn't an error — it's what the math says when your "second sample" is really just a subset of the first. And an overlap of 20 comfortably passes a naive "overlap must be at least 10" check.

So the guards aren't about size, they're about shape:

const MIN_SAMPLE = 20;
const MIN_BALANCE = 0.5;   // smaller ÷ larger

if (Math.min(a, b) < MIN_SAMPLE) return { total: null, reason: 'sample-too-small' };
if (Math.min(a, b) / Math.max(a, b) < MIN_BALANCE) {
  return { total: null, reason: 'samples-unbalanced' };
}
if (overlap < 10) return { total: null, reason: 'overlap-too-small' };

Plus one more condition outside the function: don't trust an estimate until both samples have completed at least one full sweep. The degenerate case above happened when the run stopped after one sweep plus two requests, leaving one sample with two requests' worth of data.

Note what total: null means here. It is not zero and it is not 100%. It means I cannot tell, and it travels to the output with a reason attached.

Bug 3: I fixed the wrong variable

My first instinct on the collapse was to raise the minimum sample size from 20 to 50. Bigger samples, less degeneracy, done.

That broke companies with about 25 open roles. Too many to fit in one page (10 per request), too few to ever reach a 50-item sample. Mid-sized companies became undecidable — the Actor could neither confirm nor estimate, so it withheld everything.

The collapse was never caused by absolute size. It was caused by imbalance — 20 versus 681. Raising the floor treated a symptom and disabled a whole class of correct cases. So the absolute minimum went back down to 20, and the real protection lives in the balance ratio and the two-sweep rule.

Small companies get a separate path entirely, based on a fact rather than an estimate: if no response ever filled a full page, there is no second page, and what I have is everything. Measured page churn at one page deep is 0%, so this is safe — and it matters, because mark-and-recapture needs an overlap of 10+ to work at all, which means the smaller the company, the less able it is to estimate. Exactly backwards, if you don't handle it separately.

Bug 4: my own request cap looked like a wave of layoffs

A production run reported closed: 20. Nothing had closed. I had changed the per-company cap from 30 to 20 between runs.

Truncate a list, compare it to an untruncated one, and the rows you didn't collect appear as rows that vanished. Same for changing a filter: narrow the location, and everything outside it "closes."

So before any comparison, the Actor now checks whether the two observations are comparable at all, and refuses to produce numbers when they aren't:

if (ctx.truncated) return blank('truncated-this-run');
if (prevPoint.truncated) return blank('previous-run-was-truncated');
if (ctx.filterKey !== prevPoint.filterKey) return blank('filters-changed');

opened and closed come back as null with a reason, instead of numbers that look real.

What the Actor reports now

Every company row carries the measurement alongside the data:

openJobs: 880
estimatedTotal: 880
coverage: 1
estimatedMissing: 0
changeMarginOfError: 0
collectionComplete: true
requestsUsed: 523

changeMarginOfError is the one I'd argue hardest for. If this run missed m roles and the previous run missed m', the apparent change is muddied by up to m + m'. A run at 99% coverage on 880 roles is missing about 9 — so a reported change of ±18 is inside the noise. That is why the target coverage for change tracking defaults higher than the target for just listing jobs. A number without its error bar invites a conclusion it can't support.

One more measurement, since it costs nothing to state: LinkedIn returns HTTP 400 for start values of 1000 and above. In an 800-request experiment, exactly 20 requests failed — 1000, 1010, … 1190 — and every other request succeeded. The offsets below 1000 are enough, because each pass returns a different draw. Circling 0–990 repeatedly reaches everything.

What I'd take from this

"No new results" is not evidence of completeness. It's evidence about the fraction you already hold, which is the thing you were trying to measure. Any stopping rule built on it is circular.

Repeated requests are samples, not retries. The moment I stopped merging them into one bucket, the data I already had answered the question I thought I needed a new experiment for.

A degenerate formula returns a number, not an error. 20 × 681 / 20 = 681 is arithmetically fine and semantically garbage. If a formula has a collapse mode, guard its shape — and make the guard return "I don't know" rather than a default.

Silence and completeness are indistinguishable unless you report the difference. Before this work, "660 jobs" and "880 jobs" looked identical from the outside. Now the row says which one it is, and how sure it is.

The estimator, the guards and the tests that pin down each of these four bugs are at northbell-dev/honest-scraping — no dependencies, npm test runs the 40 tests with nothing to install.

The Actor is LinkedIn Company Jobs Scraper. It reads only public pages — no login, no cookies, and the request headers are a frozen object that cannot carry one.

northbell builds honest web scrapers on Apify. If a derived number has an error margin, the margin ships with it.

I scraped 89 job postings to find out how my salary parser was lying to me

northbell — Tue, 01 Sep 2026 10:50:34 +0000

I shipped a LinkedIn jobs scraper that returns salary as structured numbers — salaryMin, salaryMax, currency, and a yearly figure so an hourly warehouse job and a director role sit on the same axis. The whole point was to make "show me jobs paying over $150k" a question you can actually ask.

Then I ran it against 89 real postings and found six separate ways it was wrong.

None of them threw an error. Every single one produced a confident-looking number. That is the part worth writing about: a salary parser doesn't fail loudly, it fails plausibly, and the only way to find out is to sit down with real postings and check them one by one.

Here's what I found, in the order I found it, including the one where the platform had already solved my problem and I broke it by "fixing" it.

Why job descriptions are hostile to salary parsing

My first assumption was that the salary would live in a structured field. It doesn't.

I pulled 64 postings across eight roles and countries — software engineer in the US, registered nurse in California, retail associate in Texas, truck driver, marketing manager in New York, software developer in the UK, data analyst in Canada, accountant in the US — and looked for where the money actually was:

The top-card salary box: 3 out of 64
JSON-LD baseSalary: 0 out of 64

So the salary is in the free-text description, mixed in with every other dollar figure a company might mention. And companies mention a lot of dollar figures:

"We're a Series A company with $20M+ raised from Headline, Village Global"
"serving an $800 billion market"
"New Hire Bonus: $3,000 after 90 days"
"[$5,000] signing bonus available to external hires"
"Annual maximum out of pocket $3,000 per individual / $7,500 per family"
"we have grown from $1 million in sales in 1983 to more than $500 million today"

Not one of those is a salary. A naive regex that grabs $[\d,]+ returns all of them.

My first version had a plausibility gate for exactly this reason: normalize any candidate to a yearly figure first, and if it lands outside $8,000–$2,000,000, it isn't a wage. That one rule kills $20M raised, $800 billion market, and $1 million in sales instantly, because nobody is paid $800 billion a year.

It also killed a real salary, which is where the trouble started.

Bug 1: the bonus in the next sentence poisoned the salary

Here's a real posting:

Salary Range: $46.00 - $50 per hour   New Hire Bonus: $3,000

My parser rejected it. Not the bonus — the salary.

The reason: I was testing my noise words (bonus, signing, raised, market) against a window of context around the candidate. $46.00 - $50 per hour had "New Hire Bonus" sitting 8 characters to its right, so the whole match got thrown out.

The fix was to stop treating context as a blob and start treating it as directional. A number's label is what comes before it. What comes after belongs to the next number:

// Drop it only if this number's own label is noise.
// Whatever sits on the right belongs to the next number, not this one.
if (NOISE_BEFORE.test(left)) return null;

NOISE_BEFORE is anchored to the end of the left context:

const NOISE_BEFORE = /(bonus|signing|sign[-\s]?on|relocation|referral|stipend|allowance|raised|funding|valuation|revenue|market|worth|in\s+(sales|arr)|series\s+[a-e])\b[\s:_*\-]*$/i;

The $ anchor is the whole trick. "Bonus:" immediately before a number means that number is a bonus. "Bonus" thirty characters later means nothing about this number at all.

Once I made that change, the pay range came back and the $3,000 bonus stayed rejected — which is exactly right.

Bug 2: a following word's first letter became a magnitude suffix

This one was invisible until I checked a number I had no reason to suspect.

1st year compensation plans range from $55,000-$95,000 Medical, dental, life

Parser said: no salary found.

I instrumented the candidate loop and printed what it actually matched:

raw=|$55,000-$95,000 M|

There it is. My money regex ended with [kKmM]? to catch $150K and $20M. The word after the range was Medical. The regex ate its first letter as a magnitude suffix, read $95,000 M as ninety-five billion, and the plausibility gate — correctly, given what it was handed — threw the whole thing away.

A parser that turns $95,000 into $95,000,000,000 and then silently drops the job is worse than one that crashes. The job just quietly isn't in your results.

The fix is a negative lookahead: a k or m only counts as a suffix if a letter doesn't follow it.

// k/m is a magnitude suffix only when it follows the digits AND no letter follows it,
// so "95,000 Medical" keeps its M.
const NUM_RE = String.raw`\d[\d,]*(?:\.\d+)?\s?[kKmM]?(?![A-Za-z])`;

Regression test, because I never want to relearn this:

test('regression: does not eat a following word initial M/K as a suffix ($95,000 Medical)', () => {
  const p = ok('compensation plans range from $55,000-$95,000 Medical, dental');
  assert.equal(p.salaryFound, true);
  assert.equal(p.salaryMax, 95000); // must not read "$95,000 M" as 95,000,000,000
  const q = ok('base pay $80,000 Monthly meetings and $18/hr overtime');
  assert.equal(q.salaryFound, true); // does not eat the M of "Monthly"
});

Bugs 3 and 4: ranges that collapsed to a single number

Two postings looked fine in the output. They weren't.

Salary Range: $81,000 USD - $105,000 USD    →  min 81000, max 81000
expected hiring range is $36.79/hr - $58.50/hr  →  min 36.79, max 36.79

Both reported the bottom of the range as if it were the whole range. If you filter on "pays at least $100k", the first job is wrongly excluded. Quiet, plausible, wrong.

The cause was the same in both cases: my range pattern was positional, NUMBER separator NUMBER, and both postings put something between the first number and the separator — a currency word in one, a unit in the other. The pattern stopped at the first number and treated it as a lone value.

I could have kept patching the pattern with more optional groups. Instead I stopped matching positions and started counting numbers:

// Take at most two numbers out of raw. A positional range regex breaks as soon as a unit
// or currency word sits between them ("$X/hr - $Y/hr", "$X USD - $Y"), so extract the
// numbers instead and let the count decide.
const nums = [...raw.matchAll(/(\d[\d,]*(?:\.\d+)?)\s?([kKmM])?(?![A-Za-z])/g)]
  .map((m) => toNumber(m[1], m[2])).filter((v) => v != null);

Two numbers inside one money token means a range. One means a single value. It stopped mattering what was sitting between them.

That refactor immediately exposed bug 4: C$90,000 - C$110,000 also collapsed, because a leftover separator check didn't recognise C$ after the dash. Deleting the check fixed it — the tokenizer already guaranteed what the check was testing for.

Bug 5: the insurance benefit that looked like a director's salary

An electrician job came back at $200,000 a year.

Electricians in that market make about $80k. The parser had found this:

Annual maximum out of pocket $3,000 per individual / $7,500 per family
... Health Insurance (In-network) $1,000 individual deductible
... Life Insurance plan with coverage up to $200,000

The word Annual was doing the damage. My period detector saw it, concluded "yearly", and the plausibility gate happily accepted $200,000 as a yearly wage. It is a plausible salary. It is not this job's salary — it's an insurance payout ceiling.

The fix was another directional rule, this time for benefits vocabulary:

// Benefits and insurance words disqualify a number only when they sit on the LEFT,
// the label side. "out of pocket $200,000" -> drop it.
// "salary $120,000 plus health insurance" -> the label is salary, so keep it.
if (BENEFIT_NEAR.test(left)) return null;

I wrote it as a both-sides check first and caught myself: base salary $120,000 plus health insurance and 401k match is an extremely normal sentence, and a both-sides rule throws that real salary away. Left-side only keeps it.

Bug 6: "40 hours per week" is not a pay period

A dental hygienist posting listed both a full-time salary and a part-time rate:

Full time hours-40 hours per week salary-$60,428.52.
Part-time-work up to 29 hours per week, $45/hour.

The parser returned the part-time hourly rate and ignored the full-time salary entirely.

Why: $60,428.52 had "hours per week" sitting in its left context. My period detector read per week, called it a weekly wage, multiplied by 52, got $3.1M, and the plausibility gate threw it out. The part-time rate survived, so the output looked complete.

Schedule language and pay-period language use the same words. So I strip schedule phrases before reading the period:

// A work schedule ("40 hours per week", "5 days a week") is not a pay period, so remove it
// from the context first. Leaving it in makes the $60,428 in "hours per week salary-$60,428"
// look like a weekly wage.
context = String(context).replace(/\b(hours?|hrs?|days?|shifts?)\s*(?:\/|per\s|a\s|an\s)\s*(?:week|day|month|hour|year)/gi, ' ');

With that, the full-time salary came back at $60,429/year — and Base pay $800 per week, a genuine weekly wage, still parses as weekly. Both are in the test suite.

What I got wrong about being wrong

I want to be honest about a seventh thing, because it wasn't a parser bug — it was me.

Separately from the salary work, I was making my Actors usable as tools for AI agents through Apify's MCP server. I compared my input schemas against what the MCP tools/list response exposed, and noticed that required didn't appear in the tool definitions. Three Actors, same result. I concluded the agent never learns which fields are mandatory, edited six Actors to prepend "Required." to those field descriptions, and deployed all six.

Then I looked at the description text an agent actually receives:

**REQUIRED** LinkedIn job URLs (https://www.linkedin.com/jobs/view/...)

Apify's MCP server already injects **REQUIRED** into the description of required fields. The information was reaching the agent the whole time. My "fix" produced **REQUIRED** Required. LinkedIn job URLs... on three live Actors before I caught it. I reverted all six and redeployed.

The mistake wasn't the edit. It was that I checked the field I expected to carry the information instead of the payload the consumer actually receives. Which is the same mistake as bug 5, one level up: I trusted a signal (Annual, required) without checking what it meant in context.

What the parser refuses to do

The finished version outputs a number only when it can defend it:

Normalize first, judge second. Everything becomes a yearly figure before the plausibility gate. Hourly × 2080, monthly × 12. $800B market never reaches the output because $800B/year is not a wage.
A number needs its own label. A bare $600,000 in a sentence about project value is not pay. Something must mark it: a unit (per hour, /yr), a pay word before it (salary, pay range, compensation), or a period word in context.
Direction matters. Labels sit to the left. Words to the right belong to the next number.
When it can't tell, it says so. salaryFound: false and no invented figure.

And every run reports the fill rate, because the honest answer to "why did I only get 74 jobs" is usually "because 6 in 10 US postings state pay, and the rest didn't write a number down":

examined: 12   withSalary: 8 (67%)   returned: 7
droppedBelowMin: 1   droppedNoSalary: 4
annualSalaryMedian: 250000   range: 135000-350000

That 67% matched my earlier measurement of 63% across a different sample, which was the first thing that made me trust the pipeline.

If you're building something similar

Three things I'd do from the start next time:

Build the corpus before the parser. I spent an afternoon collecting money phrases with their surrounding context from eight different job types. Nurses and retail workers are quoted hourly, engineers yearly, accountants monthly, and each format breaks a different assumption. If I had only tested on software jobs I would have shipped four of these six bugs.

Print what your regex actually matched. Bug 2 was undiagnosable from the output — the field was just empty. One line dumping raw=|$55,000-$95,000 M| made it obvious in seconds.

Write the regression test in the same commit as the fix. All six are in the suite. When I later replaced the range logic entirely, those tests caught a fifth bug I'd introduced in the process, before it shipped.

The parser and every regression test in this post are at northbell-dev/honest-scraping — no dependencies, npm test runs the 40 tests with nothing to install.

The Actor is LinkedIn Jobs Salary Data. It reads only public pages — no login, no cookies, and the request headers are a frozen object that cannot carry one.

northbell builds honest web scrapers on Apify. If a derived number has an error margin, the margin ships with it.