I built an actor that pulls job listings from five different job boards in one run, with a min/max salary filter as one of the inputs. It seemed like the easy part of the whole build.
The bug
The first live test used a "nurse" search in London with min_salary=30000. Zero results came back, even though the raw fetch (before filtering) had pulled in over 30 real nurse listings.
Turned out every single one of them had a salary, just not in the form I expected. One job board listed pay as "£13.78 - £15.20 per hour". My filter was comparing that 15.20 directly against 30000. An hourly rate will basically never clear a filter written with an annual number in mind, no matter how good the job actually pays.
Why it's not a formatting problem, it's a units problem
The instinct is to treat this as a text-parsing issue: extract the number, done. But 15.20 and 30000 aren't the same kind of number. One is dollars-per-hour, the other is dollars-per-year. Comparing them directly is like comparing a speed in km/h to a distance in km — the parser was working fine, the comparison itself was meaningless.
The fix was to annualize every parsed figure at extraction time, based on whatever period word appeared next to it (hour, day, week, month, year):
_ANNUALIZE = {
"hour": 2080, "hr": 2080, # 40h/week * 52 weeks
"day": 260, # ~5 working days/week * 52 weeks
"week": 52,
"month": 12,
"year": 1, "yr": 1, "annum": 1,
}
lo, hi = min(a, b) * multiplier, max(a, b) * multiplier
The original wording is still kept around separately for display (so a human still sees "£13.78 - £15.20 per hour", not a slightly surreal "£31,241/year"), but the filter itself only ever compares annualized numbers.
A second one, in the same feature
Testing the same filter against a different source turned up a second, smaller version of the same lesson. One job board tags listings with "full time" (a space). My keyword vocabulary was checking for "full-time" (a hyphen). Same concept, two spellings, one silent miss — a real "Full-Time" job simply never matched.
_JOB_TYPE_RE = {
jt: re.compile(re.escape(jt).replace("\\-", "[\\s-]"), re.IGNORECASE)
for jt in _JOB_TYPES
}
Neither bug threw an exception. Neither one showed up in a code review. Both only surfaced by running the filter against real listings from real sources and noticing the result count didn't match expectations — which is the actual reason this project has fixtures captured from live responses rather than hand-written mocks.
Actor page: https://apify.com/0xgollum/global-job-aggregator
Top comments (0)