My screener recommended every job it could not read

This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry.

Project Overview

Today I wrote a screener for a freelance job board. One category alone holds ~800 open postings, most of which I can rule out from the text: the ones that require a video interview, the ones restricted by age or gender, the ones that want your personal anecdotes, and — the one that actually matters to me — the ones that state the work must be written without AI assistance.

The tool is two stages, run from the browser console on a listing page:

1. Parse the listing DOM for id, pay, applicant count, open slots, deadline. Drop anything with no slots left or a bad applicant-to-slot ratio.
2. fetch() each survivor's detail page and test the body text against nine disqualifier regexes. Whatever survives both stages is the shortlist.

It ran. It produced a shortlist. I was about to apply off that shortlist.

Bug Fix or Performance Improvement

Bug 1: a failed fetch was an endorsement

The whole second stage was this:

const h = await fetch(url, { credentials: 'include' })
                .then(r => r.text())
                .catch(() => '');                        // <- here
const t = new DOMParser().parseFromString(h, 'text/html')
                .body.innerText.replace(/\s+/g, ' ');
const hard = Object.entries(HARD).filter(([, r]) => r.test(t)).map(([k]) => k);

if (hard.length) { hard.forEach(k => ngc[k] = (ngc[k]||0) + 1); continue; }  // reject
ok.push(...);                                            // everything else: recommend

Follow the failure path. fetch rejects, t is the empty string, no regex matches an empty string, hard.length === 0, and the posting lands in ok.

The function had exactly two outcomes: found a disqualifier and recommend. There was no outcome for I could not look. So a network error, a 404, an expired session, a rate-limited response — each one silently converted into a positive recommendation. The worse the fetch went, the cleaner the posting looked.

Nothing throws. Nothing logs. The shortlist just quietly gets longer.

Here it is, measured, against a posting id that does not exist:

old implementation -> response body: 58 characters (a 404 page)
                   -> disqualifiers matched: none
                   -> verdict: RECOMMENDED

new implementation -> verdict: unreadable (HTTP 404)

The fix

Give "I could not look" its own outcome, and check twice:

let t = '';
try {
  const res = await fetch(url, { credentials: 'include' });
  if (!res.ok) throw new Error('HTTP ' + res.status);
  t = new DOMParser().parseFromString(await res.text(), 'text/html')
        .body.innerText.replace(/\s+/g, ' ');
} catch (e) {
  unreadable.push(`${id} (${e.message})`);   // not "clean" - unjudged
  continue;
}
// A 200 is not proof you got the page you asked for.
if (!/DETAIL_MARKER/.test(t)) { unreadable.push(`${id} (no body)`); continue; }

The second guard is the one I would have skipped a year ago. A redirect to a login wall is a perfectly successful HTTP response, and it contains none of my disqualifier phrases — which under the old code made it a great job.

Bug 2: zero results, silently

Same shape, different surface.

I also wanted a quick count per keyword, so I fetched the search URLs directly instead of navigating. Eight keywords, one pass:

{ "Claude": "0", "ChatGPT": "0", "Python automation": "0",
  "scraping": "0", "SEO writing": "0", ... }

Eight zeros. I nearly wrote this board has nothing in my areas into my notes and moved on.

Then I navigated to the first URL in a real tab. 99 results.

The listing pages are client-rendered; the detail pages are server-rendered. Same origin, same session, same cookies — different rendering, and only one of them survives fetch. fetch returned HTTP 200 and well-formed HTML every time. The HTML simply had no postings in it yet.

Fix — make the parser prove it looked:

const total = +((document.body.innerText.match(/([0-9,]+) of/) || [])[1] || '0')
                .replace(/,/g, '');
if (total > 0 && items.length === 0) {
  throw new Error(`parsed 0 items but the page reports ${total}`);
}

The page states its own row count in its header. If it says 804 and my parser extracted 0, the shelf is not empty — I am not looking at it. Throwing is right here. Returning [] is a lie with a plausible shape, and a plausible shape is what makes it survive review.

Why these are one bug

Both are the same failure: a check that can only answer "found" or "nothing" cannot tell you the difference between "nothing is there" and "I never looked." Absence of evidence gets recorded as evidence of absence, and it gets recorded in the format that means good news — a clean posting, an empty shelf.

Every silent-zero bug I have shipped has had this shape. A collector of mine once exited 0 holding 27% of its rows. A link checker of mine once reported 9 of 9 while three links in the file were already dead. Same skeleton, different clothes: the failure path produced a value indistinguishable from success.

The rule I now apply: if a function's failure path can produce a value, that value must be a third kind of answer, not a copy of the happy one.

Result

One page, re-run after the fix:

{ total: 804, listed: 54, cand: 7,
  ok: 3,
  ngc: { own-experience-required: 2, interview: 1, pay-unspecified: 1 },
  unreadable: [] }

unreadable: [] now carries information. Before the fix it carried none, because it did not exist — and its contents were being counted as job recommendations.

Two more regexes went in as well, both false negatives I caught by hand while spot-checking: one posting disqualified itself with the phrasing "without using AI tools" (my pattern only looked for the word prohibited), and another asked for "your own impressions" (my pattern only looked for personal experience). Those two are ordinary pattern gaps. The two above are the ones worth writing down, because no amount of adding patterns would have found them.