Building a Lead Scanner to Filter Out Stale Freelance Jobs

I run my web development shop, Dream Futures Tech, right here in Pompano Beach, Florida. For a long time, finding solid freelance gigs meant burning an hour or more every single morning sifting through the same old job boards. Most listings were either a week old, poorly disguised full-time recruiter pitches, or posted by agencies looking for sub-contractors at steep discounts.

Instead of dealing with the noise, I built a lightweight lead-scanner that runs on my home automation server and checks a handful of tech-focused boards every hour. The core problem wasn't gathering the posts—it was figuring out how to filter them effectively. In my experience, a combination of a strict per-board recency window and a small set of deterministic rules gives me a clean, actionable list every morning without fail. Here is how I set it up for my own business.

The Hourly Fetch and Recency Window

The first step was stopping the flood of outdated posts. Many boards keep listings active for weeks after they have been filled, simply because clients forget to take them down. To combat this, my script pulls posts and immediately evaluates their timestamps against a strict recency threshold.

Depending on the board's volume, my window is usually set between 12 and 24 hours. Anything older is dropped right at the database layer. This ensures that when I sit down to look at potential projects, I am only seeing things published recently enough that the client is likely still monitoring their inbox.

function isWithinRecencyWindow(postDate, maxHours = 24) {
  const now = new Date();
  const diffMs = now - new Date(postDate);
  const diffHours = diffMs / (1000 * 60 * 60);
  return diffHours <= maxHours;
}

This simple check eliminated about half of the junk I used to wade through. If a post misses this window, it never even makes it into my review queue.

Normalizing Variable Board Layouts

Every job board uses a different HTML structure or API format, which means scraping them directly requires constant maintenance. To keep my code from breaking every time a site tweaks its CSS classes, I wrote small, isolated adapter modules for each source.

Each adapter's only job is to normalize raw board data into a standard schema: title, company name, URL, posted date, and raw description text. If a board changes its markup, I only have to rewrite a ten-line selector function instead of reworking the entire filtering pipeline.

// Example adapter structure
function parseBoardItem(rawElement) {
  return {
    title: rawElement.querySelector('.job-title')?.innerText.trim(),
    url: rawElement.querySelector('a')?.href,
    date: rawElement.querySelector('time')?.getAttribute('datetime'),
    body: rawElement.querySelector('.description')?.innerText.trim()
  };
}

By keeping these adapters decoupled from the filtering logic, the system remains stable even when upstream sites update their layouts.

Applying Deterministic Filtering Rules

Once the posts are normalized and checked for recency, they pass through a series of deterministic rule checks. My goal here is to strip out full-time listings, enterprise roles requiring five rounds of interviews, and agency job postings disguised as direct client work.

I use simple regular expression checks against the title and description body. For instance, if a listing contains phrases like "W2 only", "full-time benefits", or "agency experience required", it gets flagged and moved to an ignore list.

const exclusionPatterns = [
  /full-time/i,
  /w2 only/i,
  /years of experience required/i,
  /must be local to/i
];

function passesRules(post) {
  const text = `${post.title} ${post.body}`;
  return !exclusionPatterns.some(pattern => pattern.test(text));
}

These rules aren't fancy machine learning models; they are plain, predictable checks that I can update whenever a new type of unwanted listing slips through. If a post triggers any rule, it is dropped instantly.

Storing and Reviewing the Cleaned Queue

After passing through the recency check and the rule filters, the remaining leads are saved to a local SQLite database running on my server. I keep a simple local web interface that lets me view the fresh queue in a clean, distraction-free format.

Instead of browsing three or four different tabs every morning, I open one local page and scan the handful of real, contract-focused opportunities that made the cut. Building this tool took a few hours on a weekend, but it has saved me countless hours of manual searching and keeps my pipeline moving.

I build sites and automations for small businesses at dreamfuturestech.com.