Cheap Filters First, LLM Last: Running an AI Matcher Inside a Cron Job

I run a small SaaS called Upwork Scout. It watches Upwork around the clock and emails you only the jobs that actually fit you. The pitch takes one sentence. The engineering is a long argument with your own billing page.

I have written before about the scraping half of that argument. The short version: naive job alert tools scrape once per user, so cost grows linearly with signups and you go broke on your own success. I inverted it into one shared scrape per category per cycle, and scraping cost now tracks the number of active categories instead of the number of users.

Then I added the part people actually pay for, AI matching, and immediately created a second bill with worse scaling properties than the first one. Scraping cost tracks categories. Model calls track jobs times users. That multiplication is where most "we added AI" features quietly die.

Here is the shape I landed on, and the four rules that got me there.

Rule 1: the model never sees a job a boring filter would have rejected

Every user has hard filters. Around twenty of them: job type, budget floor and ceiling, hourly range, experience level, project length, workload, max proposals, whether the client is payment verified, minimum client lifetime spend, minimum client rating, hire counts, country allow and deny lists, keyword includes and excludes.

None of that needs a language model. It is a pure function over two structs, it runs in memory against the shared job stream, and it costs nothing:

export function jobMatchesFilters(job: JobDoc, f: UserFilters): boolean {
  const catHit = f.categories.some((c) => job.categories.includes(c));
  // ... twenty more boring checks
  return true;
}

The AI stage only ever runs on jobs that already survived this. In a normal cycle that cuts the candidate set down by an order of magnitude before a single token is spent. The most effective prompt optimization I have ever written was an if statement.

There is a subtlety in that filter that took me a while to get right. Upwork does not always expose the data a filter needs. Sometimes the client's total spend is missing, sometimes the rating is absent, sometimes an older stored job predates a field I added later. My first version rejected on missing data, which meant users silently stopped getting alerts and I could not reproduce it. Now the policy is written at the top of the file and applied consistently: filters only reject on positive evidence. Unknown data passes the filter, and the email and dashboard flag the unknown instead of hiding the job. Let the human decide what the scraper could not tell them.

Small related detail from real data: Upwork mixes full country names and ISO3 codes in the same field. I have seen "United States", "USA", "CAN", "NLD" and "United Kingdom" all in one dump. If you filter countries by string equality, your United States filter drops every job tagged USA and you will never notice, because the failure mode is silence. I keep an alias table.

Rule 2: cache the verdict, not the prompt

Prompt caching is a fine optimization. It is not the one that mattered here.

The scan runs every fifteen minutes. Without caching, the same user would be re-scored against the same job ninety six times a day, and each of those calls would return the same answer, because neither the job nor the profile changed. So the verdict is the cache key, and the cache key is the pair:

matches/${uid}_${jobId}  ->  { score, reason, scoredAt }

A job is never scored twice for the same user. Ever. That single rule detaches model spend from scan frequency entirely. I could scan every five minutes and the model bill would not move, because scoring volume is bounded by new jobs times interested users, not by how often the cron fires.

It also gave me something I did not plan for: a permanent record of why each user was shown each job. When someone emails me asking why they got a particular alert, I can answer with the exact one line reason the model wrote at the time, instead of guessing or re-running a prompt against a model that may have drifted.

Rule 3: cap it per user, and degrade instead of failing

Free plan users get 150 AI scores per day. The counter lives on the user document with the date attached, so it resets naturally without a cleanup job:

let aiToday = u.aiScoresToday?.date === today ? u.aiScoresToday.count : 0;

The important part is what happens when someone hits the cap. The tempting move is to skip the user for the rest of the day. That is the wrong call, because the product promise is alerts, not scored alerts. When the budget is gone, the pipeline falls back to filters only for the remaining jobs. The user gets slightly less precise alerts for a few hours instead of silence.

That is the general principle I keep relearning with AI features: the model is an enhancement layer over something that already works, not a load bearing wall. If your feature has no defined behaviour for "the model did not answer", you have not finished building it.

Rule 4: the scorer returns null, it never throws

The scoring function has one hard contract. No API key, network error, refusal, malformed JSON, whatever, it logs and returns null. Nothing it does can take down a scan cycle that is also delivering email for everyone else.

The consumer side is one line, and it is the line I am most careful about:

// Gate on threshold only when we actually have a verdict; AI failure => filters-only.
if (verdict && verdict.score < matching.threshold) continue;

If the verdict is missing, the job passes. Being wrong in a direction that sends a slightly off job beats being wrong in a direction that sends nothing, because a user who gets a mediocre alert shrugs, and a user who gets no alerts churns.

The prompt is mostly a scoring rubric

The prompt itself is short and unglamorous. It asks for exactly two things: an integer from 0 to 100, and one sentence of at most eighteen words naming the decisive factor.

The part that earned its keep is the rubric. Without explicit bands, scores cluster in a mushy 70 to 85 range and a threshold becomes meaningless. So I define the bands and tell the model to be decisive:

90-100 perfect fit: core expertise AND exactly the kind of work they want
70-89  strong fit: clearly within their skills, only minor mismatches
50-69  decent fit: plausible, but notable skill gaps
30-49  weak fit: tangential; they could stretch to it but shouldn't
0-29   poor fit: wrong domain, or serious red flags

Default alert threshold is 55, which users can move.

Two other things I would repeat on any project like this:

Use structured outputs. The response is constrained by a JSON schema with score as an integer and reason as a string. I still clamp and round on the way out, because trusting a schema and validating anyway costs nothing, but I stopped writing regex to dig JSON out of prose.

Make the explanation part of the product. The one sentence reason is not debug output. It goes straight into the alert email under the job, and the score sorts the email so the best fit is at the top. A number by itself asks the user to trust you. A number plus "strong Retell and n8n overlap and a verified client, but the budget sits below your $50/hr floor" lets them check your work in two seconds. That sentence does more for retention than any accuracy improvement I could buy by upgrading the model.

Speaking of which: the default model is Haiku 4.5, and the model name is an environment variable. On my own estimate a score costs somewhere around a tenth of a cent, which is the whole reason a free tier can exist. If match quality ever becomes the binding constraint instead of cost, I change one env var and every future score is smarter. I have not needed to.

What I would tell someone adding an LLM to a background job

The question that occupied me for a week was not which model. It was how many calls I could avoid making.

Deterministic filters first, because they are free and they are honest. Cache on the natural identity of the work, not on the request. Give every user a hard ceiling and define what happens underneath it. Let the model fail into the non-AI path rather than into an error.

Do that and the model becomes what it should be: the last, smallest, most expensive step in a pipeline that already worked without it.