The Model Proposes, Rust Verifies: Triaging Job Postings with a Local Gemma

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

A job posting said it offered “a blend of remote flexibility and in-person collaboration.” My tool called it remote.
For someone applying from Nigeria, that isn't a small mistake. It can mean spending time on a role that isn't actually remote.
That wasn't the only problem I found while building JobKit. Some of the bugs were in my Rust rules. Others were in my reference labels—the answers I had prepared to judge whether the tool was correct.
I built JobKit to make the first read of a job posting faster. What I didn't expect was how much time I'd spend checking my own assumptions.
The model proposes, Rust verifies.

What I Built

Reading a job posting properly takes me about 17 minutes. Most of that goes on three boring questions: is it actually remote, what hours does it want, and how many years of experience does it ask for. I live at UTC+1, so those answers decide whether the posting is even for me.

JobKit is a Rust CLI that does that first read. Give it a posting and it tells me whether it's remote, what timezone and years it wants, which skills it names, and whether each skill looks required, optional or just mentioned. Every answer comes with the exact line it was found on, so I can check it in seconds.

A small local model, Gemma 3 4B, only names skills. Rust verifies and decides everything else. It runs on my laptop, and my job search never leaves it.

Demo

Code

Lymah123 / jobkit

A local, evidence-first triage tool for job postings, written in Rust.

JobKit

A local, evidence-first triage tool for job postings, written in Rust.

You give it a job posting and a short profile of yourself. It tells you whether the posting is remote, what timezone window and years of experience it asks for, which skills it names, and whether each skill looks required, optional or just mentioned. Every answer comes with the exact line of the posting it was found on, so you can check it in seconds.

A small local model (Gemma 3 4B through Ollama) only names skills Rust verifies and decides everything else. Nothing leaves your machine.

The model proposes, Rust verifies.

How it works

1. Rules (no model). Plain regex rules read remote status, minimum years, UTC timezone windows and flag work-authorization or location sentences. Remote has three states: true, false (explicitly not remote, onsite, hybrid or in-person), or unknown.
2. Model. Gemma lists the…

View on GitHub

How I Built It

Model: Gemma 3 4B (open weights), run locally through Ollama in WSL.

Language: Rust, with regex, serde ', andureq`.
Built with an AI assistant: I used Claude to help with debugging. I ran the code, collected and hand-labelled the postings, evaluated the results, and decided what to fix.

The model doesn't get the final say on whether a skill is real. Rust checks the posting itself:

If Rust finds no matching term in the posting, it rejects the proposed skill. If it finds one, it keeps the skill with a line of evidence for the user to inspect.

This is the core idea, simplified: the model proposes; deterministic code checks the evidence.

The rule: the model proposes, Rust verifies.

1. The model names skills.
2. Rust accepts a skill only if it appears in the posting as a whole term (aliases like Postgres and PostgreSQL allowed) and attaches the line it came from as evidence. A skill the posting never mentions is rejected as invented.
3. Rust classifies each accepted skill from the headings and wording around it ("Nice to have", "preferred", "Good to have:").
4. Plain rules read remote status, years and timezone windows, and flag work-authorization sentences. Anything uncertain becomes Review instead of a confident answer.

What went wrong (the interesting part)

1. My first checker rejected 45 of 90 items, mostly for formatting. I had required the model to copy quotes exactly, and a 4B model can't do that reliably. Letting Rust find the evidence fixed it. On my first nine postings the model never named a skill that wasn't there (0 of 73). 2.** My rules started at 78% on remote**, 33% on years and 11% on timezone. The causes were a pattern that never matched "1 year", company history read as a requirement ("~2 years since we started"), and "no remote" counted as remote.
2. My labels were wrong too. The harness checks that each labelled skill appears in its posting. It caught 8 that didn't, plus one marked required that sat under "Nice to Have".
3. A section rule leaked. A "Good to have:" section ran into the stack list that followed it, because one unrecognised heading didn't end it. The heading "US citizenship/visa not required" also demoted real skills.
4. The held-out set humbled me. It found a hybrid job ("a blend of remote flexibility and in-person collaboration") scored as remote, and "12-15+ years" read as 15. It also found "10+ years … including 2+ years in embedded roles" read as 2, a mistake I had made myself when triaging by hand.

Results

Metric	9-posting tuning set (includes 1 synthetic posting)	8 held-out postings
Remote classification	100% (up from 78%)	62%, then 75% after fixes
Minimum years of experience	100% (up from 33%)	75%, then 100% after fixes
Timezone classification	100% (up from 11%)	100%
Skills — loose mode (precision / recall)	0.51 / 0.88	0.32 / 0.37
Skills — strict mode (precision / recall)	0.59 / 0.45	0.33 / 0.27
Unsupported skill proposals caught by the checker	0 of 73	1 of 69

The held-out column is the honest one, and the "after fixes" numbers are no longer held-out, because I patched the rules after seeing those failures. Loose means a skill is required unless every mention is optional. Strict also needs requirement wording or a requirements heading, so stack lists like Backend: Go, Postgres count as mentioned. That trades recall for precision.

Why Does Open Innovation Matter?

• My data stays mine. A job search involves my CV, my location, and which companies I'm looking at. With an open-weight model none of it leaves my machine.
• I could see exactly how it failed. I turned each failure of the small model into a check: invented terms, malformed JSON, repeated keys. That's why no invented skills got through on the set I tuned on.
• I could rerun it for free. I evaluated the whole set many times, and cached model output so re-scoring takes seconds and results are reproducible.
• What I can't claim: I didn't compare against a hosted model, which would likely extract long skill lists better. My result is that a small local model is usable if you build verification around it.

The Lazy Payoff

• By hand: 83 minutes for 5 postings, about 17 each.
• JobKit's rules alone: 0.08 seconds per posting.
• With the model: 43 to 143 seconds per posting, 96 on average over 8. It's slow, but it waits for me and I don't wait for it. Slow and steady.
• My time with the tool: Approximately 2 minute to read and check the output for the same 5 postings.
• Next step (an estimate): 3 of the 8 held-out postings were hard Skips from the rules alone. Skipping the model for those would save about 35% of the model time. I haven't built that yet.

Limits

• 16 real postings plus one test case, all labelled by me, with mistakes I found along the way.
• Gemma 4B under-extracts long skill lists.
• The tool matches terms, not concepts like "distributed systems". "Mentioned in the posting" isn't "required".
• Boilerplate can trip flags. A pay-transparency sentence raised a location flag.
• I never measured whether the apply-or-skip verdict was right.

Prize Categories

Best Use of Gemma. Gemma 3 runs locally as the extraction step, and the whole verification layer exists because of how a 4B Gemma fails.

What's Next

Three of the eight held-out postings were hard Skips under the rules-only evaluation. If JobKit can safely skip model inference for those cases, I estimate it could reduce model runtime by about 35%. That optimization isn't implemented yet.

I built JobKit because job hunting already takes enough time without repeatedly decoding postings that aren't a fit. The immediate goal is to make that first screening step faster and easier to verify—not to pretend a small model can make every application decision for me.

The model proposes, Rust verifies. The next step is making that process cheaper without making it less trustworthy.