DEV Community

Cover image for I built a local Gemma 4 scam checker for freelance gigs, and it has to prove every claim with a quote
Xingrun Wang
Xingrun Wang

Posted on

I built a local Gemma 4 scam checker for freelance gigs, and it has to prove every claim with a quote

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission ๐Ÿ’ธ

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

gig-guard is a scam-and-fit check for freelance gigs. Paste a job post or a client message from Upwork, Fiverr, Freelancer.com, CrowdWorks, Lancers, or a random DM, in English, Japanese or Chinese, and before you reply it tells you:

  • whether it's a scam, and the exact line that gives it away;
  • what the job really pays per hour after the platform's cut (USD, JPY or CNY, with fee presets for the big platforms);
  • what to ask the client, plus a polite reply in the post's language that keeps you on the platform.

It's built around Gemma running on your own machine, so your client conversations never go to anyone's server.

How it started. I'm a backend developer living in Japan. This month I started picking up side gigs on Upwork, Freelancer.com and the Japanese sites CrowdWorks and Lancers. In my first week, one client wanted me to "add us on LINE to answer the screening survey". That's a Japanese messaging app, and moving off the platform is the classic first step of a scam. Another wanted four pages of real production work as an unpaid "trial". Then I noticed the same playbook in English: "Upwork fees are high, let's continue on WhatsApp", "send a free sample first", "we'll mail you a check for equipment". Same scams, different languages. That's why gig-guard isn't tied to one site or one language.

Who would pay for it. Anyone freelancing on these marketplaces: tens of millions of people, many of them working in a second language, exactly where scam messages are hardest to read. A few dollars a month for a checker that never uploads your messages is an easy sell. Inference is local, so it costs nothing to run, and the JSON output means a browser extension or a bot can sit on top of it.

Demo

โ–ถ Try it in your browser: https://honru09.github.io/gig-guard/
Pick a sample (Upwork / Fiverr / Chinese / Japanese), or paste your own. The rule checks and pay math run right in the tab, using the same Python package through Pyodide. Tick "also ask my local Gemma" and the page sends the post to Ollama on your machine, talking to 127.0.0.1 directly.

gig-guard demo

Here's a post with none of the usual scam keywords. The rules let it through; Gemma 4 E4B, running locally on a CPU, catches it and quotes the evidence:

๐Ÿ”ด HIGH  Don't apply or reply

โ–  Gemma (gemma4:e4b, 35.8s, on this machine)
  The client is asking you to receive packages at your home and forward
  them to a specified address for 3,000 yen per item.
  โœ“ [high] illegal / mule ใ€Œ่ท็‰ฉใฎๅ—ใ‘ๅ–ใ‚Šใจ่ปข้€ใ‚’ใŠ้ก˜ใ„ใ—ใพใ™ใ€  โ˜… Gemma only
      Receiving and forwarding packages is a typical tactic for money
      laundering or scams.
Enter fullscreen mode Exit fullscreen mode

Code

gig-guard

A scam & fit check for freelance gigs, on any platform, in English, ๆ—ฅๆœฌ่ชž or ไธญๆ–‡. Paste a job post or a client message from Upwork, Fiverr, Freelancer.com, CrowdWorks, Lancers, Coconala or a cold DM. Gemma tells you whether it's a scam, quotes the exact line that gives it away, works out what the job really pays per hour after the platform's cut, and drafts a reply that keeps you on the platform Everything runs on your own machine.

โ–ถ Try it in your browser: https://honru09.github.io/gig-guard/ (rules + pay math run in the tab; connect your local Gemma for the full check)

demo

One command

ollama pull gemma4:e4b
git clone https://github.com/honru09/gig-guard && cd gig-guard
python -m gigguard serve                          # http://127.0.0.1:8765
python -m gigguard --platform upwork check post.txt
Enter fullscreen mode Exit fullscreen mode

Python 3.10+, no dependencies. --no-llm gives the rule and pay checks instantly without a model.

Flag
--platform fee preset: direct 0%, upwork
โ€ฆ

How I Built It

Stack: Python standard library only (zero dependencies), Gemma 4 E4B through Ollama in JSON mode, a tiny local web UI, and a GitHub Pages build that runs the same code in the browser with Pyodide.

The interesting part was deciding how much to trust the model:

 post โ”€โ”€โ–บ rules (regex, EN/JA/ZH) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
   โ”œโ”€โ”€โ”€โ”€โ–บ pay math ($ / ๅ†† / ๅ…ƒ, fees) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”คโ”€โ”€โ–บ verdict (can only go up)
   โ””โ”€โ”€โ”€โ”€โ–บ Gemma โ”€โ”€โ–บ findings โ”€โ”€โ–บ quote check โ”€โ”€โ”˜
Enter fullscreen mode Exit fullscreen mode

1. Gemma reads the post cold and must quote its evidence

Gemma gets only the post, not the rule results, so its read is an independent second opinion. It answers in a fixed JSON shape, and every finding carries an exact quote from the post, with reasons in English, Japanese and Chinese:

{"category": "off_platform_contact", "severity": "high",
 "quote": "contact me on WhatsApp",
 "why_en": "Moving off Upwork removes payment protection.",
 "why_ja": "โ€ฆ", "why_zh": "โ€ฆ"}
Enter fullscreen mode Exit fullscreen mode

Then every quote is checked against the post, with NFKC normalization (so full-width ๏ผฌ๏ผฉ๏ผฎ๏ผฅ matches LINE) and ignoring whitespace and punctuation. If the quote isn't there, the finding is dropped and shown as rejected. Only grounded findings can raise the risk. The model's own top-level "risk" field changes nothing.

This isn't theoretical. Gemma 3 1B "quoted" ใ€ŒๆŠ•่ณ‡ใฏใ—ใพใ›ใ‚“ใ€‚ใ€ ("I won't do investing") as a red flag in a transcription job. Nobody wrote that sentence, and the quote check caught it.

2. Rules are facts, and the model can only escalate

contact me on WhatsApp, lin.ee/โ€ฆ and ๅŠ ๆˆ‘ๅพฎไฟก are facts in the text. Regex rules in three languages catch the textbook flags in about a millisecond: off-platform contact, deposits or starter kits (ๆŠผ้‡‘, ๅŸน่ฎญ่ดน, ๆ•™ๆ่ฒป), unpaid tests and free samples, early ID or SSN requests, crypto, "earn $500 a day" bait, and "no AI" clauses. Gemma can never lower a rule hit. When it disagrees, the report says so.

I learned this the hard way. In my first prototype, Gemma 3 4B rated the LINE-survey message low risk.

3. Gemma catches the scams that dodge keywords

"Receive and reship parcels from home", "we buy your unused bank account", "buy this, leave five stars, we refund you", "let's deal directly so we both skip the fee". None of these trip a keyword list, but Gemma reads them the way a person would.

4. Replies: fixed templates once anything fires

Small models are polite to a fault. One draft cheerfully agreed to the unpaid trial; another said "I'll add you on LINE, but let's keep it on-platform" in one breath. So once anything at medium or above fires, the reply is a fixed, reviewed template in the post's language (EN, JA or ZH): keep it in the platform's messages, ask for a paid test, or politely decline. Gemma's own draft is only used for clean posts, and only if it names no outside tool, URL or contact detail.

Which Gemma? I measured

I wrote 20 gig posts: 14 risky and 6 normal. This first set is Japanese, where the project started. Six of the risky posts deliberately avoid the words my rules look for. The rules were written before the set and weren't tuned on it. Everything ran on an 8-core CPU with no GPU, using Q4_K_M quantization.

Config Risky posts flagged Safe posts false-flagged Expected categories found Caught only thanks to Gemma Valid JSON Findings with quote verified / rejected Median s/post (CPU)
rules only 8/14 1/6 7/15 0 โ€“ โ€“ โ€“
rules + gemma3:1b 8/14 2/6 11/15 0 19/20 56 / 5 10.1
rules + gemma3:4b 14/14 1/6 11/15 6 20/20 13 / 0 32.2
rules + gemma4:e2b 12/14 1/6 12/15 4 20/20 23 / 0 27.9
rules + gemma4:e4b 14/14 1/6 12/15 6 20/20 19 / 0 29.9

What I take from it:

  • Gemma 4 E4B takes rules-only from 8/14 to 14/14 with no new false alarms, and it never invented a quote. The single false flag in every row is my rules' fault: a normal job running a restaurant's "LINE official account" trips the LINE rule, and by design the model can't overrule a rule.
  • 1B is too small for this job. It's fast, but it missed every keyword-dodging scam and fabricated quotes. The quote check caught all five, which is exactly why it exists.
  • E4B is the default: best coverage, and about 30 seconds per post on a plain CPU. That's fine for "check before I reply". --model gemma4:e2b is the lighter option.

Twenty posts is a sanity check, not a benchmark. Fixtures and per-post outputs are in eval/, so anyone can rerun it. An English eval set is next; for now English and Chinese are covered by unit tests and samples.

Tests: 65 unit tests that need no model or network. They cover rules in three languages (including "online", ใ‚ชใƒณใƒฉใ‚คใƒณ and ใ‚ฌใ‚คใƒ‰ใƒฉใ‚คใƒณ not being read as LINE), pay parsing in $, ๅ†† and ๅ…ƒ, fee presets, messy model JSON, quote grounding, the escalate-only contract, reply safety and language, the CLI, the web UI, and that the browser copy of the code stays in sync. They run in GitHub Actions on Python 3.10, 3.12 and 3.13.

Why Does Open Innovation Matter?

  • Privacy is the product. Client messages contain names, business details, and sometimes things nobody should have sent. With Gemma on your own machine, a scam checker doesn't become one more place your data leaks to. Even the hosted demo only talks to 127.0.0.1.
  • The economics only work with open weights. A freelancer tool can't charge much, and paying per token for every message would eat the subscription. Local Gemma costs nothing per check.
  • I could study the failure modes and design around them. I ran four Gemma sizes side by side, saw exactly where the small ones hallucinate, forced JSON mode, and built the quote check and the escalate-only contract around real behaviour. Then I pinned each failure as a test. A closed API can change under you next week.
  • No lock-in. When a better open model ships, switching is one flag.

Prize Categories

  • Best Use of Gemma: Gemma 4 E4B runs locally through Ollama and returns structured, quote-grounded findings with English, Japanese and Chinese reasons. The repo includes a reproducible comparison of Gemma 3 1B and 4B against Gemma 4 E2B and E4B.

Top comments (0)