Scam or Genuine? An Offline Message Checker for a Friend (Gemma 3)

Shruti Chaudhary — Mon, 05 Oct 2026 03:03:18 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

A friend of mine wanted a way for his father and himself to check whether the messages they get from banks and other senders are scams or genuine. So I built a first version of one.

You paste an SMS or WhatsApp message. The tool answers SCAM, SUSPICIOUS or SAFE, gives a short reason in simple Hinglish, and says what to do next. It runs entirely on my laptop with Google's Gemma 3 through Ollama, so messages that can contain account and transaction details never leave the machine.

It is a second opinion, not a safeguard. In my testing it missed some scams and wrongly flagged some genuine messages, and I show those failures below.

Demo

Code

ShrutiChaudhary08 / Scam-Checker

Scam Checker: works offline, built for a friend

Submission for the Hacktoberfest Weekend Challenge: Build for a Friend (DEV, 2 to 5 October 2026).

A friend wanted a way for his father and himself to check whether the messages they get from banks and other senders are scams or genuine, so I built this for them.

You paste an SMS or WhatsApp message. The checker answers SCAM, SUSPICIOUS or SAFE, gives a short reason in simple Hinglish, and says what to do next. It runs on an ordinary laptop with Gemma 3 through Ollama. No message leaves the machine and it needs no internet after the model is downloaded.

How it works

checker.py sends the message to a local Gemma model through the Ollama Python library, asks for JSON, uses temperature 0, and includes two worked Hinglish examples in the prompt. Replies that are not valid JSON…

View on GitHub

How I Built It

Model: Gemma 3 through Ollama, called from Python with the ollama library. I ran the 4B model (the default) and the 1B model so I could compare them.
Prompt: it asks for a JSON answer at temperature 0 and lists the usual warning signs. At first Gemma answered in English even though I asked for Hinglish. Adding two worked Hinglish examples fixed that.
Fallback: if the model's reply is not valid JSON, the result is "UNKNOWN", which is treated as a warning.
Page: a small Streamlit app. When the verdict is SAFE it adds a reminder to confirm with the bank before clicking a link, sharing an OTP or sending money.
Checks: 14 unit tests and an evaluation script that scores the tool on labelled messages.
AI help: I used an AI assistant (Claude) for planning and code, and ran and tested everything myself on a Windows laptop with 16 GB RAM.

Results, Including the Failures

I tested on three small sets and kept them separate because they come from different sources. The results are indicative, not proof of real-world accuracy.

Set	gemma3 (4B)	gemma3:1b
24 hand-written messages (12 scam, 12 genuine)	23/24 right, 1 false alarm, 0 scams missed	20/24 right, 4 false alarms, 0 scams missed
7 public scam examples	6/7 caught	7/7 caught
3 genuine messages from my own inbox	1/3 right	0/3 right
Average time per message	about 18 to 21 s	about 8 to 9 s

What I learned:

In my test sets the 1B model never missed a scam, but it flagged nearly half of the genuine messages. A tool that keeps warning about real bank alerts will stop being trusted, so I made the 4B model the default.
Both models flagged a genuine one-time-password message and a genuine SBI debit alert as scams.
The 4B model marked a fake "Did you initiate this? YES or NO" bank text as SAFE. It reads almost exactly like real bank fraud alerts, and the tool only sees the text, not who sent it.

How the test messages were made: the 24 were written by me with AI assistance, using made-up numbers and links. The 7 scam examples were transcribed from publicly shown examples (Berkeley Lab IT, Aura, Net Protector Antivirus, BankFive and Doing More Today), with links and numbers replaced. The 3 genuine messages are from my own inbox, anonymised. I did not tune the prompt on any of these sets.

Why Does Open Innovation Matter?

It runs offline. I tested it with Wi-Fi off and it still answered.
Bank messages are sensitive. With a local model, they stay on the laptop instead of going to someone else's server.
I could swap models and measure the difference. The 4B against the 1B showed a real trade-off between speed and false alarms, which I could see and choose between.
I could change its behaviour myself. Two examples in the prompt were enough to change its language.
It costs nothing to run after the model download.

The trade-off: it is slow, about 20 seconds per message on my laptop. I did not compare it with a closed model, so I can't say whether a closed model would be more accurate.

Limitations

My small test sets made the tool look more reliable than it is, and it will miss scams. It cannot verify who sent a message, so for anything involving money, call the bank on the number printed on your card.

What My Friend Said

I handed it to my friend and his father. Here is what they said, shared with their permission:

This will help us in getting rough idea of the messages we receive before we proceed with our next set of actions. Thank You.

Prize Categories

Best Use of Gemma: I ran Gemma 3 locally through Ollama.

DEV Community: Shruti Chaudhary