BeAwake: a local Gemma that checks scam messages for my Turkish friend and my family

Khaled Saber — Mon, 05 Oct 2026 17:33:45 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

What I Built

BeAwake is a small tool for one specific problem. My Turkish friend and my family get messages that pretend to be a bank, a courier, or a relative, and it is not always easy to tell which ones are real.

You paste a message. BeAwake tells you if it looks safe, suspicious, or like a scam, gives one sentence of evidence, and one thing to do next. It answers in the language of the message. My friend's messages come in Turkish, my family's in Arabic, and some of my friends write in English, so it had to work in all three.

The most important part to me: it runs on my laptop. These are exactly the messages that contain names, phone numbers, and sometimes card details. I didn't want anyone to paste them into a website just to ask "is this a scam?"

Demo
There are many e

xample like : SCREENSHOT blow down the articel

The interface is a single Gradio page running at 127.0.0.1. There is nothing to sign up for, and nothing is sent online.

Code
https://github.com/Skemo754/BeAwake
How I Built It

The model. Gemma 4 12B, quantized to Q3_K_M (about 5.7 GB), served by llama.cpp's llama-server on a laptop with an RTX 5060 and 8 GB of VRAM. A verdict takes a second or two.

The pipeline. Three small pieces:

Gemma reads the message and writes three lines: VERDICT, REASON, ACTION.
Plain Python looks for signals: shortened links, lookalike domains, someone asking for a code, a "new number + send money" pattern. These are shown in the UI.
A safety floor makes sure the final verdict is never lower than the rules' own verdict. Calling a scam "safe" is the worst mistake, so the floor only raises a verdict, never lowers it.

That sounds tidy, but it is not what I started with. Most of what I learned came from things that broke.

Thinking mode ate my answers. My first request came back with an empty answer. Gemma was "thinking" first, and the 200 tokens I allowed were all spent on reasoning, so it never reached a verdict. It also took 17 seconds. Turning thinking off (enable_thinking: false) gave a clean answer in under 3 seconds.

Forcing JSON made Turkish worse. I first forced the output into a JSON schema so it could never be malformed. In Turkish, the explanation fell apart into a loop (...inininin...) until it hit the token limit. Dropping the schema and asking for three plain lines fixed the worst of it, and I parse those lines in code. If the verdict line is unreadable, the code falls back to suspicious. I later found that temperature 0 also caused repetition loops, so I use 0.2 with a repeat penalty.

My "helpful" hints made it worse. I tried telling Gemma what the code had found ("signals: link, urgency") before it made its call. I measured it: on my first test set the false alarms went from 3 to 5 out of 18 legitimate messages, and it caught no extra scams. So I removed the hints. The signals now only run after the model, never before it.

English messages got answers in Spanish. I saw one early, and when I built a second test set, 7 of 12 English replies were flagged as being in the wrong language. The fix was boring: a few lines of code decide the reply language and name it explicitly in the prompt.

Does it actually work?

I wrote two sets of 36 messages (6 scams and 6 legitimate ones in each of Turkish, Arabic and English). The second set, the holdout, was written after the first round, with different wording. I compared the model against a simple rules-only baseline, before and after my improvements (v1 and v2).

Holdout, 36 messages rules only (v1) rules only (v2) BeAwake v1 BeAwake v2
Scams missed (of 18) 12 5 0 0
Legitimate flagged (of 18) 2 0 6 1
Accuracy 61% 86% 83% 97% (35/36)
English replies in the wrong language - - 7 of 12 0 of 12

What I take from it:

The model catches what rules can't. The five scams the improved rules still miss have no link and no keyword: a relative asking for money, a job offer, a phone-call threat, an investment group. Gemma caught all five.
Most of the improvement came from the prompt and the language fix, not from the rules. The safety floor never changed a single verdict in any of my runs, so I can't claim it adds value. It stays as a safety net.
I want to be careful with these numbers. I wrote both test sets myself, so they look like the scams I already know. I rewrote the prompt after seeing mistakes on the first set. And 0 missed scams out of 18 doesn't mean the true miss rate is zero: with only 18 scams it could still be around 19%. Treat this as a sign that it works, not as proof.
Turkish is the weakest language. With a heavily quantized model, the Turkish explanations sometimes have small errors or odd words (it wrote tıklemeyin instead of tıklamayın). The verdicts were right, but the wording wasn't perfect.
What my friend and family said

I haven't been able to put BeAwake in their hands yet, so I have no feedback from them to share. I'd rather say that than make something up.

The thing I'm least sure about is the Turkish. My friend is the person who can tell me whether those small language mistakes matter in real use, and that is the first thing I want to find out next. The second is how it handles the messages that actually reach them, which are probably messier than the ones I wrote.

A note on earlier work

I had already built a personal local assistant called NEXUS, which taught me how to run Gemma with llama.cpp. BeAwake is a new repository, created on October 2 and written during the challenge. I didn't copy code from NEXUS; what carried over was the experience, not the files.

I also worked with an AI assistant while building this and writing this post.

Why Does Open Innovation Matter?

Privacy is the whole point. The messages people want checked are the ones with private details in them. With a local open model, they never leave the machine. A hosted API might have been simpler, but it would have created exactly the problem I was trying to avoid.

It works without internet and costs nothing per message. No account, no API key, nothing to renew.

I could look inside it and change it. When the answers broke, I could read the raw output, switch thinking off, change the temperature, add a repeat penalty, and rewrite the prompt. That is how I found every problem above.

I should be honest about one thing: I didn't compare against a closed API, so I can't say an open model is more accurate. My argument isn't accuracy. It's where the data goes.

One model handled three languages (Turkish, Arabic and English) with the same prompt. Most free scam filters I know of are built for English first.

Prize Categories

Best Use of Gemma. Gemma does the judging: it reads the message in context and explains it in the message's own language, running locally on a laptop. The plain-code rules can't do that part. Without the model, they missed 5 of the 18 holdout scams.

DEV Community: Khaled Saber

BeAwake: a local Gemma that checks scam messages for my Turkish friend and my family