This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend
What I Built
Last month Ma forwarded me a WhatsApp mes...
For further actions, you may consider blocking this person and/or reporting abuse
building software for your own family is the purest form of product-market fit. the fact that your mom said it's one of the best things you've built is the only metric that matters.
the "rule layer can only raise risk, never lower it" pattern is a masterclass in defense-in-depth. it's the exact same philosophy as constitutional ai guardrails: the llm provides the contextual understanding, but deterministic rules hold the hard, non-negotiable boundaries.
reading the thread about the vlm transcription gap is fascinating. you're absolutely right that relying solely on the vlm's output for the rule layer creates a single point of failure. if the model drops a blurry "otp" line, the safety net fails.
running a lightweight, deterministic ocr (like tesseract) in parallel just to scan for high-risk keywords (otp, pan, specific romanized scam phrases like "block hoye jabe") ensures that the two layers fail independently. the vlm handles the nuanced reasoning, while the dumb ocr acts as a strict, fail-safe tripwire.
massive respect for the privacy-first, local-execution approach. protecting a parent's bank details by ensuring the photo never leaves the house is a feature no closed api can ever replicate. 🐯
Yes, that's the thing. When I first identified the problem, it really stuck with me and was finding the best solution, the best user experience so that it's simple enough for them to use. It just works out of the box without any subscription or anything. I guess this is kind of the best interface and the best user experience that I wanted to build.
absolutely. building for real people with real constraints is the only way to make software that actually matters and respects the user. keep pushing the boundaries, rohit! 🐯
Thanks a lot,appreciate it
The focus on Bengali is important because scam language often depends on local phrasing, not just translation quality. A useful next test would be measuring false positives on ordinary family messages separately from high-risk requests, with the model showing which phrases drove the warning. Are you planning a small real-world evaluation across message types?
iin1005h0328
Yes absolutely. Implementing a good classifier is my next target so currently training one thinking of implementing typesafe's jev into this, which I believe will be quiet efficient
Local-first is the right call for this exact job — scam texts name banks and OTP flows, and pasting those into a hosted API is its own privacy leak. How does the Gemma classifier hold up on code-mixed messages? The ones my family forwards are half Bengali half English with the URL romanized, and that code-mixing is where small models tend to wobble most. If it catches those, this generalizes well beyond your mom.
Great question, and you called it exactly. I just tested it with four new messages written the way families actually forward them:
Romanized Bengali (Banglish): "Dada apnar SBI account aaj raat 9 tar modhye block hoye jabe… ei link e click korun… OTP ta amader executive ke bolun"
Bengali script with English words dropped in, plus a bit.ly link
Hinglish: "aapka bijli connection aaj raat 9:30 baje kaat diya jayega… turant officer se baat karein"
A real Banglish message about a bill, as a control
Gemma 4 26B (the hosted demo): 4/4. All three scams flagged, the real message marked safe. It caught the Hinglish electricity scam with no help from my keyword rules, from meaning alone.
Gemma 3 4B (the family laptop): 2/3 scams. It caught both Bengali ones, but rated the Hinglish electricity-cut scam only "caution", not "scam". That's the wobble you predicted.
It also exposed a gap in my backup layer. The keyword rules only know English and native-script words like "blocked" or "ব্লক", so romanized phrases like "block hoye jabe" and "kaat diya jayega" slipped past. The next fix is romanized patterns in the rules, so the 4B model isn't the only thing catching them.
The scam verdict was right and the advice pointed at the scammer's helpline, which a label-only test can't show. Your rules read original_text, Gemma's own transcription, so a misread OTP line won't reach them, and I'd feed them plain OCR. How often does original_text drop the OTP or KYC line on blurry photos?
Good catch, and you're right. The rules only see Gemma's own transcription, so if it misreads the OTP line, the rules never get a chance.
I haven't measured how often that happens. On the blurry photos I tried, Gemma usually gave up entirely ("too blurry, take it again closer") instead of returning a partial transcription, so it failed loudly rather than quietly dropping a line. That's a few tests, not a number, though.
Running the rules on plain OCR (Tesseract, say) alongside Gemma's text is the right fix. Then the two layers fail independently. Adding it to the list.
adding voice is a really cool idea.
Thanks a lot. Yes when i gave the app without the voice, my mom suggested that having a voice that will directly say it would be very helpful