DEV Community: Muaaz Shaikh

Sair: I built my parents a walk companion that makes them leave the phone at home

Muaaz Shaikh — Sun, 11 Oct 2026 17:04:25 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

My parents walk every evening. Tonight they came back and told me about the sparrows they heard and the mogra flowers they saw. That's pretty much the whole idea behind what I built this week. Give them a small reason to look around, and let the phone stay at home.

Last week I built them something to protect them from their phones (Rakshak). This week I wanted something that gets them off their phones.

What I Built

Sair (सैर, it means "a stroll") is a walk companion that barely has a screen.

At the door, Mom taps "Hear today's walk". Sair says one small thing to notice, slowly, twice. Something like "Notice one sound you like on your walk."
Then the phone stays home. The app actually tells them "Leave me here. I'll be waiting when you're back." There are no maps, no step counts, no streaks and no notifications. It's about 30 seconds of screen before the walk and none during it.
When they're back, Mom holds one button and just talks for up to a minute.
A model running on our MacBook turns that voice note into a short diary entry, and the things they noticed show up as little chips, like "gulmohar · near the corner" or "koel · near the temple".
The next day's mission builds on what they really noticed. For example: "On Tuesday you heard a koel near the temple. Is it there again?"

It's built for two people in their fifties and sixties who are fine with WhatsApp and UPI and who switch between English, Hindi and Hinglish in the same sentence.

The hard part wasn't the app. It was this: hallucinations pile up. If an AI journal makes up a bird, that's a bit annoying. If a walk companion with memory makes up a bird, it sends two people out the next evening to look for a bird that was never there. So Sair has one rule I didn't want to bend: nothing goes into the diary, or into memory, that nobody actually said.

Demo

I recorded this in the Android Emulator, talking to the real hub on my MacBook. Same app, same models, same pipeline. The voice note in the video is a text-to-speech stand-in, not my mom.

In order, you'll see Sair speaking the mission (that's Kokoro, running on the laptop), the "leave me here" screen, hold to talk, "Writing it down…", and then the diary entry with three checked facts and "Mission done".

One honest note about that last screen. "Mom and Dad went for a walk and did today's mission. They noticed gulmohar, koil and puddles." reads exactly like Sair's plain template, and it isn't a fallback: Sair-Writer wrote that sentence itself, on its first try. It picked up the template's style from its training data (more on that below). "Koil" is how the transcription heard "koel", and it's kept as heard because every fact has to quote the transcript. When it does write properly, it looks like this entry from another test note: "Mom and Dad enjoyed a peaceful walk where they saw kids playing cricket near the gate and a dog under the bench. They think maybe a crow was calling from the tree."

At the door	The mission, spoken twice	Back home: hold to talk	The diary entry

The first real walk

Tonight was my parents' first walk with Sair. I set up both phones, and in the evening they went out.

When they came back, they told me about the sparrows they heard and the mogra flowers they saw. That's exactly the kind of small thing I hoped Sair would make room for. Not steps, not a route, just what they noticed.

It's one walk, so I'm not drawing conclusions from it. What I'm watching over the next few evenings is simple: do they actually leave the phone at home, does the voice note feel natural to them, and does the diary entry sound like what they said. The Hinglish problem below is the part I'm most nervous about, because that's how they talk. I'll add real numbers once there are enough real walks to count.

Code

MuaazSM / sair

Sair (सैर)

A screenless evening-walk companion for two parents. Before the walk, Sair speaks one small noticing mission at the door, twice. The phone stays home. Afterwards, a voice note of up to a minute becomes a short family diary entry, and the next mission builds only on what was actually said.

"On Tuesday you heard a koel near the temple. Is it there again?"

Everything runs on a MacBook at home. No inference, and no content or timing of any real walk, leaves the house.

Why it's built this way

Hallucinations compound. If a diary invents a bird, tomorrow's mission sends two people looking for a bird that was never there. Sair keeps the diary faithful with three pieces:

a small open model fine-tuned to write grounded entries (Qwen3.5-4B + LoRA, served by llama-server with a JSON grammar);
a grounding check in code: every fact must quote the transcript…

View on GitHub

A quick map of the repo:

hub/ is the FastAPI and LangGraph hub (a journal graph and a mission graph) with SQLite. It only listens on loopback.
hub/grounding.py has the seven grounding checks, written as plain code.
hub/safety.py keeps missions safe with a verb allowlist, a blocklist and a check on where every noun came from.
training/ covers data generation, teacher labelling, LoRA training on Tinker and the export to GGUF.
eval/ is the evaluation harness, with bootstrap confidence intervals and the memory contamination chains.
web/ is the React PWA, matched pixel by pixel against the design.

There are 1,136 tests (unit, integration and privacy) and they all pass in CI (latest run).

How I Built It

Everything runs on one MacBook

It's an M3 Pro with 18 GB that sits on a shelf at home. The hub only binds to 127.0.0.1, and the phones reach it over a private Tailscale network using tailscale serve, with Funnel turned off.

Job	Open model or tool	Runs on
Transcribe the voice note	Gemma 4 E2B through Ollama, straight from audio, temperature 0	MacBook
Plan tomorrow's mission	Gemma 4 E2B	MacBook
Write the diary entry	Sair-Writer: Qwen3.5-4B with a LoRA I trained on Tinker, as a Q8_0 GGUF on llama.cpp with a GBNF grammar	MacBook
Speak the mission	Kokoro-82M, in its own process	MacBook
Glue	LangGraph (no persistent checkpointer), FastAPI and SQLite	MacBook
Tell me a walk happened	self-hosted ntfy, counts only, never any text	MacBook

The fact checking is code, not a prompt

Politely asking a model not to make things up doesn't work. So every entry goes through seven checks written in ordinary Python, in hub/grounding.py:

Every fact has to quote the transcript exactly.
The thing they saw, and where, has to be inside that quote.
Hedges survive. "I think it was a koel" stays a maybe.
Every concrete detail in the entry has to be backed by one of those facts.
No new numbers, times or names.
The entry has to look like an entry: 5 to 70 words, says who walked, no questions, no links.
If it says the mission was done, that needs a quote too.

If an entry fails, the writer gets one more try. If it fails again, Sair writes a plain entry from a template that code builds. I'd rather have a boring true sentence than a lovely false one.

Memory only holds facts that passed the first three checks. The mission planner never sees the diary text, only those checked facts and the last five missions. Missions get checked too. Each one has to be a single task, at most 30 words, use an allowed verb, avoid anything risky, speak to them as "you", never mention a time of day, and only use nouns that came from a checked fact or a short list of safe starters.

Fine-tuning a small writer on Tinker

The stock Qwen3.5-4B, even with good examples in the prompt, put at least one made-up detail into 42% of its entries. I wanted something small enough to run next to Gemma on a laptop, and a lot more honest.

Data: synthetic voice-note transcripts made from generic seeds like parks, lanes, temples and markets. Nothing from my family went in. I split by seed, so the test seeds were never seen during training.
Teacher: Qwen3.5-397B on Tinker wrote the gold entries, and each one had to pass the same seven checks or it got replaced by the code template. That gave me 1,114 training rows.
Training: LoRA rank 32 on Qwen3.5-4B. I ran a small learning-rate sweep and then the full run, and picked epoch 2 because it had the lowest made-up-detail rate on the dev set.
Export: I merged the LoRA into the base weights in f32 and converted to GGUF. Q4_K_M failed my export check (its template fallback rate moved by 2.33 points and my limit was 2), so Sair ships Q8_0, which matched the Tinker checkpoint on every metric I gate on.
Cost: about $6.65 on the Tinker usage page. That's $4.95 for the 397B teacher and roughly $1.70 for the 4B sweeps, training and dev sampling.

Results

These numbers come from a held-out synthetic test set: 130 voice-note transcripts from 8 seeds the model never saw in training, scored by the same code checker. The confidence intervals are 95%, from a bootstrap that resamples whole seeds (10,000 times).

Writer	Entries with a made-up detail	Facts kept (recall)	Fell back to the template
Gemma 4 E2B, zero-shot	20.0%	0.864	no safety net
Qwen3.5-4B base, few-shot	42.3%	0.743	no safety net
Sair-Writer (fine-tuned)	6.2%	0.939	no safety net
Gemma with the grounding check	0%	0.862	55.4%
Qwen base with the grounding check	0%	0.732	24.6%
Sair-Writer with the grounding check	0%	0.937	6.2%

Fine-tuning took made-up details from 42.3% down to 6.2% compared with its own base. That's 36.2 points lower, with a 95% interval of 22.9 to 49.3 points.
It didn't get there by saying less. Recall went up from 0.743 to 0.939 (19.7 points, interval 13.4 to 27.7), and hedges survived more often too, from 0.849 to 0.949.
With the grounding check switched on, every writer gets to 0% made-up details, because that's what the check enforces. The real difference is how often the safety net has to catch them. Sair-Writer falls back to the plain template on 6.2% of notes. Gemma falls back on 55.4%.
There's a catch I only found while making the demo. Sair-Writer learned the template's style from its training data, so sometimes it writes the template sentence itself. Counting those, 16.2% of its final entries read word for word like the template (8 real fallbacks plus 13 it wrote itself), against 55.4% for Gemma and 24.6% for the base model. Still far fewer, but not the 6.2% the fallback number alone suggests.

What didn't work

I'd rather tell you than have you find it.

My contamination experiment came back null. I ran 7-day chains comparing memory built from diary text against memory built from checked facts, expecting the diary-text memory to leak made-up details into later missions. It didn't, for any writer. Diary-text memory gave 0 contaminated chains out of 20, and fact memory gave at most 1 out of 20, which was a number word like "one". It turns out Gemma's missions are generic enough that they rarely reuse details at all. I still think the rule is right, but this run doesn't prove it.
Hinglish voice notes currently come back from Gemma in Devanagari script, and then the writer falls back to the template. English works well. This is the first thing I'm fixing.
Live audio is harder than clean transcripts. All the numbers above are on text. Tonight I ran 12 synthetic voice notes end to end through the real app and hub, and only 3 came out as a properly written entry. 5 fell back to the template and 4 were Sair-Writer copying the template's wording. Small transcription slips, like "koel" heard as "koil", make the checker reject facts, and the same file doesn't always come out the same way. This is the thing I most want to fix next, together with the training data that taught it the template's style.
Speed: in my demo runs, an 18 to 20 second note was ready in about 20 to 25 seconds (I was checking every 5 seconds). I haven't measured it properly on real notes yet.
Every number above is synthetic. Real-walk numbers will come after real walks, and I'm not going to publish any before they exist.

Privacy: nothing about a walk leaves the house

A record of when two older people leave home and how long they're out is sensitive. So it isn't only what they say that stays at home. The timing stays home too.

The hub refuses to start on anything other than 127.0.0.1. Phones reach it through tailscale serve, and Funnel is off.
The only outbound connections are to the model servers on the same machine. Models are downloaded ahead of time and Hugging Face runs in offline mode.
Raw audio is deleted right after transcription. No audio or text ever goes into a log, a trace or a notification. When something fails, the reason is a short code.
A canary test pushes a marker phrase through the whole pipeline, and CI fails if that phrase shows up in any log, trace, notification or model-server log.
There's no telemetry in production. Sentry only runs in dev and eval on synthetic data, and the settings won't turn it on if any data path points at the real data folder.
No location, no streaks, no gamification. The app never asks for location, and CI scans the built app for geolocation, beacons and analytics.

Training wasn't fully local: synthetic transcripts went to Tinker for teacher labelling and LoRA training. No family voice note, transcript or entry was ever part of it. At runtime, nothing about the walks leaves the house.

Why Does Open Innovation Matter?

It can live in our house. With a closed API, every voice note, plus a timestamped record of when my parents go out, would sit on someone else's server. With open weights, inference never leaves the MacBook.
I could make it honest. I fine-tuned a 4B model for this one job and cut made-up details from 42% to 6%. I couldn't do that with a closed model this cheaply, and I definitely couldn't run the result offline.
I can check it. Open weights and a GBNF grammar let me force valid JSON and compare the export byte for byte. 105 of the 129 dev outputs were identical between the Tinker checkpoint and the Q8_0 GGUF.
It's free to run. About $6.65 to train, and then nothing per walk, on a laptop we already own.
It keeps working when the internet doesn't. I unplugged the router's internet connection and Sair kept working.

Prize Categories

Best Use of Tinker (Thinking Machines): a LoRA fine-tune of Qwen3.5-4B on Tinker with a measured gain over its own base. Made-up details went from 42.3% to 6.2%, recall from 0.743 to 0.939, and template fallback with the safety net from 24.6% to 6.2%.
Best Use of Gemma: Gemma 4 E2B runs locally through Ollama and does two jobs on every walk. It transcribes the voice note straight from the audio, with no separate speech model, and it plans the next mission from checked facts. It's also the zero-shot baseline I measured the fine-tune against.
Best Use of GitHub Copilot (GitHub Actions): every push runs the 1,136 unit, integration and privacy tests, scans the built app for outbound URLs, geolocation and analytics, and replays the cached evaluation against a committed baseline. If a change makes the diary less faithful, the build fails.

Built solo by @muaazsm. My parents get the credit for the walking.

Rakshak: I built my parents a scam-checker that never leaves the house

Muaaz Shaikh — Sun, 04 Oct 2026 21:09:36 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

My parents have never lost money to a scam. I'd like to keep it that way.

My mom lives on WhatsApp. My dad pays for everything with UPI. Their phones get a steady drip of messages written to make someone panic: a KYC deadline "today", a parcel "held at customs", the electricity "disconnected tonight", a "police officer" on a video call. They've been careful so far. But these messages are designed for the one moment you're tired, alone or scared. And they only need to work once.

So I built them Rakshak ("protector"): a second opinion that lives on a laptop in our house. You hand it a suspicious message, a screenshot, or a description of a phone call, and within seconds a calm voice tells you in plain English whether it's real and what to do.

Mom (Android) shares a message to it straight from her share sheet. A big card comes up: red with a raised hand for This is a scam, amber for Be careful, green for This looks like a normal message, grey for Couldn't check right now. The explanation is read aloud. For a suspicious call, she holds a button and describes it.
Dad (iPhone) uses two Shortcuts: "Rakshak" in the share sheet, and "Rakshak Bolo" (bolo means "speak") to describe a call out loud. Same card, same voice.
Me: on a scam verdict my phone gets a push with which parent, what kind of scam, and what time. Never the message.

Every explanation is three short sentences at most: the verdict, the one or two strongest warning signs quoted exactly from the message, and one thing to do. Don't tap. Don't pay. Never share the OTP. Call me.

Nothing leaves the house. That's not a slogan, it's the architecture.

Demo

There's no public link, and that's on purpose. Rakshak runs on a MacBook at home and is reachable only over our private Tailscale network, because the whole point is that my parents' messages never touch someone else's server. Here it is running for real:

Left to right: the home screen; the check in progress; an app-install scam caught with the dangerous link quoted, plus Listen again for the spoken explanation; a real bank OTP correctly left alone.

The hub's status page for me: three local models, latency against targets, today's verdicts per parent, and a "was this right?" queue. Metadata only. It never shows a message.

Code

MuaazSM / rakshak

Rakshak

A scam checker for my parents that runs on a MacBook at home.

My mom uses an Android phone and my dad uses an iPhone. Both get a steady stream of messages written to make a parent panic: fake KYC deadlines, parcels "held at customs", electricity cut off "tonight", fake police calls. Rakshak gives them a second opinion. Mom shares a message or screenshot to the Rakshak web app from her share sheet; Dad shares it to an iOS Shortcut. A few seconds later they see a large verdict card (scam, be careful, looks normal, or couldn't check) and hear a short spoken explanation in plain English. If it's a scam, my phone gets a notification that says which parent and what kind of scam, never the message itself. Every model runs on the Mac. The messages never leave the house.

The verdict comes from a small model fine-tuned…

View on GitHub

Everything is in the repo: the rules engine, the LangGraph pipeline, training, export and evaluation scripts, the web app, and the privacy tests. The dataset is regenerated locally and the model weights are built by the export script rather than checked in, but nothing about them is secret. They're trained only on synthetic and public data.

How I Built It

A MacBook Pro (M3 Pro, 18 GB) is the hub. It binds to localhost only; phones reach it over a private tailnet. Each check runs through a small LangGraph graph: normalize, perceive the image or audio, rules, detect, fuse, explain, then notify and store. SQLite keeps the records. Every model runs locally through llama.cpp and Ollama.

Two open models, three jobs

I wanted the verdict to come from a model I'd fine-tuned and measured, not a prompt. Tinker doesn't offer Gemma, which made the split natural. It also gave Gemma a second, less obvious job.

Gemma 4 E2B wrote the training set. I had no scam dataset, and I refused to build one from my parents' private messages. So Gemma generated it: from about 60 short seed patterns across every scam and genuine category, it produced around 1,300 realistic messages in English, Hinglish and Devanagari, varied across channel and the tricks scammers use: misspellings, odd spacing, look-alike characters, shortened links. Every quoted red flag had to appear exactly in the generated text or the item was dropped.

Tinker trained the judge. Rakshak-Detector is Qwen3.5-4B with a LoRA adapter trained on Tinker on that data, then merged, converted to GGUF and quantized to Q4_K_M (2.9 GB) for llama-server. It returns only JSON: verdict, category, red-flag quotes. I read the scam probability from the log-probabilities of the verdict token, not from the text, so the fusion step has a real number to work with.

Gemma explains the verdict. It transcribes screenshots and voice notes on-device, and turns the verdict into a short explanation in plain words. It never decides. It receives the verdict as input, and a post-check rejects any explanation that drops the verdict word, negates it, adds a link or number that wasn't in the message, or runs long. On failure a fixed template takes over.

Under all of it sits a rules engine of plain, unit-tested functions. An .apk link, a look-alike of a bank's domain, or "enter your UPI PIN to receive money" is a scam whatever the model says. Softer signals push toward Be careful: a bank claiming to text from a plain mobile number, a link shortener, arrest threats, urgency plus a payment ask. The word lists cover English, Hinglish and Devanagari, because that's how scams actually arrive.

Two safety rules live in code, not prompts: red flags shown to my parents must be exact substrings of their message, or they're dropped; and if the detector is down the answer is Couldn't check, never looks normal.

Training

LoRA rank 32, loss on the assistant's JSON only, batch 32, 3 epochs, thinking mode off, learning rate 4.91e-4 from tinker_cookbook's get_lr. In 20% of training examples I blanked the rule signals so the model can't just copy the rules. I ran a smoke run, a learning-rate sweep, and then a retrain, because two independent review passes caught the quiet failure of synthetic data: the first model had learned shortcuts like a link means scam and an unknown sender means scam, which would have meant false alarms on real bank SMS. I fixed the generator to balance those cues and trained again. Total: about $3.80 of the $10 Tinker credit.

Results, on data it never saw

I did not dataset my parents' messages, so there is no family test set. Instead, the evaluation is 100 public fraud reports: scams and genuine messages documented by RBI, PIB, India's cybercrime advisories, news outlets, fact-checkers and court records, each with a source URL and verified verbatim on its page (65 scam, 35 genuine). The model never trains on them. 95% bootstrap CIs; scam recall counts any warning on a real scam.

System	Scam recall	False alarms on genuine	On alarming-but-genuine	Macro-F1	Valid JSON
Rules only	0.43	9%	11%	0.05	n/a
Gemma 4 E2B zero-shot	0.86	14%	22%	0.38	0%
Qwen3.5-4B base, few-shot	1.00	43%	89%	0.54	33%
Qwen3.5-4B + Tinker LoRA	0.92 [0.86–0.98]	9%	22%	0.59 [0.55–0.63]	100%
Full Rakshak (rules + fusion)	0.94	14%	22%	0.58	n/a

The interesting number isn't the F1. It's the false alarms. The base Qwen model with few-shot prompting catches every scam, but only by crying wolf. It flags 43% of genuine messages and 89% of genuine-but-alarming ones (real OTPs, real debit alerts), and only a third of its replies are even valid JSON. Fine-tuning the same model on this one narrow task cuts false alarms on genuine messages to 9% and makes the output 100% machine-usable, while keeping recall at 0.92, on data it never trained on.

That's the whole project in one row. Not missing scams is easy. Not terrifying your mom about a real OTP is the hard part, and that's exactly what a small, fine-tuned open model fixed.

(Adding the rules on top nudges recall to 0.94 but raises genuine false alarms, because a few real government advisories trip the threat lexicon. I can see that trade-off in the numbers, which is the point of measuring.)

These are small-sample numbers on public data, not a benchmark claim. The synthetic dev set is near-saturated and I don't report it.

Speed on a laptop

All three model servers run together on 18 GB. End to end:

Genuine (detector only): p50 0.8 s, p95 0.9 s
Scam (detector + spoken explanation): p50 4.1 s, p95 11.1 s

The detector dominates the scam path; the explainer adds about 1.6 s. The scam path is over my 6-second goal. Shorter red-flag output, constrained JSON on the first attempt, and streaming the verdict before the explanation are next.

Tracing without reading anyone's messages

Sentry gets one transaction per check and one span per graph node, so I can see where the seconds go and which model answered, but never the content. Span attributes come from an allowlist (node, model, token counts, latency, verdict, category, p_scam, rule-signal names, retries, error type), and a scrubbing hook deletes any text, quote, explanation, image or audio field at any depth, including inside exception messages. Request bodies and local variables are never captured.

I don't just trust that. A canary test pushes a fake message with a unique marker through every input path of the real app and fails if the marker appears in any Sentry payload, ntfy request, log line or console output, or if the hub talks to anything but the local model servers and ntfy. I broke the scrubber on purpose three ways; the test caught all three.

Why Does Open Innovation Matter?

Privacy that's literally true. The messages my parents get are about their bank, their pension, their family. Asking a hosted API "is this a scam?" means one more stranger reads them. With open weights, the model comes to the data, and checks keep working with the Mac's internet off.

A small specialist beats a big generalist here. "Is this a scam, and which words prove it?" is narrow and structured. Open weights let me fine-tune a 4B model on exactly that and measure it against its own base: false alarms went from 43% to 9%, valid output from 33% to 100%. You can't do that with a model you can't fine-tune or run.

Open models can make the data, too. Gemma generated the whole training set locally. No scraping, no labeling service, no one's private messages.

Zero marginal cost. Each check costs electricity, so there's no reason to ration them. My parents can share every message that feels off.

The honest caveat. Training wasn't fully local: synthetic and redacted public examples went to Tinker's servers to train the adapter. No family message was ever part of it. At runtime, nothing leaves the house.

My Agent Session

I built Rakshak with Claude Code, working from a written requirements document that fixed the schemas, prompts, thresholds and privacy rules before any code existed. The agent did the heavy lifting in parallel (rules engine, hub pipeline, training and export, web app), and the most valuable thing it did was review its own work. Two independent review passes caught the detector throwing away suspicious probability mass, and the synthetic-data shortcuts that would have caused false alarms on real messages. The judgment calls I kept for myself were the ones about honesty: never letting public reports become a "test set", and insisting the evaluation say plainly what data it actually ran on.

Prize Categories

Best Use of Gemma. Gemma 4 E2B generated the training set, reads screenshots and voice notes on-device, and writes every warning my parents hear, with a post-check that keeps it from ever changing the verdict.
Best Use of Tinker. The verdict comes from Qwen3.5-4B fine-tuned with LoRA on Tinker, exported to a quantized GGUF, and measured against its own base model and Gemma on public data it never trained on, where fine-tuning cut genuine false alarms from 43% to 9% and raised valid structured output from 33% to 100%.
Best Use of Sentry Agent Tracing. One trace per check and one span per LangGraph node, with a canary test proving no message text ever reaches Sentry.