Gautam

Posted on Oct 3

Mitra Local: A Private, Local-First AI Voice Memory & Task Assistant Built for My Friend Ketan

#devchallenge #hf26challenge #hacktoberfest

Hacktoberfest Weekend Challenge: Build for a Friend Submission 🤝

This article is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

What I Built

Meet My Friend Ketan

My childhood friend Ketan runs a textile and garment trading firm in Ahmedabad, Gujarat. If you spend even one morning with him at his shop or warehouse, you will notice three things immediately:

His phone rings unceasingly. Between 9 AM and 7 PM, he takes over 50 phone calls from yarn spinners, dye mills, courier agencies, brokers, and retail store owners.
He switches languages three times in a single sentence. He will greet in Gujarati, negotiate quantities in Hindi, and discuss dispatch timelines or bank transactions in English.
He is constantly making verbal commitments on the go: > "હા Maheshbhai, કાલે સવારે તમને sample swatches મોકલાવી દઉં છું." (Yes Maheshbhai, sending you the sample swatches tomorrow morning.) > "Sureshji, aapka 45,000 ka payment RTGS kar diya hai, shaam tak credit ho jayega." (Sureshji, your 45,000 payment was sent via RTGS, will credit by evening.) > "Priya, please make sure the GST invoice for bill #1042 is cleared before Friday 4 PM."

By evening, half of those verbal promises were forgotten or scribbled hastily on scraps of paper, receipts, or scattered WhatsApp self-chats.

Why Off-the-Shelf Tools Failed Ketan

Ketan tried Siri, Google Keep, Notion, and standard Todoist apps. None of them worked for him:

Language Inflexibility: Siri and Google Assistant choked on code-mixed Gujarati-English sentences ("Maheshbhai ne call kari de" became unintelligible gibberish).
High Friction: Notion and Todoist demanded manual typing with date pickers—impossible while inspecting fabric rolls or driving between warehouses.
Privacy & Business Secrets: Ketan strictly refused to upload his customer contact books, invoice figures, and verbal business memos to public corporate AI clouds.

The Solution: Mitra Local

So I promised him: "I will build an assistant designed specifically for how you speak and how you work—running completely on your laptop, 100% private, with zero subscription fees."

That project became Mitra Local (મિત્ર / मित्र — "Friend").

Mitra Local is an open-source, local-first voice memory and task companion. Ketan simply hits a hotkey or clicks the microphone, talks naturally in whatever mix of Gujarati, Hindi, or English comes out of his mouth, and lets Mitra handle the rest:

🗣️ Native Code-Mixed Understanding: Seamlessly parses Gujarati (ગુજરાતી), Hindi (हिंदी), English, and hybrid mixtures without requiring manual language toggling.
🎯 Automatic Intent & Action Extraction: Distinguishes between actionable tasks (calls, deliveries, payments, follow-ups) and knowledge memories (client preferences, phone numbers, notes).
👤 Entity & Person Graph: Mentions of "Maheshbhai" or "Sureshji" automatically link to contact records with conversation histories and pending to-dos.
⚡ Local Voice Feedback: Mitra speaks back a concise confirmation in natural audio so Ketan knows his commitment was captured without staring at the screen.
🔒 100% Offline & Private: Powered by open-source AI (Ollama + Gemma / Llama) and local speech-to-text. Zero cloud telemetry.

Demo

Here is how Mitra Local works in real life when Ketan speaks to it:

1. Real-World Voice Interaction Scenarios

Language Input	Spoken Command	Extracted Action & Intelligence
Gujarati + Eng	"કાલે Maheshbhai ને call કરી દે, shipment વિશે પૂછવાનું છે."	Call Task linked to contact `Maheshbhai`, scheduled for tomorrow, categorized under shipments.
Hindi + Eng	"Sureshji ko bol dena ki 45,000 ka payment RTGS kar diya hai."	Payment Memory & Verification Task linked to contact `Sureshji` with amount `₹45,000`.
English	"Remind me to file GST tax documents by Friday 3 PM."	High-Priority Task with absolute deadline set to Friday 15:00.
Pure Gujarati	"યાદ રાખજે કે રમેશભાઈની દુકાને નવા સેમ્પલ પહોંચાડવાના છે."	Delivery Task linked to `Rameshbhai`'s shop record.
Pure Hindi	"याद रखना कि राजेश का जन्मदिन 15 तारीख को है."	Personal Memory tagged under `Rajesh` in the local memory bank.

2. The Real-Time Workflow

Speak naturally: Ketan clicks the central microphone button (or triggers the local hotkey) and speaks in Gujarati, Hindi, English, or a mix.
Instant Structured Extraction: Within 400 milliseconds, the dashboard dynamically updates with:
- Extracted task title with original code-mixed transcript preserved
- Categorized action type (Call, Payment, Meeting, Delivery, Note)
- Identified person / contact entity
- Normalized due date & priority level
Verbal Confirmation: The local Web Speech API generates an instant natural voice response: > "Call task for Maheshbhai regarding shipment scheduled for tomorrow."
Relational Organization: The task is instantly filed in the Task Matrix and linked to the Person's profile in the SQLite database.

3. Ketan's Reaction

When I handed Ketan the first build on his laptop, he laughed and immediately tested it with his fastest Kathiyawadi Gujarati:

"કાલે સવારે 11 વાગ્યે શર્માજી સાથે પેમેન્ટનું સેટલમેન્ટ કરવાનું છે, યાદ રાખજે!"

Within less than half a second, the card appeared on his screen:

Title: Payment settlement with Sharmaji
Action: Payment / Financial
Person: Sharmaji
Due: Tomorrow at 11:00 AM
Audio Response: "Payment task created for Sharmaji tomorrow at 11:00 AM."

He looked up and said:

"Gautam, this is the first app that actually talks like an Indian businessman. No complex forms, no English-only barriers, and no fear of my data leaving my laptop."

Code

Mitra Local is 100% open source under the permissive MIT License.

GitHub Repository: https://github.com/gunmasterg9/mitra-local

Quickstart: Running Locally in 3 Minutes

1. Clone the Repository

git clone https://github.com/gunmasterg9/mitra-local.git
cd mitra-local

2. Start the Backend

cd backend
python -m venv .venv
# On Windows:
.venv\Scripts\activate
# On macOS/Linux:
# source .venv/bin/activate

pip install -r requirements.txt
python -m uvicorn app.main:app --reload --port 8000

Backend runs on http://localhost:8000 (Interactive API docs at http://localhost:8000/docs).

3. Start the Frontend

cd ../frontend
npm install
npm run dev

Frontend runs on http://localhost:5173.

4. Automated Verification Suite

We built a comprehensive test suite covering multilingual entity extraction, relative date parsing, contact deduplication, and database transactions:

cd backend
python -m pytest tests/

======================= 36 passed, 16 warnings in 8.02s =======================

All 36 unit and integration tests pass, ensuring zero-regression reliability.

How I Built It

Mitra Local is built from the ground up as a resilient, local-first system designed for zero latency and complete data sovereignty.

Architecture Overview

┌────────────────────────────────────────────────────────┐
│             Modern React 19 + Vite Frontend            │
│  - One-click Voice Mic (Web Audio API / MediaRecorder) │
│  - Real-time Task Matrix & Priority Filters            │
│  - Memory Bank & Person Profiles                       │
│  - Local Browser Speech Synthesis (TTS)                │
└───────────────────────────┬────────────────────────────┘
                            │ REST API
                            ▼
┌────────────────────────────────────────────────────────┐
│                FastAPI Asynchronous Engine             │
│  - Audio Transcription Pipeline (Local Whisper / API)  │
│  - Structured Multilingual Extraction Pipeline         │
│  - Heuristic Fallback Engine (Zero-Downtime Guarantee) │
└───────────────┬────────────────────────┬───────────────┘
                │                        │
                ▼                        ▼
┌──────────────────────────────┐ ┌───────────────────────┐
│     Local AI with Ollama     │ │  Local SQLite Engine  │
│  - Gemma 4 12B / Llama 3.2   │ │  - aiosqlite Async DB │
│  - Zero Cloud Telemetry      │ │  - Full-Text Search   │
│  - Structured JSON Schema    │ │  - People & Memories  │
└──────────────────────────────┘ └───────────────────────┘

Technical Stack

Frontend: React 19, Vite, Lucide Icons, Vanilla CSS design tokens with sleek dark mode, Web Speech Synthesis API.
Backend: FastAPI (Python 3.12), Pydantic v2 schemas, SQLAlchemy with aiosqlite for asynchronous local database transactions.
Local AI Inference: Ollama running open-weight models (gemma4:12b, llama3.2, or mistral).
Storage: Local SQLite database stored securely in backend/data/mitra.db on Ketan's machine.

Solving the Multilingual Code-Switching Challenge

Indian business conversations freely mix Gujarati, Hindi, and English with honorifics ("-bhai", "-ji", "-ben"). Conventional prompts often fail by either translating names into English words (e.g. converting "Sureshji" into "Sir Suresh") or dropping dates.

To achieve bulletproof parsing, Mitra combines two layers:

Structured Few-Shot Prompting with Open LLMs: We prompt local models with strict JSON output schemas that explicitly understand Indic code-mixing:

   {
     "intent": "create_task",
     "task": {
       "title": "Call Maheshbhai regarding shipment",
       "title_original": "કાલે Maheshbhai ને call કરી દે, shipment વિશે પૂછવાનું છે.",
       "action_type": "call",
       "person_name": "Maheshbhai",
       "due_date": "2026-10-04",
       "priority": "medium",
       "language": "gu-en"
     },
     "speech_response": "Call task for Maheshbhai regarding shipment scheduled for tomorrow."
   }

Heuristic Linguistic Fallback Engine: Laptops in the field don't always have GPU acceleration, and background tasks might temporarily tie up resources. Mitra includes an integrated regex- and rule-based Indic NLP engine that recognizes common action verbs in Gujarati ("call kari de", "moklavano chhe", "poochhvanu chhe"), Hindi ("bol dena", "bhej do", "yaad rakhna"), and English ("remind me", "pay"), along with relative temporal expressions ("kaale", "parso", "kal"). Even with Ollama paused, Mitra processes inputs flawlessly in under 5 milliseconds.

Why Does Open Innovation Matter?

When starting this project for Ketan, the easiest route would have been wiring up a quick commercial cloud API (OpenAI or Claude) with a credit card. But for Ketan, and millions of independent business owners like him, closed cloud AI is fundamentally the wrong answer.

Here is why open innovation and open-source AI were strictly essential to solving this problem:

1. Absolute Privacy and Data Sovereignty

In the textile trading world, commercial relationships and pricing are hard-won trade secrets. A typical memo contains:

Supplier identities and credit balances
GST numbers and pending payment amounts
Private personal numbers and commitments

Sending audio snippets or text logs of these conversations to third-party public clouds exposes small business owners to data harvesting, telemetry profiling, and compliance vulnerabilities. With open-weight models running on Ollama and local SQLite storage, Ketan has 100% mathematical certainty that not a single byte of his business data ever leaves his hardware.

2. Zero Recurring Overhead for Everyday People

Proprietary AI SaaS platforms charge $20 to $50 per user per month, plus metered API token fees. For an independent merchant, recurring software subscriptions add up quickly and become a financial burden. Open-source models (like Gemma and Llama) democratize state-of-the-art intelligence: once downloaded, they run indefinitely with zero subscription costs, zero token meters, and zero paywalls.

3. Freedom to Adapt for Underrepresented Languages

Commercial AI models are optimized primarily for formal Western corporate English. They frequently misinterpret code-mixed regional Indian dialects like Gujarati-English or Hinglish, treating common honorifics as foreign errors or hallucinations.
Because open models and open-source code can be inspected, steered, and fine-tuned locally, we were able to build customized prompt pipelines and hybrid heuristic engines tailored specifically to Gujarati merchants. Open innovation puts the power to build AI in the hands of the communities that need it most, rather than waiting for Silicon Valley giants to prioritize regional dialects.

4. True Offline Autonomy

Wholesalers and traders frequently operate in basement warehouses, noisy textile markets, and transport hubs with spotty cellular reception. A tool that depends on roundtrips to an external cloud server will fail right when it is needed most. Open-source local inference delivers instant, reliable performance anywhere—on a flight, in a godown, or during an internet outage.

Conclusion & What's Next

Building Mitra Local for Ketan showed me that the most impactful AI tools aren't generic chatbots—they are personalized, empathetic utilities built for the specific rhythms of real people's lives.

Our upcoming roadmap includes:

Fine-tuning a quantized Whisper model on regional Gujarati and Kathiyawadi accents
Local WhatsApp Desktop webhook bridge for automated reminder dispatch
Offline vector search using DuckDB / ChromaDB for multi-year business archives

Thank you to the DEV Community and Hacktoberfest for championing open innovation that makes technology truly personal and accessible.

Links:

GitHub Repository: https://github.com/gunmasterg9/mitra-local
License: MIT License

devchallenge #hacktoberfest #hf26challenge