Zero APIs, Zero Data Leakage: Running an On-Device AI Developer Assistant

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What It Does

DevWhisper is a 100% offline desktop tool that transcribes client meeting recordings locally using whisper.cpp and uses a local open-weight coding model (Qwen2.5-Coder via Ollama) to automatically extract technical action items, code requirements, and draft client follow-up emails.

Who Would Pay For It

Freelance software developers, independent engineering consultants, and small dev agencies.

The Problem It Solves

NDA & Privacy Breaches: Cloud note-takers (Otter, Fireflies) upload sensitive client audio and proprietary code specs to third-party servers, breaking NDAs. DevWhisper processes everything on-device with zero data leakage.

Generic Output: Cloud tools miss technical context (API routes, bug reports, variable names). DevWhisper uses code-specialized open models to output structured, Git-ready developer tasks.

High Monthly SaaS Fees: Instead of paying $15–$30/month per seat for cloud tools, developers get unlimited execution for $0 in marginal API costs.

Code

coderForLife-A1 / DevWhisper

🔒 DevWhisper Desktop

100% Offline, Privacy-First AI Meeting Notes & Task Extractor for Developers
Native Windows desktop app capturing Google Meet and Zoom audio via WASAPI loopback with zero cloud dependencies.

---

💡 Why DevWhisper?

Standard AI meeting assistants (Otter, Fathom, Fireflies) upload sensitive client audio recordings to third-party cloud servers. This violates client NDAs, risks proprietary IP leaks, and incurs recurring token fees.

DevWhisper keeps 100% of execution on your local machine:

• Zero Cloud Leakage: Audio streams and transcripts never leave localhost or system RAM.
• Direct Google Meet & Zoom Capture: Records both sides of the call simultaneously (system speaker audio via WASAPI loopback + your microphone) without needing meeting bots or invite permissions.
• Developer-Focused Context: Open-weight coding models (qwen2.5-coder:7b) extract structured technical tasks, priority tags, assignees, and client email drafts instead of generic chatter.
• Searchable Meeting Vault: Local SQLite database with FTS5 full-text search across all meeting…

View on GitHub

How I Built It

DevWhisper runs on a 100% on-device, zero-cloud execution pipeline.

1. The Open AI Stack
2. Local Speech-to-Text (whisper.cpp): Converts audio to text locally using C++ bindings (pywhispercpp). No cloud uploads, zero latency, complete NDA compliance.
3. Local Open-Weight LLM (Qwen2.5-Coder via Ollama): Uses qwen2.5-coder:7b locally. It understands technical specs, API endpoints, and Git workflows far better than generic models.

4. Structured Inference (Pydantic + Ollama): Enforces strict JSON schemas natively through Ollama to guarantee reliable task structures.

5. How It Works

6. Drop a meeting recording into the local dashboard (localhost:8000).

7. whisper.cpp transcribes the audio directly on-device.

8. Qwen2.5-Coder extracts technical tasks, assignees, and drafts a client follow-up email.

9. Temporary audio files are unlinked from disk immediately after processing.

10. Why Open AI Wins

11. 100% Data Privacy: Zero audio or transcript data leaves the machine.

12. $0 API Bills: Runs indefinitely with zero token costs.

13. Code-Aware: Fine-tuned open coding models beat closed general APIs at parsing developer tasks.

Why Does Open Innovation Matter?

• Building DevWhisper on open-source AI wasn't a design preference—it was the only way to solve the core problem. Closed APIs (like OpenAI or Anthropic) present structural dealbreakers for freelance engineers handling sensitive client work:
• Zero Cloud Data Leakage (NDA Compliance): Closed models require sending raw audio and project transcripts to external servers. This explicitly violates client Non-Disclosure Agreements (NDAs) and exposes proprietary code architectures. Open-weight models allowed me to build an entirely air-gapped pipeline where data stays 100% in local RAM and disk.
• $0 Marginal API Costs: Closed APIs charge per token and per audio minute, penalizing users for long client syncs. Open local inference allows freelancers to run unlimited 2-hour call transcriptions for $0 in API costs.
• Targeted Model Selection: Closed APIs are generalists. Open innovation let me swap out generic conversational LLMs for a specialized coding model (Qwen2.5-Coder), which dramatically outperforms closed models at recognizing framework syntax, API endpoints, and Git task structures.
• Complete Vendor Independence: Closed APIs can change terms, increase pricing, or alter system prompts overnight. Owning the local pipeline via Ollama and whisper.cpp guarantees the tool runs offline indefinitely without vendor lock-in.

My First User

To validate DevWhisper in a real-world setting, I had a senior developer and tech club lead use it to process audio recordings from our recent technical project syncs. They used the tool to parse complex hardware and software specs from long group discussions without uploading sensitive member data to external servers. They loved how accurately it captured technical action items—like pin configurations and API endpoints—that generic note-takers completely missed, and confirmed they would definitely pay a monthly subscription fee to use it regularly for club operations and freelance work.