Noted AI: A Local-First, Privacy-Preserving AI Note Assistant

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built Noted AI ("Your thoughts, organized by AI. Your notes, under your control.") for my friend, who is currently managing a packed schedule of university lectures, academic study sessions, and team project meetings.

The Problem It Solves

My friend constantly ran into three major friction points:

1. Scattered, Messy Inputs: Information was coming in from everywhere—spoken lectures, raw audio voice recordings, slide deck PDFs, and quickly typed scratch notes.
2. Cognitive Overload & Manual Revision: Manually combing through hour-long lecture transcripts or 30-page slide PDFs to extract key definitions, action items, and flashcard revision questions was consuming hours of valuable study time.
3. Privacy Concerns & API Paywalls: Commercial AI note apps require recurring monthly subscriptions ($15–$25/month) and send private voice recordings and proprietary course materials to third-party cloud servers.

What Noted AI Does

Noted AI is a local, privacy-first personal AI note assistant designed specifically around my friend's workflow:

• Instant AI Summarization: In one click, local AI extracts a concise executive summary, bulleted key takeaways, important terminology, and actionable next steps—without altering or overwriting the original notes.
• Voice-to-Notes: Upload spoken audio recordings (.mp3, .wav, .m4a). Noted AI transcribes them locally and automatically structures stream-of-consciousness speech into clean notes with clear headings and summaries.
• PDF-to-Notes: Upload slide decks and reading materials. Noted AI extracts digital text and generates structured study guides while detecting scanned/image-only documents.
• Strict Note-Scoped Q&A: A dedicated Q&A assistant scoped strictly to the active note. If a fact isn't present in the note, the AI explicitly states that rather than hallucinating an answer.
• Interactive AI Revision Quiz: Generates a 5-question multiple-choice revision test with an interactive in-browser quiz runner, instant scoring, and explanation cards for active recall practice.

Code

The entire codebase is open-source and available on GitHub:

song-code-won / noted-ai

Noted AI: A Personal AI Note-Taking Assistant powered by local open-weight AI (Gemma 3 & faster-whisper). Built for Hacktoberfest 2026.

Noted AI — A Personal AI Note-Taking Assistant

Tagline: Your thoughts, organized by AI. Your notes, under your control.
Built for the Hacktoberfest 2026 DEV Challenge — Build for a Friend.

Python 3.11+ · FastAPI · Ollama · Speech-to-Text · License: MIT

1. Project Overview

Noted AI is a lightweight, simple, and functional AI-powered note-taking application designed to capture, organize, summarize, and retrieve knowledge effortlessly.

Rather than being another generic AI chat wrapper, Noted AI is engineered as an active cognitive assistant that converts raw thoughts, recorded audio, and PDF documents into clean, structured notes—running 100% locally using open-weight AI models.

2. The Problem & Who It Is Built For

The Friend & The Use Case
This project was built for my friend, who is actively balancing intense study sessions, university lectures, and collaborative team meetings.

The Pain Points

• Scattered Information: Information arrives in unpredictable formats—spoken audio from lectures, lecture slide PDFs, and messy quick thoughts.
• Cognitive Overload: Manually reading…

Repository: GitHub Repository

Tech Stack Overview

• Backend: Python 3.11+, FastAPI, Uvicorn
• Local Open-Weight LLM: Google Gemma 3 (gemma3:1b / gemma3:4b) running locally via Ollama
• Speech-to-Text: faster-whisper (CTranslate2) running locally on CPU
• Document Processing: PyMuPDF (pymupdf)
• Database: SQLite with Write-Ahead Logging (WAL) mode for persistent storage
• Frontend: Clean, responsive HTML5 / CSS3 / Vanilla JavaScript dashboard with split-view editor
• Testing & Containerization: 19 automated Pytest cases, Dockerfile, and docker-compose.yml

How I Built It

1. Open-Weight AI Architecture with Ollama & Gemma 3

Rather than calling commercial cloud APIs like OpenAI or Anthropic, Noted AI's core intelligence is powered by Gemma 3 running locally through Ollama (http://localhost:11434).

We designed a dedicated AIService module that handles:

• Structured prompt formatting for key-point and action-item extraction.
• Strict anti-hallucination prompting for the Q&A engine so answers remain grounded only in the user's note.
• JSON-formatted output parsing with robust regex fallbacks for 5-question MCQ generation.
• Dynamic fallback error reporting with actionable troubleshooting instructions if Ollama is paused or the model needs to be pulled.

2. Local Speech-to-Text with faster-whisper

For voice recordings, we integrated faster-whisper, an optimized CTranslate2 reimplementation of OpenAI's Whisper model. By utilizing the quantized tiny or base weights on CPU, Noted AI transcribes audio files up to 4x faster than standard PyTorch models with minimal memory overhead, followed by an automatic Gemma 3 structuring pass.

3. PDF Parsing with PyMuPDF

Text from multi-page PDFs is extracted directly using PyMuPDF. We implemented automated character-density detection so that if my friend uploads a scanned or image-only document, Noted AI flags it immediately and prompts for a digital PDF rather than failing silently.

4. Zero Data Loss with SQLite WAL

To ensure notes survive system crashes and restarts without requiring a bulky database server, notes and AI summaries are stored persistently in a local SQLite database configured with PRAGMA journal_mode=WAL; and indexed by modification timestamps.

Why Does Open Innovation Matter?

Building Noted AI with open-weight models and open-source tooling demonstrated why open innovation is critical for real-world personal software:

Complete Data Sovereignty & Privacy

Personal thoughts, course notes, and voice recordings never leave the laptop. For students and professionals dealing with confidential lectures, proprietary team discussions, or sensitive research, closed cloud APIs represent an unacceptable privacy trade-off. With Gemma 3 and faster-whisper, inference happens 100% on localhost.

Democratizing Accessibility (Zero Subscription Barriers)

My friend shouldn't have to budget $20/month just to summarize study notes or prepare for an exam. Open models turn consumer laptops into self-sufficient AI workstations without credit card requirements or surprise monthly billing.

True Model Flexibility

Because the system communicates with Ollama through open standards, my friend isn't locked into a single provider. With a single environment variable change, the model can switch between gemma3:1b for battery conservation on an airplane, gemma3:4b for deep revision, or open models like llama3 or mistral.

Auditability and Transparency

Every prompt, processing pipeline, and storage schema is open and verifiable. There are no black-box system prompts or hidden telemetry collecting personal data.

Real User Feedback (From My Friend)

After testing Noted AI with actual course materials and lecture recordings on an Intel i7 laptop (16GB RAM):

"The voice-to-notes workflow saved me nearly half an hour after a two-hour lecture. Instead of re-listening to messy audio, having the transcript cleaned up with headings and action items in one place made review instantaneous.

The revision quiz was surprisingly fun and accurate—it asked questions about the exact concepts covered in the notes rather than generic trivia."

Prize Categories

• Primary Track: Build for a Friend
• Open-Source AI / Open-Weight Model Track: Powered by Google Gemma 3 via Ollama & faster-whisper