DEV Community

Syeda Kanza Abid
Syeda Kanza Abid

Posted on

TALK_TO_TEXT

#llm #ai #productivity

`

Building TalkToText Pro: An AI-Powered Meeting Notes Rewriter

The Problem

Meetings moved online — Teams, Google Meet, Zoom — but the way we document them didn't. Someone still has to scribble notes while also listening, and that means missed decisions, forgotten action items, and inconsistent records. Teams needed a way to turn a recording into clean, structured, actionable notes automatically.

The Idea

TalkToText Pro is an end-to-end system that does exactly that:

  1. Transcribe meeting audio with Whisper (speech-to-text).
  2. Translate to English when the meeting wasn't in English.
  3. Optimize the text (strip filler words, manage token limits).
  4. Generate structured notes with an LLM — summary, key points, decisions, action items, sentiment.
  5. Store every version in MongoDB and let users export PDF/Word and share.

The Tech Stack

  • Backend: Django + Django REST Framework + JWT auth
  • Async processing: Celery + Redis (jobs run in the background; the UI shows live per-step progress)
  • Databases: MongoDB (meetings & jobs) and SQLite (auth users)
  • AI pipeline: Whisper for transcription — local by default, with a pluggable STT backend that can switch to faster-whisper (CTranslate2, roughly 4× faster on CPU) or the OpenAI Whisper API without touching pipeline code; free web translation; and a local Qwen LLM for summarization — all behind pluggable engine interfaces
  • Frontend: a responsive vanilla HTML/CSS/JS single-page app served by Django
  • Exports: PDF (reportlab) and Word (python-docx)

Design Highlights

The most important design decision was to hide every AI service behind a small interface:

python
class STTEngine: def transcribe(self, audio_path, language) -> str
class TranslationEngine: def translate_to_english(self, text, lang) -> str
class SummarizerEngine: def summarize(self, optimized_text) -> dict

That means the free, local implementations (Whisper, free translate, Qwen) can be swapped for paid, faster backends (OpenAI Whisper API, ChatGPT) with zero changes to the pipeline code. We started free and left the door open to paid.

The pipeline runs inside a Celery worker, so an upload returns a job_id immediately and the frontend polls /api/jobs/<id> to render a live 4-step progress bar — transcribe → translate → optimize → generate.

Beyond v1 — a product-grade frontend

Once the pipeline worked, I rebuilt the UI to feel like a real AI SaaS product instead of a demo:

  • Voice recording right in the browser — a microphone button, live recording timer, and a real-time waveform drawn from the Web Audio API. Stopping the recording packages it as a .webm and pushes it through the exact same pipeline as an upload.
  • Audio upload alongside recording, with drag-and-drop and clear per-step progress while Whisper + the LLM do their work.
  • Transcript editor — the raw/translated transcript appears in an editable text area with Copy and Download .txt, next to the structured notes (summary, key points, decisions, action items, sentiment) with PDF/Word exports and share.
  • Transcription history with status badges, plus a dark, premium UI — glass cards, gradient accents, animated waveform, smooth transitions, and a responsive layout.

Results (NFRs)

We measured the system against the SRS non-functional requirements:

Requirement Target Measured
Accuracy ≥ 85–90% 95.1% (WER 0.049)
Secure Access restrictions JWT + ownership + protected audio ✅
Scalable Concurrent uploads Verified with 2 parallel jobs ✅
Usable Simple + responsive Responsive SPA ✅
Performance 30-min → 1–2 min ~77 min projected on CPU — hardware-limited; fixable by swapping in a faster engine

The performance number is the honest trade-off of running everything locally on a 2-CPU cloud box. The architecture already has the escape hatch: point the summarizer (or the transcriber) at a paid API and the same code meets the target.

Lessons Learned

  • Pluggable interfaces pay off. Swapping a whole model (or an entire AI vendor) was a one-line change.
  • Async processing makes progress UX easy. With Celery + a status document, "live progress" is just polling.
  • Test data with known ground truth is gold. We generated TTS audio from a script, so we could compute a real Word Error Rate.
  • Document the hardware reality. NFRs like "1–2 minutes" are only meaningful with the hardware stated.

Try It

The full source is on GitHub (public): https://github.com/syedakanza13/talk_to_text

bash
pip install -r requirements.txt
docker compose up -d
cd backend && python manage.py migrate
STT_BACKEND=faster WHISPER_MODEL_SIZE=base celery -A config worker --pool=solo &
python manage.py runserver

Register, upload a recording (or click the mic), and watch your meeting turn into notes.

Built as a Generative AI Odyssey project. AI-generated visuals, if any, would be attributed here.
`

Top comments (0)