DentaNote: dental clinical notes with local Gemma, no audio leaves the operatory

#devchallenge #hf26challenge #ai #opensource

Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady Submission 🦥

I see patients, dictate the visit note, and still lose time building the structured record — fields, tooth numbers, the whole drill. DentaNote is the tool I wrote to cut that step: audio in, structured clinical note out, running 100% locally, with not a single byte leaving my machine.

What I built and how much effort it saves

The full pipeline — transcription with Whisper, extraction with Gemma via Ollama, schema validation, and rendering — took 11.2 seconds to turn a 30-second audio clip into a complete note (transcription: 7.7 s; extraction: 2.1 s; rendering is a deterministic template and doesn't register in the timing).

The small Whisper model on CPU transcribes ~3.8× faster than real time. So a 3-minute dictation becomes text in about 45 s, plus ~2 s of extraction — under a minute for a note I would have dictated entirely.

My own baseline: typing a structured visit note takes me about 10 minutes. Against the pipeline's under-a-minute, that is roughly 9 minutes back per appointment — and across a full day of visits, that time compounds into more than one appointment's worth of work.

Demo

▶ Watch the 2-minute demo — dentanote run visit.wav in the terminal and the Streamlit interface opening with the finished note, the always-visible draft warning and the uncertain_items flagged for review.

The real demo I ran: two synthetic cases, audio to note, with timings recorded in metrics.jsonl (durations and counts only — zero PHI, and an automated test guarantees that).

uv sync && dentanote doctor      # checks Ollama, models and GPU
dentanote run visit.wav          # audio -> structured clinical note
uv run streamlit run src/dentanote/app.py   # upload interface

Why open-source and local beat the closed APIs here

Privacy / LGPD (Brazil's data protection law): appointment audio is patient data. In a local pipeline it goes from the microphone to Whisper on the same machine. The only network call is localhost:11434. This isn't "trust us" — it's architecture.
Offline: works with the internet unplugged (and the test suite runs with sockets blocked, precisely to prove it).
Cost: Ollama + open weights, no per-token bill.
Swap models and compare: I benchmarked two of them on the same corpus of 12 synthetic cases:

Model	Precision	Recall	F1	Hallucination	`uncertain_items`	Avg. latency
`gemma4:e4b`	0.92	0.89	1.00	11.1%	100%	1.70 s
`gemma3:4b`	0.75	0.65	0.92	31.8%	100%	5.59 s

Both produced valid JSON on 100% of first attempts. Both failed on the same case (urgencia_dor predicted tooth 37 instead of 47 — the model "picked" a neighbor on the arch). With a closed API I couldn't swap models and compare like this — my corpus, my hardware, as many times as I want.

How it was built

flowchart LR
    A[Audio wav/mp3] --> B[Local Whisper]
    B --> C[Ollama + Gemma]
    C --> D{Pydantic validation}
    D -- invalid --> C
    D -- valid --> E[Deterministic template]
    E --> F[Clinical note + uncertain_items]

Three decisions that prevent the classic "LLM invents dental facts" failure:

Structured output via Ollama's format (JSON Schema) — constrained decoding puts zero probability on tokens outside the schema. Even so, the client validates with Pydantic, because a "thinking" model can skip the schema (yes, this really happens — documented as ADR-001 in the repo).
Deterministic template, never an LLM, in the final step. The rendered note only transcribes fields that were already validated. If a field didn't come from the audio, it doesn't appear — no "hallucination" where silence was required.
uncertain_items as a safety valve. Invalid FDI number? A tooth mentioned ambiguously? It's listed for the clinician to check, never silently resolved. In one of my cases, an FDI 56 (molar in the wrong quadrant) was discarded and surfaced exactly that way.

Structure: 7 phases, one commit per phase, lint + tests before every one (153 tests today), short ADRs in the repository.

What went wrong and honest limitations

The evaluation covers 12 synthetic cases, with transcripts I wrote myself and synthetic audio (piper-tts). It is not a clinical study, it has no statistical power, and real speech is much messier.
Hallucination is not zero: gemma4:e4b invented at least one item in 11.1% of cases (gemma3:4b, in 31.8%). That's why the output is always a revisable draft — it's pre-filling, not a signed record.
The neighbor-swap error: 37 → 47 in the urgency case. The framework doesn't catch that on its own; only human review does.
Whisper's CUDA stack was broken on my machine (libcublas.so.12 missing) — the CPU fallback worked live, but it's a reminder that "local" also means "you are the tech support".
No real data was used, by decision and by rule. Testing with real data requires compliance with LGPD and professional council regulations — out of scope for this project.

What's next and an invitation to contribute

Manual field editing in the UI before the final text — the clinician adjusts item by item in Streamlit and only then renders the draft.
More TTS voices and languages in scripts/make_synthetic_audio.py (synthetic audio in ES/EN to exercise the bilingual pipeline).
A bigger corpus: package more synthetic scenarios (implantology, periodontology, orthodontics) with the matching expected/ — scripts/evaluate.py already measures the impact of each one.

The repository is open source under the MIT license: if you work with clinical notes and want to tune the procedure vocabulary or add cases from your own day (necessarily synthetic), the issues and CONTRIBUTING.md are there.

Repository on GitHub · Legal and ethical notice in the README