Clarity: Understanding, Out Loud

This is a submission for Weekend Challenge: Passion Edition

Inspiration

Passion doesn't only live on a pitch or in a stadium chant. Sometimes it's quieter than that — it's the stubborn, protective love that makes you want to shield someone from feeling stupid or scared.

Everyone has a person like this in their life: a grandparent who freezes when a dense hospital bill shows up in the mailbox. A parent whose English is a second language and who reads "final notice" and assumes the worst. A friend with a reading disability, low vision, or dyslexia who has to ask someone else to make sense of their own mail for them. Official mail is written by institutions, for institutions — dense, jargon-heavy, and never for the person actually holding the envelope, hands shaking a little, afraid to open it alone.

The real problem was never that these people can't hear — it's that they can't understand. So Clarity isn't a text-to-speech gadget. It's a comprehension tool first: it reads a confusing document the way a patient, well-informed friend would — breaking it down, translating the jargon, telling you what actually matters — and only then does it speak, because understanding something and having no way to absorb it is its own kind of cruelty.

Clarity is built out of that same fire people usually save for rivalries and fandoms, just pointed somewhere quieter: nobody I love should ever feel powerless in front of a piece of paper they can't make sense of.

What it does

You hand Clarity a document — a photo of a paper letter, a PDF, or just pasted text — and it:

• Understands it and breaks it down, not just transcribes it. A calm, structured explanation: what this document actually is, the parts that matter, exactly what you need to do and by when, and anything to watch out for (fees, penalties, easy-to-miss deadlines) — turning a wall of legal or medical jargon into something a person can actually act on.
• Adapts to how you understand best — "explain it like I'm 10," plain everyday language, or full precision that preserves legal/technical detail for someone who wants it straight.
• Speaks the explanation out loud, with each word highlighting in perfect sync as it's read — genuinely useful for low vision, dyslexia, or anyone who processes speech better than dense paragraphs.
• Speaks it in 9+ languages — the same English medical bill, understood and read back in Spanish, Hindi, Arabic, and more.
• Takes follow-up questions by voice — tap the mic and ask "how much do I actually owe, and by when?" — answered directly from the document, and read back to you too.
• Turns real deadlines into calendar reminders — one tap adds "pay by April 22" straight to your calendar as a .ics file, but only when the document actually states a real date (it won't guess at vague phrasing like "within 30 days").

It runs as a single, no-build Node/Express server with a vanilla HTML/JS frontend — your document is processed entirely on your own machine. The only thing that ever leaves it is a few sentences of already-plain-language explanation text, sent to ElevenLabs to become audio.

See it in action

I ran Clarity against three completely different kinds of confusing paperwork, live, with a real AI model doing the understanding and real ElevenLabs audio doing the speaking — no canned demo text:

Document	What Clarity understood and explained
🏥 Hospital bill ($1,284.67 balance)	Deductible + coinsurance breakdown, the exact due date, late-fee and credit-impact warnings, a glossary entry for "coinsurance"
🧾 IRS CP2000 tax notice	The $2,161.05 total owed, the April 22 payment deadline (with a working "Add to calendar" button), the 30-day response window, glossary entries for "Form 1099-NEC" and "CP2000"
⚖️ Jury duty summons (explained "like I'm 10")	Eligibility rules, the May 1 confirmation deadline, pay rate, and a plain-English glossary for "probation" and "felony conviction"

And the signature moment — the explanation read aloud with word-by-word highlighting, driven by ElevenLabs' actual audio timing, not a guess:

Code

Aryakoste / Clarity

Clarity 🔊

A voice companion that reads and explains confusing letters, bills, and medical notices — in plain language, out loud.

Millions of people freeze when official mail arrives: dense legal wording, medical jargon, "final notice" red ink. That burden falls hardest on elderly people, people with low vision or dyslexia, and non-native speakers. Clarity takes a photo or PDF of that letter and gives back a calm, spoken explanation: what it is, the parts that matter, exactly what you need to do, and what to watch out for — read aloud with the words highlighting as it speaks. You can then ask follow-up questions with your voice.

Built to run entirely on your own machine — your documents never go to a third party (except the few sentences of explanation sent for text-to-speech).

---

What it does

• 📄 Reads any document — upload a PDF, snap a photo of paper…

View on GitHub

How I built it

• Node.js/Express server, single file, no build step. Holds the ElevenLabs key server-side — the browser never sees it.
• Text extraction: pdfjs-dist for PDFs, tesseract.js for OCR on photographed letters.
• The understanding layer: a swappable LLM — local Ollama (llama3.2, $0 cost, runs entirely on-device), or hosted OpenAI/Anthropic — that turns raw, jargon-heavy document text into a structured plain-language breakdown: summary, key points, actions with deadlines, warnings, and glossary.
• The voice: ElevenLabs' text-to-speech API, specifically the with-timestamps endpoint, which returns character-level timing alongside the audio. The frontend maps that real alignment data onto individual word spans, so the highlight isn't a fake proportional estimate — it's actually locked to the audio.
• Voice input: the browser's built-in Speech Recognition API for asking follow-up questions hands-free — no extra key required.
• Calendar export: client-side .ics generation, only offered when a real, parseable date exists — no guessing at vague deadlines.

Challenges I ran into

• Real word-level sync is harder than it looks. My first pass at highlighting used a length-weighted time estimate per sentence — good enough to demo, but not actually synced to the audio. Rebuilding it on ElevenLabs' character-level alignment data meant reworking the transcript into per-word spans and mapping character offsets back to real timestamps, with a graceful fallback to the old estimate if alignment ever comes back empty.
• A live 402 from ElevenLabs. Mid-testing, the app's original default voice started returning payment_required — turns out ElevenLabs recently restricted "library" voices to paid plans via the API. I caught it, pulled the account's actual available voice list, and swapped in a working free-tier voice ("Sarah") so the app isn't silently broken for the next person who clones it.
• Trustworthy dates, not guessed ones. For the calendar export, I deliberately only parse deadlines that resolve to an unambiguous date — "April 22, 2026" gets a calendar button, "within 30 days" doesn't. For a tool handling bills and legal notices, a wrong guess about understanding is worse than no shortcut at all.

Accomplishments I am proud of

Getting the whole pipeline — OCR/PDF extraction → real AI comprehension and plain-language rewriting → real ElevenLabs voice with true word-level sync → calendar export — working end-to-end, verified live against three genuinely different real-world document types, with zero console errors and a recorded video + audio to prove it.

What I learned

That "passion project" doesn't have to mean something loud. Some of the most stubborn devotion shows up as making sure a scary, confusing envelope never gets the last word — and that understanding, not volume, is the thing that actually needs delivering first.

What's next

• A caregiver "share this explanation" mode, so an adult child can see exactly what their parent's letter said and meant.
• A lightweight "does this look like a scam?" pass, given how much of this mail is designed to look official whether it is or not.
• True multilingual voice models (eleven_multilingual_v2) for richer non-English read-alouds.

Built with

Node.js · Express · pdfjs-dist · tesseract.js · ElevenLabs (Text-to-Speech + timestamps API) · Ollama (llama3.2) · Web Speech API · vanilla HTML/CSS/JS

---

Prize category: Best use of ElevenLabs

Clarity's core job is comprehension — the real AI-driven breakdown of a confusing document into plain language is what makes any of this useful. But comprehension that stays trapped on a screen doesn't reach everyone it needs to: the whole point is to hand that understanding to people who read poorly, see poorly, or read a different language — and that handoff happens entirely through ElevenLabs. Specifically, I use:

• The with-timestamps TTS endpoint, not just plain audio — its character-level alignment data is what drives real, accurate word-by-word highlighting, rather than a fake time estimate, so the explanation is followable in real time, not just audible.
• Multiple output languages through the same voice pipeline, so the same plain-language explanation can be read back in Spanish, Hindi, Arabic, and more.
• Voice configuration as a first-class setting (voice ID, model ID) — and I hit and fixed a real free-tier API policy change along the way, which says something about how deep the integration actually goes.