A lecture is one of the most common ways we learn, and one of the least accessible. A student who is deaf can't follow the audio. A student who is blind can't read the slides. A student with a cognitive or attention difficulty may struggle with a 60-minute recording and no summary.
Accessibility tools exist for each of these problems, but they are usually separate products. I wanted to see how far one pipeline could go, so I built AccessLearnAI: upload a lecture, and get back a transcript, a concise summary, spoken audio, and Braille output.
This post covers how it's built, the problems I ran into, and what I'd change.
The idea
Students need different formats of the same content. So instead of building one feature per disability, I built one pipeline that produces every format from a single input:
Whisper (speech to text) → T5 (summarization) → edge-tts (text to speech) → Braille generation
Each stage feeds the next:
- Whisper transcribes the lecture audio. I chose it because it is open source and holds up well on varied accents and recording quality, which matters for real classroom audio.
- T5 condenses the transcript into a short summary, so a student can review a lecture in minutes.
- edge-tts converts the summary back to natural-sounding speech for students who prefer to listen.
- Braille generation converts the text into Braille for blind and low-vision users.
Architecture
The system is split into three layers:
- Backend: Fast API with PostgreSQL for storage
- Frontend: React with TypeScript
- Deployment: Docker containers on GCP, with the frontend on Vercel
I used Fast API because the ML code is in Python, and it let me expose each pipeline stage as a clean API endpoint without a translation layer between the models and the web service.
The hard part: running four models on a free tier
The pipeline works fine on a development machine. Production was a different problem.
When I deployed the backend to Render's free tier, it kept running out of memory. The cause was that Whisper and T5 were both loaded into memory at the same time, and together they went over the free tier's limit.
This was a useful lesson
A model that works locally can still be unusable in production because of resource limits. Deploying an ML system means managing memory as carefully as you manage code.
Top comments (0)