DEV Community

Cover image for I Built an Eye-Controlled Communication App Using Just a "Webcam"
Mustafa Medhat
Mustafa Medhat

Posted on Edited on

I Built an Eye-Controlled Communication App Using Just a "Webcam"

What if someone could communicate using only their eye movements and a regular webcam?

That question inspired me to build Eye Voice, an open-source, browser-based assistive communication prototype.

It allows users to navigate a communication board, select phrases, and speak them using gaze-based controls.

See It in Action

Eye Voice demonstration showing a user controlling a communication board through eye movements using a regular webcam

How Does It Work?

Eye Voice combines several technologies:

  • MediaPipe: Detects facial and iris landmarks through a webcam.
  • Gaze Calibration: Adapts eye tracking to the user's position.
  • Dwell Selection: Triggers actions when the user looks in a direction for a certain time.
  • Speech Recognition & Text-to-Speech: Enables voice interaction.
  • Local Processing: Keeps camera frames on the user's device.

Instead of precise cursor tracking, the system uses five gaze directions: Up, Down, Left, Right, and Center. This makes interaction more practical with an ordinary webcam.

Why Build It?

Dedicated eye-tracking devices can be expensive and inaccessible to many people.

I wanted to explore a more accessible alternative using everyday hardware:

  • No dedicated eye-tracking hardware.
  • No backend server required.
  • Browser-based operation.
  • Keyboard and touch alternatives.
  • Privacy-focused local camera processing.

Optional Gemini-powered suggestions send only transcribed text to the API, not camera frames or audio.

Try It Yourself

Live Demo: Eye Voice

GitHub Repository: Source Code

The project is open source under the MIT License, and contributions are welcome.

I'm especially interested in feedback on calibration accuracy, accessibility, Arabic localization, and browser compatibility.

Note: Eye Voice is an experimental prototype, not a certified medical device or a replacement for professional assistive technology.

What's Next?

I'm continuing to improve the project and would love to hear from developers and accessibility enthusiasts.

If you tried Eye Voice, what would you improve first?

Feel free to share your thoughts, questions, or ideas in the comments!


Written by Mostafa Medhat, with AI assistance for editing and English phrasing. Technical details and project decisions were reviewed by the author.

Top comments (0)