DEV Community

SHAIK TAUFEEQ AHMAD
SHAIK TAUFEEQ AHMAD

Posted on

Building Shruthi Bandhu: How We Engineered an AI Gesture Tool for the Deaf-Mute Community (And Won the Vishwakarma Awards)

Some wins take time. Over the past year, I’ve walked out of innovation halls with more lessons than trophies. Every post I made was about participation, never victory. Each time, I clapped for others while swallowing the frustration of my own near-misses.

But building Shruthi Bandhu was different.

At IIT Indore, our team won 1st Prize in the HealTech Category at the Vishwakarma Awards 2025 for our AI-powered gesture communication tool. Hosted by the Maker Bhavan Foundation, the competition brought together over 3,600 STEM students from India and SAARC nations. After nine months of prototyping and mentoring, we made it from the top 1,000+ teams down to the final 12—and ultimately took home the win.

But this post isn’t just about the trophy. It’s about the six months of prototyping, the roadblocks we hit, and the engineering decisions we made to build a platform that bridges the communication gap for the deaf and mute community.

The Bottleneck: The Data Drought
When we set out to build an AI that could translate Indian Sign Language (ISL), we hit a massive wall almost immediately: the unavailability of a robust ISL dataset.

You can't train a reliable model without quality data. Since off-the-shelf datasets for ISL were either non-existent or heavily fragmented, we realized we couldn't rely on open-source repositories. We had to build it ourselves.

We didn't just scrape the web; my teammates and I actually sat down and learned Indian Sign Language. We then recorded and manually curated a custom dataset of over 1,000+ videos. It was a tedious, brute-force approach, but it gave us the clean, high-quality foundational data our models desperately needed.

The Architecture: Two Perspectives for Real-World Use
Sign language translation isn't a one-size-fits-all problem. How a user interacts with the world daily is very different from how they interact in a digital workspace. To solve this, we engineered two distinct approaches:

  1. Egocentric View (Daily Communication): We designed this for a spectacles/smart-glasses point-of-view. This allows for real-time translation as the user navigates their physical environment, capturing gestures from a first-person perspective.

  2. Exocentric View (Digital Workspaces): We optimized this for standard laptop webcams. This approach is specifically tailored for virtual meetings and conferences, capturing the user from a standard front-facing angle to ensure accessibility in professional environments.

**
 **
As engineers, it is dangerously easy to build solutions in a vacuum, assuming we know what the user wants. We knew early on that if Shruthi Bandhu was going to be truly accessible, we couldn't just guess at the community's needs from behind our laptops.

We stepped out of the lab and conducted extensive field visits at local deaf and mute schools and specialized clinics. We sat down with educators, clinicians, and the students themselves to observe their daily friction points and understand the nuances of how they communicate.

This wasn't just research; it was strict product validation. We aligned our engineering goals with their real-world needs. By the end of our prototyping phase, this user-centric approach allowed us to secure formal Letters of Intent (LOIs) from several of these institutions, proving that we weren't just building a cool project—we were building a product they genuinely wanted to deploy.

The Village Behind the Build
Seeing hardware-first and assistive tech innovation valued at this scale was validating. Receiving this award in the presence of industry leaders like Hemant Kanakia, Prof. Suhas Joshi (Director, IIT Indore), and Gautam Khanna reminded us that accessibility deserves serious engineering attention.

You don't build something like this alone. A massive thank you to:

My incredible co-founders and teammates: Shivaraj Gollapally, Varun Chiguru, and Subbarayudu Bolisetty. We solved technical bugs, logistical nightmares, and motivational slumps together.

This win isn’t a finish line—it’s a starting point. Competitions end on a stage, but impact begins in the real world.

If you are working in assistive tech, computer vision, or multimodal AI, I’d love to connect. Have you ever had to brute-force a dataset from scratch? Let me know in the comments!

Top comments (1)

Collapse
 
anna2612 profile image
Anna Jambhulkar

This is genuinely inspiring work.

The strongest part for me is that you did not treat accessibility as a “cool AI demo” problem. You went through the hard part: learning ISL, building your own dataset, validating with schools and clinics, and separating egocentric and exocentric use cases instead of assuming one model/interface would fit every situation.

The dataset work is especially impressive. For many real-world AI products, the bottleneck is not the model architecture — it is the quality, relevance, and care behind the data.

Also loved this line in spirit: competitions end on a stage, but impact begins in the real world. Assistive tech needs exactly this kind of serious engineering attention.

Congratulations to the whole team. Wishing Shruthi Bandhu real-world adoption beyond the award stage.