DEV Community: Bright Etornam Sunu

Bridging the Silence: Building a Real-Time Sign Language Translator (Part 1)

Bright Etornam Sunu — Fri, 10 Apr 2026 11:55:49 +0000

Communication is the cornerstone of human connection, yet for the deaf and hard-of-hearing communities, a significant barrier exists when interacting with those who don't understand sign language. While text-to-speech and speech-to-text technologies have advanced rapidly, translating visual, spatial languages like American Sign Language (ASL) into spoken word in real-time has remained a formidable challenge.

Enter the asl-to-voice project.

In this four-part technical series, we will take a deep dive into how we built an end-to-end, continuous sign language recognition (CSLR) pipeline. This system doesn't just recognize isolated gestures; it watches a person signing via a standard webcam, understands the continuous flow of movements, translates those signs into fluent English, and speaks the translation aloud—all in real time.

The Complexity of Sign Language

Before diving into the architecture, it's crucial to understand why this is hard. Sign language isn't just about hand shapes. It involves complex grammar, facial expressions, body posture, and non-manual markers. Furthermore, in natural signing, there are no clear "spaces" between words like there are in written text. This is known as continuous signing.

Traditional computer vision approaches often fail here because they treat signs as static images or isolated video clips. To truly translate sign language, a system must understand spatio-temporal data (space and time) simultaneously. Furthermore, sign language has its own syntax, often represented as "glosses" (capitalized literal representations of signs, like YOU NAME WHAT), which don't map one-to-one to English grammar.

Our Solution: The 5-Stage Pipeline

To tackle these challenges, the asl-to-voice codebase is structured around a highly modular, five-stage pipeline. By breaking the problem down, we can optimize each component independently.

Stage 1: Keypoint Extraction

Processing raw video frames through a heavy neural network is computationally expensive and slow. Instead, we use Google's MediaPipe Holistic to extract 2D and 3D landmarks from the signer's hands, body (pose), and face. This dramatically reduces the dimensionality of our data from millions of pixels down to a dense feature vector (up to 1,662 dimensions) that describes exactly how the body is moving.

Stage 2: Temporal Sequence Modeling

With our stream of keypoints, we need a model that understands time. We implemented a Transformer encoder (with a BiLSTM available as a baseline). The Transformer looks at a sliding window of keypoint frames and learns the temporal relationships between them. Because we are dealing with continuous streams without explicit word boundaries, the model is trained using Connectionist Temporal Classification (CTC) loss.

Stage 3: Gloss Decoding

The output of the Transformer is a probability distribution over our vocabulary of signs. The CTC decoder (using either a fast greedy search or a more accurate beam search) collapses these probabilities into a discrete sequence of glosses.

Stage 4: Gloss to Natural Language

If the system outputs ["STORE", "I", "GO"], the user experience is poor. Sign language glosses need to be translated into grammatically correct spoken language. To achieve this, we route the gloss sequence through a Large Language Model (LLM). Our system uses a resilient fallback chain: it tries Google Gemini first, falls back to OpenAI if it fails, and then to Anthropic.

Stage 5: Text-to-Speech (TTS)

Finally, the natural English sentence (e.g., "I am going to the store.") is sent to a TTS engine. We rely primarily on Microsoft Edge TTS for high-quality, neural voice generation. Crucially, this runs in a background thread so the video feed and inference loop never freeze while the computer is speaking.

A Look at the Tech Stack

The beauty of this project lies in how these diverse technologies are stitched together:

Computer Vision: mediapipe, opencv-python
Deep Learning: torch, transformers
Metrics: jiwer (for Word Error Rate), sacrebleu
APIs: google-generativeai, openai, anthropic
Audio: edge-tts, pyttsx3

Everything in the system is entirely configuration-driven. A single config.yaml file controls model architectures, feature subsets, sliding window sizes, and API fallback chains, making it incredibly easy to experiment.

What's Next?

In Part 2, we will roll up our sleeves and look at the data. We will explore how we process datasets like WLASL, how we normalize keypoints so the model works regardless of where you stand in the camera frame, and how the Transformer model is actually trained to understand sign language.

Stay tuned as we move from concept to code.

uploaded through Distroblog - a platform i created specifically to post to multiple blog sites at once😅

Deploy your Nodejs + Auth0 REST API to Cyclic.sh under 4 minutes

Bright Etornam Sunu — Thu, 16 Dec 2021 06:15:45 +0000

Deploying APIs can sometimes be a pain in the butt when your service provider overcomplicates the deployment and setup process. This short article will demo how to deploy your Restful Nodejs application to Cyclic.sh in less than 4 minutes.

Yes!, you heard right, less than 4 minutes🔥😱😱.

Cyclic is a provider that helps you Launch your API in seconds. Push your code to Github and let the CI/CD (continuous integration/continuous delivery) integration trigger and deploy your service onto a global infrastructure in seconds. No cryptic CloudFormation errors. No mysterious API Gateway errors. No YAML parse errors. No hunting for CloudWatch log groups. No wasted time.

Important!
I already have my Nodejs Auth0 backend done.

To follow along with this project, clone the repo from here.

Deployment Demo
To deploy your codebase, follow the following steps:
The first thing you must do is create a repository on github.com for your project and push your code.

Next, signup to Cyclic.sh. The signup process is seamless, and all you need is to signup using your Github account.

After successful signup, you will see a dashboard; where all the magic happens. You can locate the docs at the top right corner, just right before the profile.

Now you need to deploy your code. Click the "deploy" button (green button) and select the "Link your own" tab.

Search for the repo you want to deploy, in your case "nodejs-auth0," and select it and connect it to your Github account.

At the prompt, you need to confirm your Github access, and after confirming, all you have to do is to approve and install, and that's it 🎉

Once you approve and install, the deployment process will start. 2–3mins should do it 🎊🎉🎊🎉🎊🎉

The final step is to set your environment variables on the dashboard. The dashboard for your project looks like this.

This is a ".env" file; you can also include those configurations on the dashboard by clicking on "Variables." After this configuration, everything should be up and running 🔥

Deploying new changes
After all the setups and configurations, to deploy new changes, push your code to Github, and Github actions will do the rest 😀😉

Conclusion
Deploying a RESTful API shouldn't be hectic and cyclic.sh has made sure deploying your backend code to the cloud is a simple as possible.

If you find any difficulty in the deployment process, you can reach out to the cyclic.sh team on discord.

Do well to follow me on Twitter and LinkedIn to connect.

originally publish on medium.com

How to limit window resize in Flutter Desktop

Bright Etornam Sunu — Mon, 26 Oct 2020 09:27:34 +0000

Hi there!
It has been such a looooong time since I wrote an article or even created a youtube ☹️ tutorial but hey, there’s good news 🎤, I’M BACK 🔥🔥🔥.

In the past few weeks, I have been working on a Desktop app using Flutter (you know one of those side projects you start then drop it then pick it up again…? yeaaaah!) and I had problems with the window screen resizing.
At some point, you can resize it such that nothing else displays which is an undesirable feature. Hence I started doing some research on how to limit the resize of the window.

After talking to a number of developers and also reading some articles, this effect can “only” be done natively. This requires editing some files for the MacOS, Windows and Linux file folder. This will be a bit stressful especially if the developer is not accustomed to these platforms, I did a deeper research by looking at the source code of existing Flutter desktop applications (FVM GUI) and discovered a google package that does it all for you 🔥🔥🔥 (flutter desktop embedding). Here is a simple example on how to use limit the window size of a desktop app:

After applying the above code example, it was all fixed!!! Whew!!!!..
And now, I have a functional and beautiful page.

And that’s all for now. I will share the progress of the desktop application I’m developing in due time ⏰. And if you need any clarifications on this topic, do well to reach out to me on twitter @_iamEtornam.

Akpẽ Kaka (Thank you in EʋE)

originally posted on medium.com

UI Challenge 2020 (part 1)

Bright Etornam Sunu — Thu, 02 Jan 2020 06:50:49 +0000

As 2020 starts, i'm trying to sharpen my UI skills in flutter and also digging deep into Animation and Native Dart features. I started the year by implementing an Awesome mockup i saw on dribble.com by Mickael Guillaume. The first Page implemented and second page to go!
source code here: https://github.com/RegNex/InstaPic

Don't forget to give it a 🌟
🎊Happy New Year 🎉