DEV Community

Zain Ul Abideen Rizvi
Zain Ul Abideen Rizvi

Posted on Edited on

How I measured latency in my AI smart glasses

When you ask a wearable a question, the wait starts as soon as you finish speaking. A fast model helps, but the user still has to wait for speech detection, transcription, camera capture, and audio playback. I built a voice-and-vision prototype with a Raspberry Pi, a USB camera, Whisper, Groq, and text-to-speech. Here’s how I broke that wait into stages and found where to spend time optimizing.

Measure the whole interaction

My original notes break the pipeline into stages. To measure perceived latency in your own build, start a timer when the speaker finishes, record timestamps after transcription and frame capture, and stop the timer when the first audio plays. If you only time the model call, you miss delays like the silence threshold at the end of a sentence and the time it takes to start playback.

Stage What to measure
Speech detection Time from the last spoken word to the end-of-speech event
Transcription Time from handing off the audio to receiving text
Camera Time to capture and encode one frame
Vision model Time from sending the question and frame to receiving the answer
Speech output Time from sending the answer to the first audible output

For your own tests, include both a cold run and warm runs. Try a short question about a price tag, a longer question about a sign, and a noisy-room test. Keep the inputs consistent when comparing two changes so a different question or network condition doesn’t hide the effect.

Capture only what the question needs

I used voice activity detection to decide when to run speech-to-text and grabbed one camera frame when the question ended. That let the model answer questions about the current view without a continuous video feed. I also asked for a one-sentence answer. On a wearable, that makes the response easier to listen to and gives text-to-speech less text to process.

My original build notes report end-to-end responses around one to two seconds in that prototype. Treat that range as a reference for this build. For your own measurements, record the hardware, connection, question type, and model alongside the timings. Once you have enough runs, compare the median and the slowest runs separately. A system that usually feels quick can still frustrate users if the first answer is consistently slow.

The original smart-glasses build notes include the hardware and software stack. For more of my voice, AI agent, and automation projects, see my engineering portfolio.

Top comments (0)