This is a submission for the MLH x DEV Writing Challenge
What We Built
Moda is a mode system for AI glasses. Most smart glasses today wait for a command: you ask, they answer. We wanted to flip that. Moda lets the glasses figure out what kind of moment you’re in, such as working, cooking or meeting, and then automatically change how the assistant listens, sees, speaks, and stays quiet.
Each mode has its own context, rules, and intervention policy. We built three example modes:
Work Mode starts when you settle into a task. It creates a focus block and can enable calm music and Do Not Disturb. It uses on-device vision plus the glasses’ IMU signals to detect sustained distraction, close viewing, or poor posture. It ignores single glances and small movements. Prompts are rate-limited and fire only on persistent signals. When the block ends, Moda asks whether you want a break or more time, and it remembers your answer.
Cooking Mode is a hands-free kitchen companion. Open the fridge and it can suggest recipes from the ingredients it sees. It guides you through steps and timers by voice. It can also flag things like a pan handle sticking out or a stove left on when you walk away.
Meeting Mode recognizes a meeting context and offers to start, delay, or skip assistance. It always asks for recording consent first, stays silent during the conversation, and afterwards saves a short local summary of decisions and action items.
The design principle we kept coming back to was proactive help without proactive noise. A state machine handles mode transitions. Lightweight preferences like “give me 10 more minutes” are remembered. When no intervention is needed, we count silence as a successful outcome.
Privacy is part of the architecture. Visual detection runs on the phone by default, cloud processing is opt-in, meeting recording requires consent, and settings and session data stay local.
What I learned: [your own takeaway. For example: designing for restraint is harder than designing for features, because deciding when not to speak takes as much work as deciding what to say.]
Demo
🎥 Video: https://youtu.be/-lnlBRSjZvY
🖥️ Interactive demo: https://htmlpreview.github.io/?https://github.com/xingzhuonan-blip/moda/blob/main/demo/moda-project-demo.html
💻 Source code: https://github.com/junctionmoda/moda
📑 Presentation: https://files.hackjunction.app/presentations/45/aa94de73-7243-4e1d-9197-3c0f578d0f59.pdf
Partner Technologies
Meta Wearables Device Access Toolkit (DAT 1.0.1)
Moda is an Android companion app for Ray-Ban Meta Gen 2 glasses, built on the Meta Wearables DAT. It gives us the glasses’ camera stream and IMU signals, which feed the mode detection and the Work Mode posture and distraction checks.
Gemini Live
Where voice makes sense, Moda uses Gemini Live for natural, hands-free conversation, for example when guiding a recipe step by step. Because glasses need to keep working when the network doesn’t, we also built offline fixed-phrase voice controls as a reliable fallback.
On-device vision + optional cloud AI
Local vision processing runs on the phone by default, so camera data doesn’t leave the device unless the user opts in.
Hackathon Experience
I took part in Junction 2026 in Espoo, Finland, on the “AI Glasses for Everyone, Everyday” challenge, together with my team HOTGOD.
It was a lot of fun. Working with a brand-new device was the highlight. Building for Ray-Ban Meta glasses is very different from building a regular app, because you have to think about what someone actually wants while they’re cooking, working or sitting in a meeting. The venue was big and well equipped, with plenty of space and facilities to work in. The atmosphere was great throughout. Next time, I’m hoping to bring home a prize. 🏆
Top comments (0)