The Problem That Wouldn't Leave Us Alone
There are 285 million people living with visual impairment worldwide. That's not a statistic we stumbled on in a paper. That's roughly the entire population of Pakistan — the country we live in — walking through the world with a tool designed in the 1800s as their primary navigation aid.
The white cane is genuinely brilliant engineering for what it does. But it has a fundamental physical constraint: it scans the ground. Obstacles at chest height, head height, or anywhere in the mid-air zone between? Invisible.
Guide dogs are extraordinary. They're also expensive to train, unavailable in most of the developing world, and not everyone can care for an animal.
We're four CS students at Sukkur IBA University. We don't have research grants or hardware labs. What we had was the question: what if a piece of clothing could give someone a spatial sense they were born without?
That question turned into IRIS — Interactive Responsive Inclusive System.
The Idea: Make the Jacket Feel
The core insight was deceptively simple.
Skin is an incredible input device. We use tactile sensation constantly — the gentle pressure of a doorframe, the change in floor texture, the warmth of sunlight from a window. We know from research on sensory substitution (shoutout to Paul Bach-y-Rita's work) that the brain is remarkably good at reinterpreting tactile input as spatial awareness, given enough meaningful signal.
So: what if we mapped space onto the body?
A vibration on your left shoulder = something approaching from the left. A strong buzz on your center chest = obstacle directly ahead, close. Light hum everywhere = clear path for meters.
Not just on/off. Proportional. The closer the obstacle, the more intense the vibration. The more off-center it is, the more precisely the corresponding motor responds.
That was the dream. Now we had to build it.
The Hardware: What Goes Inside a Jacket
Let's be honest with you — early conversations about this project involved a lot of very naive assumptions about how simple this would be.
The core hardware stack ended up being:
The sensing layer — Three HC-SR04 ultrasonic sensors mounted left, center, and right on the jacket. They pulse ultrasound and listen for the echo. Simple, cheap, battle-tested. Range up to 2 meters, which gives roughly a 1–2 second warning window at walking pace.
The brain — An ESP32 microcontroller runs the real-time loop: read sensors, run the math model, drive the motors. It's fast enough, it has built-in WiFi, and it's cheap enough that we didn't cry when we fried one (we fried one).
The feeling — Seven ERM (Eccentric Rotating Mass) vibration motors distributed across the jacket. Left, center, right for the ultrasonic zone. Plus three for the AI vision layer. Plus one dedicated "URGENT" motor for critical alerts.
The eyes — A Raspberry Pi 4 with a camera module runs in parallel, capturing frames every 3 seconds and sending them to Microsoft Azure Computer Vision. The Pi is the brains for the AI layer; the ESP32 handles the real-time haptics.
The first prototype looked like someone had attacked a jacket with a soldering iron and lost. By the third iteration it looked almost intentional.
The Night Everything Was Wrong: A Love Story With Motors
Here's something nobody tells you about haptic feedback projects: ERM vibration motors are little monsters.
We'd wire everything up, send a PWM signal, and... nothing. Motor just sat there. Or it would hum faintly — clearly receiving power — but not spin.
This is static friction. ERM motors need significantly more torque to start rotating than to keep rotating. If your target intensity is, say, PWM 80 out of 255, and the motor's breakaway torque requires PWM 120 to overcome — it just won't start.
We lost a full weekend to this. We thought we had bad motors. We thought our transistor circuit was wrong. We tried different resistor values, different supply voltages, different PWM frequencies.
The fix, when we finally found it, is almost embarrassingly simple:
Motor was OFF + new signal received?
→ Blast full 255 PWM for 50ms (kick it over the static friction hump)
→ Then settle to target PWM (proportional intensity)
A 50 millisecond kick-start. That's it. That's the fix for a weekend of misery.
If you're ever building a haptic system — save yourself the pain. Implement the kick-start from day one.
The Math That Made It Real
This is the part we're most proud of, and the part that took the longest to get right.
The naive approach to directional haptics is simple: if something is on the left, buzz the left motor. On the right, buzz the right. In the middle, buzz the center.
That works. But it produces jarring, binary feedback. On. Off. On. Off. For someone relying on this as a navigation sense, that's tiring and imprecise.
We wanted something that felt continuous. As an obstacle drifted from left to center, the motors should smoothly interpolate — the left motor fading down, the center motor rising up, like a balance needle swinging.
The solution came from thinking geometrically. Each motor has a fixed angular orientation θ in the jacket's coordinate plane:
- Left motor → θ = π (facing 180°)
- Center motor → θ = π/2 (facing forward)
- Right motor → θ = 0° (facing right)
Then, given two distance readings — center distance and side distance — the PWM intensity for any motor at angle θ becomes:
255 × side × center
PWM = 255 - ──────────────────────────────────────
√( center² · cos²θ + side² · sin²θ )
Visually, think of an ellipse defined by the two distances. The motor's angle determines where on that ellipse we sample — giving it natural directional sensitivity. Close in the center? The ellipse is narrow and tall — the center motor maxes out, the side motors stay relatively calm. Close on the left? The ellipse stretches sideways — the left motor fires hard, center fires moderately, right barely registers.
We built an interactive 3D Desmos model to visualize this before we committed to the formula. Watching the motor intensities shift smoothly as we moved a virtual obstacle around the space was genuinely one of the best moments of the project.
| Scenario | Left Motor | Center Motor | Right Motor |
|---|---|---|---|
| Everything 3m away | ~127 (mild) | ~127 (mild) | ~127 (mild) |
| Left obstacle at 0.3m | ~245 (strong) | ~145 (moderate) | ~10 (barely) |
| Center obstacle at 0.3m | ~145 (moderate) | ~245 (strong) | ~145 (moderate) |
| Everything at 0.2m | 255 (max) | 255 (max) | 255 (max) |
This is what made IRIS feel like a sense rather than a burglar alarm.
The AI Layer: Teaching the Jacket to See
The ultrasonic sensors are fast and reliable, but they have no concept of what they're detecting. A wall and a child both return the same echo.
For that, we added an Azure Computer Vision pipeline running on the Raspberry Pi.
Every 3 seconds, the Pi camera captures a 1280×720 JPEG. That image goes to Azure's dual API — one call for scene description ("a person standing near a flight of stairs"), one for object detection with bounding boxes.
Anything below 50% confidence gets discarded. The rest gets classified.
The frame is divided into three horizontal zones (left / center / right), and each detected object's bounding box center determines which motor zone fires. Object size in the frame estimates physical distance — a bounding box covering more than 25% of the frame means it's very close, triggering maximum intensity plus the dedicated URGENT motor.
Some objects earn special treatment:
URGENT_OBSTACLES = ["car", "truck", "person", "stairs",
"hole", "wall", "pole", "dog", ...]
If any of these appear close or center, the jacket interrupts whatever it's doing and fires the urgent motor while the Pi speaks a voice alert via espeak. The voice can describe the full scene — "stairs ahead" — something no ultrasonic sensor could ever tell you.
The Communication Problem Nobody Warned Us About
Two devices. One jacket. They need to talk to each other over WiFi.
Our first instinct was MQTT — it's the standard IoT messaging protocol, it's reliable, it has guaranteed delivery. We spent time setting up a broker, testing it, latency testing it.
Then we noticed something uncomfortable: ~50ms latency between the Pi sending a command and the ESP32 firing a motor. For haptics, that's a lifetime. The user's body is expecting a response now — delayed feedback teaches the nervous system the wrong thing.
We switched to raw UDP sockets. The command format is as minimal as we could make it:
"V,{motor_index},{intensity},{duration_ms}"
Examples:
"V,3,255,500" → AI-Left motor, full power, 500ms
"V,1,100,200" → Center motor, gentle pulse, 200ms
"STOP" → Kill all motors immediately
Latency dropped to ~1ms. Yes, UDP is "fire and forget" — if a packet drops, the motor misses one pulse. But for haptic feedback, that's fine. A missed vibration is imperceptible. A late vibration is actively wrong.
We also scrapped hardcoded IP addresses (which would break on any new network) and had the ESP32 advertise itself via mDNS as esp32.local. The Pi finds it automatically on startup. Plug in anywhere, it just works.
Submitting to Microsoft Imagine Cup
We submitted IRIS to Microsoft Imagine Cup 2026 in the Accessibility category.
Writing the submission forced us to articulate something we'd been feeling but not quite saying: this project isn't about the technology. The ESP32, the Azure API, the trigonometric model — those are just means. The thing we actually built is a small reduction in the margin between what sighted people take for granted and what everyone else has to work around.
285 million people. The cane has been the primary tool for most of them for over a century. We're four students from Sukkur, Pakistan writing a few hundred lines of C++ and Python and we built something that works today, with components anyone can order, that gives directional spatial awareness no cane has ever given.
That's not nothing.
Try It, Break It, Improve It
Everything is open source under MIT — hardware, firmware, Python scripts, the math model.
The full architecture, pin mappings, circuit diagrams, setup instructions — it's all there. If you build a version, change the motor layout, swap Azure for a different vision API, add a GSM module so it works offline — we want to know. Open a PR. File an issue. Send a message.
There are 285 million people in the world this could matter to.
That's a lot of reasons to keep building.
Built by Shaikh Muhammad, Shahzeb, Khizar, and Shahrukh at Sukkur IBA University.
This post was written with AI assistance for structure and language.
All technical decisions, debugging, and math are our own.
#showdev #iot #accessibility #opensource
#raspberrypi #esp32 #python #cpp #computervision #hardware #azure #webdev #beginners #ai #programming
Top comments (0)