The idea started with a conversation about math. A lot of recent progress in AI-assisted math comes from verifiers: a model proposes a proof, and a proof assistant like Lean checks every step. If the proof is right, it passes; if it's wrong, it fails, and nobody has to argue about it. People doing combinatorics and complexity research have been doing something similar: a model proposes a construction, then a short program scores it when you run it through thousands of tests. The model doesn't have to be right, it just has to be checkable.
Then one of us asked, "What's the equivalent for physical hardware? And we couldn't think of one. When you build a prototype today, you put it in front of a few people, someone writes down "button 2 feels sticky", a week later the designer guesses a fix, and you reprint. Then you test it all again by hand and hope it's better. There's no fast loop, no score, nothing for an AI designer to learn from.
The gap seemed worth a weekend.
We wanted a robot that tests a real product, writes up what's broken with evidence, suggests a fix in CAD, and tests the fix to see whether it actually helped. We called it Robot-As-A-Verifier, or RAAV.
Why medical devices?
We kept coming back to medical hardware, because that's where a bad button stops being an annoyance. Infusion pumps, insulin pens, and lab equipment get used by tired nurses, older patients, and people with arthritis. If a button takes too much force to push, or a latch needs a pinch to open, some of those people can't use it. Medical devices already go through usability testing, but it's slow, manual, and hard to repeat with every revision.
So instead of a toy, we built two mock lab appliances to test: a centrifuge we called the MC-6, and a microwave called the MW-1. Each has real buttons, a switch, a knob, a lid or door, and a little display. An Arduino inside reports what the appliance actually did.
The first plan didn't survive
Our robot is an SO-101, a cheap hobby arm with its stock gripper and a $19 USB camera at the wrist. Joseph built a MuJoCo simulation of it, so we could develop before touching the hardware. The sim taught us more than we expected — mostly by telling us how wrong we were.
The first plan used a close-range depth camera to measure the appliance. In the sim, it looked perfect. When we modelled what the camera actually is — a stereo pair with no projector — it fell apart on the plain plastic. It found 28 of 66 parts, and missed every door and display. So we dropped depth entirely.
Now the robot finds parts in 2D with OWLv2 and SAM, and works out how tall they are by comparing photos taken from a little further away.
Force was the next surprise. The arm has no force sensor, just the servos' own estimate of how hard they're working. Our "skills" — the code that moves the arm — were written as if we had a perfect sensor. On the servo estimate, they passed 0 of 89 steps. Gear friction hides small pushes completely.
After a lot of tuning — zeroing the estimate right before contact, keeping thresholds above the noise — we got to 84 of 89.
Keeping the AI away from the motors
Early on, we decided that the language model would never control the arm directly. A model takes a few seconds to answer, and the arm's control loop runs 50 times a second. We also just didn't want the prompt to be able to move a joint.
So we split the project into two systems. System 1 — the robot itself, vision, motion, safety limits, and rules to decide whether a test passed — never calls an LLM. System 2 does everything else: reads the manual, plans the tests, reacts to failures, takes instructions from a person through our web Console, and writes the reports. The two talk through seven MCP tools.
System 2 can say "press button 1", and System 1 decides how to do it, clamps every number it's given, and reports what the sensors saw. We started saying "the model thinks, the code moves, the sensors decide", and it stuck.
For System 2's model calls, we used Snowflake Cortex. Before that, we'd tried free API keys from different providers, and kept hitting quota limits and overload errors. 85 seconds for a response. Cortex gave us Claude Haiku, Claude Sonnet, GPT-5.4, and Grok through a single endpoint, so we could give all four the same prompts and tools and actually compare them.
The first round was humbling in a useful way. All four scored perfectly — so the test was too easy. We added randomly planted faults, with a small chance of something going wrong on every step, and the comparison started to mean something. We also logged every call and its cost in Snowflake credits. The whole first round came to half a credit.
The hotspot problem
The appliances matter as much as the camera, because they're how the robot knows whether a test passed. If the robot presses "START" on the centrifuge, the Arduino reports whether it actually started.
At the venue, the arm, its laptop, and both Arduinos were on a phone hotspot. A hotspot won't accept incoming connections. Max and Bayan worked around it with a Vultr server. The Arduinos post their readings over HTTPS to Vultr, and the robot's laptop opens one outgoing WebSocket to the same server. Everything else travels down that tunnel: the appliance readings, and the tool calls from System 2, which also runs on Vultr.
If a reading is more than two seconds old, the robot won't judge a step on it.
It took a few tries. The Arduino would only trust a proper hostname with a real certificate. Its Wi-Fi calls blocked the code that reads the buttons. Quick button presses could start and end between two updates, so the robot never saw them. We ended up queueing every press on the board, so none got lost. By early morning, both appliances were reporting through Vultr, and we could watch every press arrive at the server.
5 a.m. with the real arm
Simulation only gets you so far. Most of Saturday night was Joseph moving the real arm a few millimetres at a time: how far the camera should sit above the panel (140 mm? 155? 170?), how many photos to take, how far past a button's cap to push, how long to wait for the wrist camera to stop shaking before trusting a frame. The commit log from that night is mostly one-line changes like "keep the real scan's camera 160 mm above the panel".
The run that made it click
This is the part we're proudest of: a run in simulation on the centrifuge.
The robot dropped a tube into one of the wells, then pressed "START". The centrifuge refused, and showed "TUBE". The tube had stopped 8.15 mm above where it should sit, so the sensor under the well never saw it.
The finding went to our CAD agent, which only gets what the robot logged. It looked at the part's geometry, and found the cause in a couple of minutes: the hole was 0.4 mm narrower than the tube, so the tube's tapered end wedged on the rim. It wrote a new part in CadQuery with a wider hole, and a chamfer to guide the tube in. Our automatic checks confirmed that the part still fit, and didn't collide with anything else. That took 14 tool calls, 102 seconds, and $0.43.
Then the robot ran the same test on the patched centrifuge. The tube seated within a millimetre, and the centrifuge started.
The agent never saw the fix our own designer made for that part, and it landed within 0.2 mm of it.
That's the loop we'd been describing since Friday: test, find, fix, retest.
What's next
The obvious next step is doing that loop with real plastic: print the agent's fix, put it in the physical centrifuge, and watch the score go up on the real arm. After that, we want the robot to read displays and lights with its camera, so it can test products nobody has wired up.
In the longer term, we'd like RAAV to be the sort of score that an AI hardware designer learns from, the way proof checkers are for math.
We don't think a hobby arm with a $19 camera is the final version of this. But it's enough to show that hardware can have a verifier too.
Top comments (0)