I reverse-engineered a webcam so Claude could point it at things

The OBSBOT Tiny 2 is a webcam bolted to a motorized gimbal. It pans, it tilts, it zooms, and it has an on-device AI that follows you around the room. It's a nice piece of hardware and OBSBOT ships a perfectly good GUI for it.

What I wanted was different. I wanted an MCP server, so a model could take a snapshot, look at the picture, find the thing it cared about, and then point the camera at it. Close the loop: see, aim, see again.

That needs a protocol. There isn't a published one.

Ghidra, headless

The vendor app knows how to talk to the camera, so the protocol is in the vendor app. I ran Ghidra headless over the binaries and went looking for whatever assembles the USB control transfers.

It's a class called FrmPacketV3, and once you find it the frame is almost tidy:

off 0    : 0xAA              magic
off 1    : 0x25              flags (bit 0x60 => there's a nested segment)
off 2-3  : seq    u16 LE
off 4-5  : len    u16 LE     bytes covered by the checksum (12)
off 6-7  : token  u16 LE     the checksum itself
off 8    : sender   0x0A
off 9    : receiver 0x04
off 10-11: cmd    u16 LE     the command id
off 12+  : payload           float32/int32, little-endian

Sixty bytes, zero-padded, handed to a UVC Extension Unit as a SET_CUR on selector 2. The token is CRC-16/USB — poly 0xA001, init 0xFFFF, reflected, xorout 0xFFFF — computed over the first twelve bytes with the token field itself zeroed:

def token(data: bytes) -> int:
    crc = 0xFFFF
    for b in data:
        crc ^= b
        for _ in range(8):
            crc = (crc >> 1) ^ 0xA001 if crc & 1 else crc >> 1
    return crc ^ 0xFFFF

Gimbal moves don't fit in twelve bytes, so they go in a nested segment starting at offset 12 with its own length and its own CRC over its own bytes. Roll, pitch, yaw as three float32s. A move-to-angle is cmd = 0x6444, a speed move is 0x6484, wake and sleep are both 0xA0C2 with an inverted flag byte (wake is 0, sleep is 1, which I got backwards the first time).

There turned out to be four separate ways into the device, and the vendor app uses all of them depending on what it's doing: framed V3 packets on XU selector 2, flat byte reads on XU selector 6, and then plain IAMCameraControl and IAMVideoProcAmp for the boring stuff. Knowing which channel a given feature lives on is most of the work.

The bug that taught me not to trust my own opcode table

Track speed didn't work. I'd send the command, the camera would acknowledge nothing in particular, and the tracking speed would stay exactly where it was. For weeks I assumed I had the payload encoding wrong, because that's the kind of thing that's usually wrong.

I had 0x0944. The real opcode is 0x0cc4, AI_SET_TRACK_MODE. I only found it by capturing USB traffic while OBSBOT's own app changed the setting, and diffing that frame against mine.

This is the thing about an undocumented binary protocol that I hadn't internalized: a wrong opcode doesn't fail. There's no error, no stall, no NAK. The firmware looks at a command id it doesn't recognize and drops it on the floor, and from the host side that's indistinguishable from a command that worked and had no visible effect. Every silent no-op I chased after that, I went to the wire first instead of re-reading my decoder.

Static analysis tells you what the code could send. A capture tells you what it does send. Those are different documents.

Then half of it turned out not to be vendor protocol at all

Somewhere in the middle of this I checked whether the gimbal answers CT_PANTILT_ABSOLUTE — control selector 0x0D, a bog-standard UVC Camera Terminal control that every OS already knows how to talk to.

It does. And it's not echoing back what I told it. I polled it every 40 ms while a 0° to 90° pan was in flight:

t+0ms     pan=0°
t+709ms   pan=3°
t+832ms   pan=10°
t+1913ms  pan=51°
t+3259ms  pan=90°   <- physical arrival
t+3300ms  pan=90°   (steady)

That's an encoder tracking a physical slew. Tilt stayed at zero the whole time, having never been commanded. It even reports motion the host never commanded through that control at all — vendor speed moves, recenters, the AI panning itself around after a face.

That reframed the project. My end goal was cross-platform, and vendor XU access means writing and maintaining a native helper for every OS — DirectShow on Windows, IOKit on macOS, V4L2 and libusb on Linux — and then keeping three implementations of the same protocol honest against each other. Standard UVC controls cost none of that. The OS already has the code.

So the rule became: use standard UVC wherever it's sufficient, drop to the vendor extension unit only where it isn't. Zoom is UVC. Focus, exposure, white balance are UVC. Pan and tilt are UVC in both directions, read and write. What's genuinely vendor-only is the AI tracking, the presets, sleep/wake, and speed-mode gimbal moves. That's maybe a third of the surface, and everything else stopped being three implementations.

And it worked everywhere except Linux

Same control, same camera, same USB transfers. On Linux, pan_absolute reads back whatever value I last wrote, forever, no matter where the gimbal actually is.

That's uvcvideo caching. __uvc_ctrl_load_cur() short-circuits on a ctrl->loaded flag, and that flag gets cleared in exactly two places: the driver's own commit path, and on receipt of a Control Change interrupt from the device. The Tiny 2 never sends one. I dumped GET_INFO across five Camera Terminal controls and got a constant 0x03 for every single one — the firmware isn't computing the response per-control, it's returning a stub. Bits D3 (Autoupdate) and D4 (Asynchronous) are clear on a gimbal that takes seconds to complete a move and repositions itself autonomously as a headline feature. Both should be set. That's a real spec violation, and I've now reported it to OBSBOT.

But "the vendor is non-compliant, work around it" is a weak argument to bring to a kernel mailing list, and it's also not what's actually going on. Trace the cache on a fully compliant PTZ camera, one that does set D3 and does send its interrupts:

1. S_CTRL(pan=90) — commit clears loaded
2. first G_CTRL — live GET_CUR, sets loaded = 1
3. every read after that — cache hit, same value, until the move ends and the interrupt finally arrives

One live sample per write. And if userspace reads promptly after commanding the move, which is the natural thing to do, that one sample is taken before the actuator has appreciably moved. It's the least useful sample available. Polling is useless on any UVC PTZ camera, compliant or not.

The spec more or less concedes this. §4.2.2.1.15: a relative-control write "shall result in a Control Change interrupt for the Absolute Control at the end of the movement." The notification mechanism is event-shaped, and it's specified for the end of the move and nothing during it. That's fine for "auto-exposure changed mode." It's the wrong shape for a quantity that varies continuously for three seconds. V4L2_CTRL_FLAG_VOLATILE already exists for exactly this concept and uvcvideo just never applies it here.

So: patch.

The first patch was wrong in an interesting way

Four lines. Add a UVC_CTRL_FLAG_VOLATILE bit, set it on the pan/tilt entry, bypass the cache, surface the flag to userspace. Clean under checkpatch.pl --strict. I never sent it, and I'm glad.

One UVC control holds both axes — PANTILT_ABSOLUTE is 8 bytes, and PAN and TILT are 32-bit mappings at offsets 0 and 32. Because the mapping doesn't span the whole control, uvc_ctrl_set() does a read-modify-write on every single pan or tilt write. Make that read live and:

1. S_CTRL(pan=90) commits, gimbal starts moving
2. S_CTRL(tilt=20) a millisecond later re-reads live position. The gimbal hasn't physically moved yet, so it reads pan=0
3. commits SET_CUR(pan=0, tilt=20) — the pan command is cancelled

The write path only works today because the cache holds the commanded setpoint rather than reality. My own server issues pan and tilt as two parallel single-axis writes, so I'd have shipped a patch that broke the tool I'd hardware-verified, the moment it merged.

The commit message had a second problem: I'd quoted the UVC spec asserting the control "indicates the pan/tilt actuator's current position." Correct section number. The quote does not exist. I'd reconstructed it from memory and presented it as verbatim, and the real text uses setpoint language throughout. The citation that actually supports the argument was two subsections away the whole time.

The version I sent inverts the fix. Instead of adding a setpoint shadow so the write path can keep working — which means modifying the write path, which is exactly where the regression lived — it adds a separate read buffer. uvc_ctrl_set() comes out byte-for-byte identical to mainline. The regression isn't avoided, it's impossible by construction, and that's a much better thing to be able to tell a reviewer.

55 insertions, 4 deletions, two files. Hardware results on a self-built 7.2.0-rc4: the two-axis write passes with the hazard window genuinely exercised (pan still read 0 at the second write, and it still landed); live position steps 0→5→17→24→36→47→55→66→78→90 during a slew; 100 frames at 30.5 fps while polling position at 5 Hz with no frame loss; v4l2-compliance identical patched versus pristine.

It went to linux-media on July 25th, and Ricardo Ribalda picked it up six days later, opening with "I am not against the idea proposed by this patch" — then three questions and a counter-proposal. He suggested V4L2 control events instead, which fails three ways here: uvcvideo's only device-originated event is fed by the Control Change interrupt, the spec promises that interrupt only at the end of a move, and this camera never sends it at all. He also found a real hazard on the write side — once the AI has tracked the gimbal away from the last commanded pose, a single-axis write merges from the setpoint and yanks the other axis back. Real, but pre-existing: the patch leaves the write path byte-identical to mainline.

He came back the same evening, added Hans Verkuil to the thread, and said he was curious what HansV thought. So it's waiting on a second maintainer now, not on silence.

Two of his points I conceded outright. I'd overstated the accuracy requirement — nothing here correlates a position with a particular frame, and I'd answered a question as though it did. And he pointed out that VIDIOC_S_EXT_CTRLS sets both axes in a single commit, which sidesteps the write-path hazard entirely from userspace. I'd missed it completely. That one became a shipped fix in my own server the next day, which is the better outcome anyway: the patch is about the read side, and the write side had a userspace answer the whole time.

There's a nice accident buried in the retest, too. One run failed with the exact v1 signature — pan cancelled back to zero — and the cause wasn't my patch at all. The module reload had re-probed the camera while it was asleep, and asleep it doesn't answer GET_INFO, so the driver couldn't override the static table and pan/tilt kept AUTO_UPDATE. With that flag set, mainline clears loaded after every commit and does the live read-modify-write by itself. It's a stock-kernel bug on this camera. It's also not going in this patch — it's orthogonal, and bundling a second bug story into a commit message whose only real virtue is narrowness is how you lose a reviewer.

Where it landed

obsbot-mcp is on npm and does the thing I wanted: snapshot, find something in the frame, aim at that pixel, snapshot again to confirm. The aiming math reads the camera's own magnification out of its status block, so it works at any zoom without being told what the zoom is, and it refuses rather than guessing when it can't — AI tracking is fighting it for the gimbal, the zoom is still ramping, the camera had to be woken (waking moves the gimbal, so the frame you measured is stale).

35 tools on Windows and macOS, 34 on Linux, where speed-mode gimbal moves are hidden — a speed times duration burst can't be bounded without live position feedback, and until that patch lands, Linux doesn't have any.