Switch Bench

Putting a firmware bench in a Discord room with Switch

We built a workflow where a Claude Code agent fixes a firmware bug, a Codex agent tests the fix on a real ESP32, and a human approves the merge. All three work in one Discord channel, connected with Switch. This post covers what it does, how to set it up, and what went wrong along the way.

Repo: https://github.com/skanda137/HackDevs · Demo video: https://youtu.be/Y0VsXAqXQDA

What it does

The board sits on a lab laptop. The person who can fix a bug is on another laptop. Normally someone has to be at the bench to flash the board and read the serial output.

In our room, a person types @lab-bench run main on the board. The bench agent (Codex) runs a script that flashes the ESP32 and prints a JSON verdict. The baseline is 2 tests passed and 2 failed. The person then asks @lab-dev to find the cause. The dev agent (Claude Code) reads that raw output and the source, states one theory, pushes a fix to a new branch, and asks the bench agent to run it. The bench flashes the board again and posts the result. The dev agent merges only after a person replies approve merge.

The bench agent never edits files. The dev agent never touches hardware. Each only sees what the other posts in the room.

Why a room and not a script

Claude Code and Codex are separate tools with no shared channel. A script that calls both would need its own identity, handoff and approval logic. Switch gives that to us as a room. Each agent has a name, @lab-dev and @lab-bench reach the right one, both read the same history, and a person can step in from Discord on any machine.

Two details of Switch shaped the design. Agent-to-agent handoff is off by default, so we set both agents to accept messages from anyone in a private channel. And Switch's built-in permission prompts cover risky agent actions, while our merge rule lives in the dev agent's instructions. They are two separate gates.

Setup checklist for a first-timer

These are the steps that cost us time. The docs cover each one.

Create the Discord bot. On the Bot tab, enable Server Members Intent and Message Content Intent, then save. Without Message Content, agents receive messages with no text.
Invite it with two scopes, bot and applications.commands. If you forget the second, slash commands are missing, and you have to rebuild the invite link.
Give it the right permissions: View Channels, Send Messages, Manage Webhooks, Manage Channels, Manage Roles, Read Message History, Attach Files, Add Reactions. Webhooks let agents post under their own names.
Connect it in Switch Console with the bot token and the server ID (Discord calls it the Guild Id), then link your own Discord account.
Create the agents, set addressing, and give each its instructions.
Create the room. Posting a message in a channel creates it. Inviting the bot does not. Add agents with !invite-agent @name.
Restart Console after installing tools. It reads PATH once, when it starts.
What went wrong

Our first rehearsal cheated. The README described the bug, so the dev agent had the answer. When we removed it, the fix was still in the git history and in the chat history. We squashed the repo to one commit, made a new room, and created fresh agents.

The first end-to-end run used the emulator. The bench agent's run command took its target from a default (--target host), so it tested the host build and not the board. The "target" field in the JSON verdict is what exposed it, which is why we print it.

The wrong Codex model. The agent's first turn failed with model_not_found because gpt-6-sol is not supported with a ChatGPT login. Pinning gpt-5.6-sol fixed it.

Windows and PlatformIO. PlatformIO installs as pio.cmd on Windows, so our harness could not find it until we resolved it with shutil.which("pio").

The template did not provision on Windows. We never found out why, and set up the agents and room by hand. The template file is in the repo, but we have not verified it end to end.

What happened in the recorded run

The baseline run on main came back from the board as 2 passed, 2 failed, with "target": "esp32" and exit code 1. The failing window average read 1952 when the input was sustained at 4000.

I then gave @lab-dev only that output and the source. It created a new branch, fix/sensor-sum-overflow, and pushed a one-line change to commit fc16378: the running sum in sensor_t went from uint16_t to uint32_t. A 32-sample window of 4000 sums to 128000, which wraps around in 16 bits to 62464, and 62464 / 32 is the 1952 we saw. The bench agent flashed that branch and posted 4 passed, 0 failed, "target": "esp32", exit code 0.

[MERGE: after I typed approve merge, the dev agent merged the branch into main]

One thing did not go smoothly: the bench agent's shell had no python or pio on PATH, so it had to request escalated host permissions, and we approved a prompt on the bench laptop during the run. Do not assume this is hands-off.

Limits

The bug is planted. The board runs a self-test of the firmware logic, with no wired sensors. Both agents run on one laptop and only the human is remote. We tested one board on Windows. Anyone in the private channel can instruct the agents, so keep it private.

Try it

The bench runs without a board: python bench.py test --target host on the tag demo-baseline-esp32 needs only Python and gcc. The full workflow needs Switch Console, a Discord server and both agent CLIs. The README has the steps.