For the Flip the Switch Hackathon, my teammate and I built Coding Arena, a workflow for reviewing AI-generated code with executable evidence and a human decision.
Two Codex agents receive the same task. They work independently, submit their code, and review each other's frozen submissions. Tests run through our engine, and the human chooses which patch to export—or chooses neither.
We used a booking-system task for the demo: reserve seats, cancel bookings, and never exceed capacity. Keeping the task small helped us focus on the harder part: connecting conversations, code versions, test results, and human decisions into one working process.
Where Switch fits
We connected our native Codex agents to Mattermost using Switch Console on Linux. Switch gave us named agents, coding sessions, and rooms where a human could give instructions and read the replies.
We created separate build rooms for arena-a and arena-b, then used a shared room for cross-review. The agents kept their own working directories and slot-scoped engine tokens.
The dashboard and Switch meet through the Coding Arena engine:
Human → Switch and Mattermost → Codex agents → Python client
↓
Human → Coding Arena dashboard ← Coding Arena engine
↓
Docker verification
When an agent submits code through the Python client, the engine records a candidate and starts a verification job. The dashboard reads that engine state. A passing result appears there because the job finished, rather than because an agent wrote “passed” in chat.
The human directs agent turns in Switch and records formal actions in the dashboard. We deliberately kept those responsibilities clear: room messages carry instructions and discussion; the engine controls verification and selection eligibility.
Setting up the local prototype
Our tested stack used Python 3.12, Node 22.23.4, Docker, Switch Console, and a signed-in Codex CLI. The Switch quickstart explains installing Console and starting a local server. We chose On this computer and used the local Mattermost connection.
The application setup starts from the public repository:
git clone https://github.com/kumar-varun024/CodingArena.git
cd CodingArena
python3 -m venv .venv
.venv/bin/python -m pip install -r engine/requirements.txt
export ARENA_STATE_DIR="$PWD/var/runtime"
We pin the Python runner image so verification uses a known environment:
docker pull python@sha256:4840d514c583a4d3d8cad3940e68dee6f0f6f3635c12fcd0a3d74c21a5b93a54
PYTHONPATH=engine .venv/bin/python -m arena_engine init
PYTHONPATH=engine .venv/bin/python -m arena_engine doctor
PYTHONPATH=engine .venv/bin/python -m arena_engine serve
In a second terminal, from the repository root:
npm --prefix web ci
npm --prefix web run dev
The dashboard opens at http://127.0.0.1:5173, with API requests proxied to the engine on port 8765. Engine initialization creates the operator token and the two contestant workspaces. We enter the operator token locally in the dashboard and keep it out of chat and source control.
The runtime installer prepares the agents' client and instructions. In another terminal from the repository root:
python3 switch/install_runtime.py --state-dir "$PWD/var/runtime" --api-url http://127.0.0.1:8765 --check
python3 switch/install_runtime.py --state-dir "$PWD/var/runtime" --api-url http://127.0.0.1:8765
We then registered both agents in Console, using each installer-reported slot root as its Directory and copying its generated instructions. That root is the stable agent folder; the current assignment identifies the editable task checkout inside it.
The installer prepares files. Agent registration and room invitations still happen in Switch. The repository's setup guide has the full procedure.
Taking a task through the rooms
Our recorded run used arena-demo-build-a, arena-demo-build-b, and arena-demo-review.
We created a booking match in the dashboard and gave each agent the real match ID. Each fetched its assignment, implemented the allowed source file, committed its changes, ran public practice, and submitted its clean commit for official verification.
A submission becomes a frozen candidate. Subsequent review and test results refer to that exact code version, which prevents a result from silently becoming evidence for some later edit.
After both initial jobs finished, the human revealed the submissions. We asked each agent in the shared Switch room to inspect the opponent's frozen code against the written requirements.
An agent can submit a concrete, requirement-linked test challenge. The engine can check that challenge, and the human decides whether to accept it. Accepted challenges become regression checks for later verification. Repairs are bounded to at most two rounds.
An agent can also finish with no findings. That happened in our recorded run; we did not manufacture a bug to demonstrate the challenge feature.
What actually happened in the demo
Both candidates passed their initial official verification with 11 out of 11 checks. Both completed cross-review without submitting challenges. We then ran final verification, and both passed 11 out of 11 again.
The operator selected A and downloaded its patch and evidence pack. Both candidates were eligible; this run did not establish that A was objectively better than B. Passing the recorded suite also does not prove that either implementation is free of untested bugs.
The evidence pack preserves the recorded checks, candidate references, match events, and decision. The selected patch contains the corresponding code changes. That gives another developer something concrete to inspect after the conversation ends.
Separately, our automated verification covered 174 tests across the engine, client, dashboard, fixture service, and runtime installer. Challenge and repair paths were exercised through labelled fixtures; no challenge or repair was recorded in this native demonstration.
The setup problems that taught us the most
Our first obstacle was a model setting inherited from a global configuration. It was not supported by the account used by our Codex CLI. Choosing a supported setting for the registered agents resolved it.
The more confusing problem looked like an engine outage. An agent's assignment request failed, while the same installed client worked from a terminal. We checked the session logs and found that the Codex execution sandbox had network access disabled. After the normal approval flow allowed the local client command, the request succeeded. We encountered the permission boundary again in a new review session.
That distinction matters when debugging: a healthy local service can still be unreachable from an agent's execution environment. We would like Console to make these permission blockers more visible alongside agent status.
Inviting an existing agent into another room also took some discovery. Clearer invitation controls would help first-time users get from two registered agents to a shared workflow faster.
What we would build next
The current MVP is local, supports one preset Python task, and uses two agent slots. Separate rooms organize independent work; they are not a security boundary between hostile agents sharing an operating-system account.
Next, we would add more task packs and make phase instructions easier to send from the operator interface, while keeping reveal, challenge decisions, and final selection explicit human actions.
The useful outcome for us was a complete conversation-to-patch workflow: agents work through Switch, verification produces recorded evidence, and the human remains responsible for the choice.
Top comments (0)