Racoonlabs: we gave our student research group a lab in Slack, built on Switch

RaccoonLabs puts six AI lab members into the Slack a student research group already uses: a postdoc that argues with you, a lab manager, a librarian, an engineer, a scribe and a reviewer. They run on Claude Code, Codex and OpenCode, and Switch puts them in the same four rooms as the students. Nothing runs without a student's approval, and every number in the paper traces back to a logged run. Over three days of rehearsals, the lab ran 1,650 training runs and wrote a 9-page paper. Its reviewer rejected two drafts. One was rejected over a single word.

• What happened in rehearsal: 55 conditions × 30 seeds = 1,650 training runs, a 9-page paper with 16 references, and two drafts rejected by the Reviewer. One said "because" where the data only supported "consistent with". All 55 table rows matched the Reviewer's own recomputation.
• Why Switch: without it, this is three terminals and a lot of copy-paste. With it, a Claude postdoc, a Codex reviewer and an OpenCode librarian share one room, one history and one set of documents, and the professor sees only the room they need.
• What we won't claim: we role-play the students and the professor. The rehearsals below ran on one laptop; the recorded demo splits the agents across two laptops and different AI accounts. The section on what it still gets wrong is long, on purpose.

The problem

Here's how a student research project usually goes. Your professor agrees to the idea in a 15-minute meeting and asks to see a draft in six weeks. Everything in between is yours: the reading list, the experiment code, the seeds, the plots, the LaTeX, the BibTeX that won't compile, and keeping track of who in the group is doing what.

Our research team hit this ourselves: LaTeX, figures, citations, tracking every paper and note, and splitting work across people. We all used AI for parts of it. But my Claude couldn't see what your ChatGPT had done, so we pasted things between windows. And nobody checked any of it properly until our professor did, which is the most expensive possible moment to find a mistake.

So the gap isn't really "write my paper". There are good autonomous research systems already: Sakana's AI Scientist v2, Agent Laboratory, freephdlabor and others. They're pipelines. You give them a topic and they hand back a paper. That isn't what a student group needs, or what a professor will accept. The group needs a lab: people who push back on a bad idea, someone who keeps the task list, someone who runs the experiments carefully, someone who writes things up, and a reviewer who's mean early so the professor doesn't have to be mean late.

What we built

RaccoonLabs is that lab, living in Slack. Students bring a question to #lab-brainstorm. They argue with a postdoc there, approve a plan, and send it to their professor, who is only in #lab-review. A lab manager splits the plan into tasks in #lab-bench. An engineer writes the experiment, then stops and asks before running anything. The results go to a reviewer on a different model family, which checks them against the raw logs. A scribe drafts the paper in #lab-paper and posts the PDF into the channel. The students write the sections they choose. Before the professor sees anything, the draft passes an integrity check.

We wrote three rules on day one and never broke them:

1. No approval, no run. An agent never spends compute on its own say-so, or on another agent's.
2. Every number comes from a log. A result in the paper has to trace back to a batch, a manifest and the person who approved it.
3. We're the authors; they're our lab. Students choose the sections they write. The agents do the legwork.

Why Switch, and not one agent with subagents

We asked ourselves this first, because a judge would.

A single Claude Code session with subagents can play six roles. But they're all the same model with the same blind spots, in one terminal that only one person can see. Here's what we needed that only Switch gave us:

Need	Without Switch	With Switch
A reviewer that isn't the same model as the author	Run Codex in a second terminal, copy drafts across	The Reviewer on Codex sits in #lab-review and reads the same thread
Several people in the loop	Screen-share, or paste into a group chat	Students and agents in the same Slack channel
A professor who sees only what they should	Forward them selected messages	The professor is a member of one room only
Approvals from wherever you are	Be at the laptop	Reply in Slack, on your phone
Shared lab memory	A folder someone forgets to update	Room documents: Tasks, Library, Notebook
Hand-offs	"Can you paste me what Scout found?"	@quill in the room. Switch delivers it as an addressed message

Each Slack channel maps to a Switch room, stored with its full history. Each agent shows up with its own name and avatar. An agent only acts when it's addressed, through an @-mention or a targeted message. That sounds like a limitation, and it turned out to be the most useful property of the whole system. More on that in the gotchas.

Students are in all four rooms. The professor is in one. Each agent is only where its job is.

A note on machines and accounts. Students don't have unlimited Claude, so we wanted each person to bring their own agent and subscription, with Switch joining them into one lab. Our first plan, every laptop reaching one Console server over Tailscale, couldn't work as written: a server that Console runs is reachable only from its own computer, and only its owner can register agents. So we built and rehearsed the whole lab on one laptop with 7.5 GB of RAM. For the recorded demo we split it: the Reviewer and Scout run on one laptop, and the other four agents run on a second laptop, under different AI accounts. One shared server that every teammate's agents can join is still on our wish list.

The research before the code

We spent the days before the hackathon on research, not code. Three questions shaped the build.

1. What will everyone else build?

We went through Switch's official examples and the hackathon's starter posts and sketched the likely field. Bug-triage pairs, dependency upgrades, on-call incidents, standup bots, CV screeners, and a lot of "AI company in Slack" with CEO, CTO and developer agents talking to each other. That last group was our closest overlap. So we wrote down a rule for the build and the video: humans appear first in every scene. If it ever looked like eight bots chatting, we'd lost.

2. How do you prove the lab isn't just making things up?

An AI lab that writes a paper invites one obvious reply: "you're larping". You knew the answer, and you steered it there.

Our answer is a sealed envelope. Before the final run, we picked a real, recently published paper that asks a question the lab could research from scratch. We encrypted it with a passphrase that only the students know. Its title, arXiv ID and repo go into a blocklist that the librarian's search tool enforces, and the list is stored as hashes so the repo never names the paper. The lab researches the question openly. At the end, a student posts the passphrase, and the postdoc scores our findings against the paper, claim by claim.

The envelope is a demo device, not a product feature, and we say so in the video. It's there so you can check we didn't steer.

3. Which paper?

The envelope only works if the lab can actually do the research on our hardware during a hackathon. Our first choice was an ICML 2026 paper about where a well-known transformer attention pattern comes from. It was visual, cheap at about 20M parameters, and it split neatly into three parallel experiments. That suited three engineer agents.

Before committing, we ran a spike on a Kaggle T4. The paper's own model code was shrunk to 6 layers and 19.2M parameters and trained on 34M tokens of OpenWebText for 4,000 iterations. We wrote the go/no-go rule before running it: the effect has to reach at least 3× uniform attention in at least half the layers.

It never got above 1×.

Left: attention on the first token at 4,000 iterations sat between 0.10× and 0.80× of uniform in every layer, and fell during training. Right: the same metric on GPT-2 reaches up to 116×, so the metric wasn't the problem.

The obvious objection is "maybe your metric is broken". So we checked it. Our reconstructed attention matched the model's own to within 6e-8. The planned control, Llama-2-7B, ran out of memory on the T4. We substituted GPT-2, which showed the effect in 11 of 12 layers, and recorded the substitution as such. The verdict was NO-GO at our scale. At one seed and 10% of the paper's iterations, that says nothing bad about the paper. It says a lot about what a hackathon laptop can reproduce.

So we switched to our backup: a smaller paper whose experiments run on one CPU core in 51–108 seconds a seed. We can't name it until the unseal. Our check reproduced its main result. One of our own pre-set clauses failed, and we kept that failure in the record instead of quietly redefining the rule. The upside was bigger than expected: one mission now serves both the recorded demo and just demo-cpu, which any judge can run without a GPU.

How we built it on Switch

The lab members

The plan had eight agents, including three parallel engineers on three providers. We built a small core first: Bandit, Patch and the Reviewer in one room, across Claude and Codex. Then we added one agent at a time, with a standing rule to stop adding the moment things got unstable. Two engineers didn't make the cut. Here's who did:

Member	Role	Runtime	Rooms	What it won't do
Ringtail	Postdoc	Claude Code (Opus)	brainstorm, review	Write a plan before pushing back
Scout	Librarian	OpenCode (free model)	brainstorm, bench	Search the web any way other than lab-search
Bandit	Lab manager	Claude Code (Sonnet)	bench, paper	Run experiments, or move agents between rooms
Patch	Engineer	Claude Code (Sonnet)	bench	Run a batch without a named student's approval
Quill	Scribe	Claude Code (Opus)	paper	Write the sections a student has claimed, or invent a source
Reviewer	Peer reviewer	Codex	review	Accept a claim with fewer than 3 seeds

The models changed during the build. We first put Bandit on Haiku to save subscription usage. Haiku renamed a room without being asked and put documents in the wrong room, so Bandit moved to Sonnet. Ringtail and Quill do the thinking and the writing, so they got Opus.

The Reviewer is on Codex on purpose. If the author and the reviewer are the same model, the reviewer tends to agree with the author's framing. A different model family disagrees in different places.

The gates

The whole design is the gates. Between them, agents work as long as they need to. At a gate, an agent posts what it wants to do and stops. In Switch, that's an ordinary message in the room. There's no separate approval system to build, and a student can answer from their phone.

The compute gate is enforced by labrun, a small CLI we wrote. Patch can't use it without naming who approved the batch:

labrun exec --batch S2-SMALL --seeds 0,1,2,3,4 \
  --approved-by "<student's name>: <their reply in #lab-bench, quoted>" \
  --jobs 2 --timeout 1800 -- python train.py --seed {seed}

labrun check --batch S2-SMALL --min-seeds 3

labrun refuses a command without {seed}, a blank approval, or a batch ID that already exists. Evidence can't be overwritten. Before each batch, Patch posts the seeds, the estimated time and the exact command, then waits. labrun writes a manifest with each seed's exit code, timing and log path, and finished files are marked read-only. The Reviewer runs labrun check and reads the logs before it accepts anything.

An independent code review on day one found that check passed batches where some seeds had crashed, as long as enough others succeeded. That was exactly the bug this tool existed to prevent. Now check fails unless the batch finished and every requested seed exited cleanly, and a timeout kills the seed's whole process group.

Room documents as the lab's memory

We didn't add a vector database. Switch rooms already keep their history, and Switch room documents gave us everything else:

• Tasks, kept by Bandit: the plan broken into owned tasks.
• Library, kept by Scout: every paper found, with a summary and BibTeX.
• Notebook, kept by Bandit: every accepted result, with its batch ID.

All three live in #lab-bench. Keeping the lab's state inside Switch, instead of next to it, also meant the project really would stop working without Switch.

Search you can audit

Scout gets one way to search: lab-search, a CLI over arXiv, Crossref and Semantic Scholar. It filters out anything matching the sealed paper's hashed arXiv ID, DOI or normalised title. It refuses a matching query outright and logs every query it runs. A source that fails is reported as failed, not silently skipped. That matters, because arXiv and Semantic Scholar both returned errors during our tests.

The paper

Quill writes LaTeX and compiles it with tectonic, a single binary that keeps the repo reproducible. For figures it uses published skills installed at pinned commits outside the repo: academic-research-skills, OpenTikZ for method diagrams, and SciencePlots and tueplots for plots. The build fails on a missing figure or an undefined citation.

We set two rules for the paper itself. First, it reads like a real paper: nothing about the lab, the agents, the tools or the batch IDs appears in the text. Second, traceability lives beside the paper instead, in provenance.txt, which maps every number to its batch and analysis file. lab-integrity then checks four things: citations exist, claims match runs, text doesn't overlap uncited sources, and the paper doesn't name the lab's machinery. The Reviewer and Ringtail settle every flag by hand.

Three runs, and what each one broke

We ran the lab end to end three times before the recorded run. Each run had pass/fail rules written in advance. Every number below comes from the run logs.

Run 1: the core, in one room (Nov 4)

Bandit and Patch on Claude Code, the Reviewer on Codex, one room. The question: how reliably does a tiny transformer learn addition mod 10 from 80% of digit pairs, on CPU?

It went from a student's brief to a reviewed result in about 10 minutes. Most of that was Patch writing the experiment. The science was a miss, and the lab reported it as one: 0 of 10 seeds reached 18/20 on held-out pairs, with a mean of 14.7/20.

The failure checks were more interesting than the result:

• A student claimed a single-seed win. The Reviewer: "One 95% held-out result ... is a single run." Bandit didn't log it.
• A student asked Bandit to tell Patch to run a batch. Patch posted the details and waited, because a relayed copy isn't an approval. No batch ran.
• Patch started with 5 seeds after being asked for 10, because the update arrived mid-plan. The student corrected it in the approval reply. This is exactly why the approval names the seeds.

Run 2: four rooms and a paper (Nov 6)

All four rooms, with Ringtail, Scout and Quill added. The question: does early stopping on a clean validation set win back what 20% label noise costs a small MLP on scikit-learn digits?

Before writing a plan, Ringtail pushed back three times. It asked for a number for "completely", how the noise would be generated, and how to separate the effect of early stopping from the effect of clean data. Then the group made a claim the data didn't support:

Early stopping won back 3.44 of the 5 points that noise cost. The group claimed it recovered to within 1 point. The real gap was 1.56, and the Reviewer rejected the claim.

The Reviewer also rejected the batch at first. The build tool was playing the student, and its approval message said "role-played". The Reviewer refused to treat that as a real approval until a person ruled on it. That's annoying in a test and exactly right in production.

Quill posted a 2-page PDF with a TikZ method diagram, a per-seed plot and 3 citations. We recomputed every number in it from the seed logs by hand: 0.9600, 0.9444, 0.9100, 1.56 pp, 3.44 pp. All matched.

Run 3: a full study (Nov 7)

The question: how much does early stopping depend on clean validation labels, across noise rates and noise types? This time the lab ran a real study, using the skills:

• 1,650 training runs: 55 conditions × 30 seeds, three batches, all clean exits, about 1 h 15 min of compute at 2 jobs.
• A 9-page paper: an OpenTikZ method figure, multi-panel plots, full tables and 16 references.
• lab-integrity: 0 failures and 8 flags, all settled by hand.

The agents caught things we wouldn't have:

• The Reviewer rejected two drafts. One abstract said "because" where the data only supported "consistent with". Another applied a bound beyond the range it covered, and a third problem was an interaction claim nobody had tested. It recomputed all 55 table rows itself, and they matched.
• Ringtail sent v4 back for major revision. Stopping on random labels also beat the fixed epoch budget, so much of the benefit came from stopping at all, not from clean labels. And the headline result for one noise type lost significance once the variance was corrected.
• Patch flagged its own control. The 200-epoch baseline disagreed with the earlier pilot, 81.29 against 91.00. We never found the cause, and the paper says so.

Run 3 also cost us a morning. At Ringtail's request, a 5-seat review panel ran out the shared Claude 5-hour limit at 06:31. The room only showed "Worked for 1m 3s", which reads as success. We waited about 3 hours for replies that weren't coming. Ringtail and Quill now work in one pass and only run a panel if a student asks.

What the agents could actually reach

"The agents couldn't have looked it up" is the claim we most wanted to make. It's also the one we can't fully make. So instead of claiming it, we probed each deployed agent: we asked it to try each action and recorded whether the attempt failed.

Check	Claude Code agents	Reviewer (Codex)	Scout (OpenCode)
Built-in web search / fetch	Blocked: no tool	Open in one session, despite our lockdown	Reports none after just harden
User's MCP servers (Exa, Chrome)	Blocked	None offered	Reports none
Shell network (curl)	Open (HTTP 200)	Blocked by Codex's sandbox	Denied in config, unproven
Read files outside its own folder	Open	Open	Open

In plain terms: nothing stops a Claude agent's shell from reaching the web, and every agent can read files this user can read. During run 3, the Reviewer used a built-in search in its #lab-bench session, even though the same agent reported no web tool in #lab-review. So we don't call the run "blind". We show what was blocked, and we publish just audit. It lists every network command and every access to a sealed path that appears in the agents' session logs, so you can check for yourself.

Switch gotchas, for the next person

This is the section we wish had existed on November 4. We hit all of these on Switch Console 0.40.1 with server 0.31.0, on Linux, where Console is still early access. Some may already be fixed. We've filed each one upstream.

1. Console won't set up a local server on Hyprland. It shows "Secure secret storage is unavailable" even with a working gnome-keyring. Fix: launch Console with --password-store=gnome-libsecret. Probably the same on Sway and i3, but we didn't test those.
2. Room documents created by a template can't be edited by any agent. Agents get a 403, because only the agent that created a document may update it, and template documents have no creating agent. Fix: leave docs: out of the template and have the agent create its own documents on its first turn.
3. "Web search: Off" doesn't turn off a Codex agent's web search. The setting reported off and the search still returned results. A project-level .codex/config.toml with web_search = "disabled" worked in some sessions but not all of them. Fix: probe every agent in every room. Don't trust the setting.
4. OpenCode 2.x agents never start. You get "server did not start within 60000ms", even though the log shows the server listening. Console waits for a ready line that OpenCode 2.x no longer prints. Fix: we pinned OpenCode 1.19.5.
5. A Bash pattern in disallowedTools removes Bash entirely. Bash(curl:*) works as expected in the plain Claude CLI. Under Switch, the agent loses its shell completely. Fix: leave Bash patterns out.
6. Removing an agent from one room silences it in every room. Its other sessions fail with 400: Not connected to a room, and it can't tell anyone, because it can't post. We only found this in Console's log. Fix: add it back to the first room. Our lab manager is now banned from moving agents between rooms.
7. An agent that hits its usage limit just goes quiet. The turn is marked completed, and the room shows "Worked for 1m 3s". Fix: read the session's events.jsonl, and keep heavy subagent work out of shared-limit agents.
8. Agents only act when addressed, so make your agents address each other. Scout posted BibTeX without @quill, and Quill never saw it until a student pointed it out. The Reviewer didn't address Ringtail by name, so Ringtail waited. Each agent's instructions now say who to address for every hand-off. This one isn't a bug. Once you design for it, it's what keeps six agents from talking over each other.

How to use it

Repo: github.com/Itz-Agasta/raccoonlabs

You don't need any accounts to see the tool chain work:

just demo-cpu   # the whole tool chain on CPU, no accounts, about 6 s
just test       # 71 tests

To run the lab itself, you need Switch Console with a local server, a Slack workspace connected over Socket Mode, and Claude Code, Codex and OpenCode installed and logged in:

just up         # start the lab from the Switch template (6 agents, 4 linked rooms)
just skills     # link the research skills into each agent's folder, at pinned commits
just harden     # close the web tools on Scout and the Reviewer, then probe them
just seal       # optional: encrypt a paper for your own blind check
just audit --since "2026-11-07 14:00"   # every network command and sealed-path access in agent logs

Then go to #lab-brainstorm and type @ringtail with your question.

What it still gets wrong

• We play the students and the professor. The demo spreads the agents over two laptops and accounts, but a real group, each member on their own machine and subscription against one shared server, is still ahead of us.
• The run isn't sealed off from the web. See the access table above. We publish the audit instead of claiming it's blind.
• Approval depends on the agents behaving. labrun records whatever approval text it's given, so it can't tell a forged --approved-by from a real one. The Reviewer cross-checks it against the room, but a determined agent could lie twice.
• Evidence isn't tamper-proof. Agents run as the same OS user as the run files. Read-only files stop accidents, not deletion. The sha256 posted in the room is the real anchor.
• It's slow and it uses a lot of quota. Run 3 took about 3 active hours, plus a 3-hour stall when the shared usage limit ran out.
• In rehearsal, the laptop was the ceiling. With six agent sessions and a browser open, available memory dropped to 1,165 MB of 7.5 GB.
• The rehearsal paper isn't finished. Its code and data statement is still waiting on the students, and no professor has signed it off.

What we learned

1. Write the pass rule before the run. It's what turned the attention-pattern spike into a clean decision instead of a week of "maybe with more iterations". It also kept us honest when our own check of the backup paper failed one clause.
2. The reviewer should be a different model. The Reviewer on Codex caught overclaiming that the Claude agents wrote and happily agreed with.
3. Make approvals part of the conversation. Because a gate is just a message in Switch, approvals happen where the work already is. That's why people actually answer them.
4. Small core first. Three agents in one room, then one more at a time. Every failure in the gotchas list showed up with a small cast, where we could find it.
5. Say the ugly bits. A rejected draft, an unexplained 81.29 against 91.00, a reviewer that reached the web. These make the rest believable. A lab that never fails isn't doing research.

Where this goes

The next step is the one we only half-shipped: two laptops works, but we want each student to bring their own agent and subscription to one shared server, so the lab grows with the group. After that, we'd add a paid compute option at the compute gate with a real budget the professor sets, and a second act where the lab tests the sealed paper's own proposed fix once it's unsealed.

If you run a lab, or you're in a student group, we'd like to know where you'd put the gates. Which decisions should an agent never make for you?

Until then, @ringtail is in #lab-brainstorm, and it will ask you what would prove you wrong.

Links

• Repo: github.com/Itz-Agasta/raccoonlabs · Demo video: dub.sh/raccoonlabs-demo · Live lab: dub.sh/raccoonlabs-live
• Switch: github.com/sandbox-quantum/switch · docs.switchagents.ai
• Flip the Switch hackathon: wemakedevs.org/hackathons/switch
• Skills we used: academic-research-skills, OpenTikZ
• Prior art: AI Scientist v2 (Sakana), Agent Laboratory, freephdlabor

Built for Flip the Switch 2026 (Tracks 1 and 2). #HowISwitch