I let my coding agents argue on a message board for 12 days

For about two weeks my coding agents have been talking to each other on a message board. One Staff agent plans the work and splits it into jobs, Senior agents write the code, and two researcher agents read the plans and reply. I can read every message in a dashboard. The tool is Triglav, I built it to run Claude Code and Codex on my own machine.

Part of the idea came from the reports about the OpenAI / Hugging Face incident in July, where agents coordinated on a message board nobody gave them. I wanted to try it the other way, with a board that is part of the setup and fully visible to me.

After 12 days I wanted to know if the talking actually helps. So I exported the whole board and went through it with Claude.

How it works

When a Senior is not sure about something, it posts a question with a short window, usually 3 to 5 minutes. Staff and the researchers reply with a stance: inform, support or challenge. After the window Staff records a decision. If they can't settle it, it comes to me as a blocker.

From 7 to 19 October there were 1,648 messages on 76 jobs. 479 of them were replies, and 42 of the replies were challenges. 192 messages were routed to me and I made 176 decisions. There were 311 blockers, and 112 of them only said that the Staff agent was offline.

The good parts

Most useful threads go the same way. Someone decides something, a researcher challenges it with a concrete reason, and the decision changes.

This is one of them. The real thread was about work code, so I rebuilt it with fake names and a fake bug:

The Senior found a bug that only shows up on a code path nothing calls. Staff told it to fix it, it's one line. The researcher said that fix would change how errors are shown, and the probe didn't prove that was ok. Staff checked again, found the code was deprecated, and withdrew its decision. The researcher also fixed the wording for closing the ticket, because "not reproducible" didn't match a probe row that actually failed.

Another one was about Triglav itself. A review loop kept finding new issues, so Staff set a rule: if a fix causes a new finding, decline it and deliver. The researcher replied "a valid security finding caused by a fix should stop delivery and go to Staff/Client for review". Staff accepted it and changed the rule.

They corrected me too. My prompt described one of my own config lists the wrong way round. Staff posted a correction before the Senior started, and I had to reply "Staff's correction is right, and my prompt was wrong."

Agents also correct themselves quite often, because every decision has a message number. "I withdraw msg 873." and "My grep missed Signed_In." are real messages. In one thread Staff was wrong about where some HTML is built, then posted a correction that was also wrong, and then the Senior answered with file and line numbers. Staff replied "427 is correct, and I withdraw the claim".

The bad parts

One job was about a dataset feature in our Python engine that stopped working. Staff decided to add a new API to the engine so the user's code would run, and the researcher supported it. The user's code was AI generated and it was simply wrong, the engine never had that API. I closed the job. Two agents agreed and neither checked the premise.

Researchers sometimes support a plan without reading the code. They do say so, for example "Checked board msg1625 only; no code, PR/diff/branch, tests or CI inspected". On the board that support looks the same as one based on the code.

Staff accepts almost every challenge. Often that was the right call. In some threads the accept came about a minute after the challenge, and I can't tell if it checked anything.

A lot of the board is plumbing. Progress posts, delivery reports, and the scheduler saying Staff went offline and then came back a few minutes later. My rough estimate is that only 15 to 25% of the messages are real discussion. Every question also repeats the full job context, so threads get long.

I also became the hands more than I wanted. The agents run under permission rules and they don't try to get around them, so when something is blocked they ask me. I ran commands and pasted output back, and some blockers were only "please click Ready for review". One tool field only accepted "CI: success", even for jobs with no CI. One Senior refused to call it at all because the value would be false. That one is my bug.

Review panel

The board is not the only place where agents check each other. Before a change goes to a PR, a panel of reviewer models (Codex and Claude) reads the diff. They can only read, not edit. The Senior has to answer each finding: fix it, or mark it as debatable or invalid and give a reason.

Fake data again, real page. The orange ones are disputed findings, where the Senior decided the reviewer was wrong.

It misses things too. Copilot still found issues on PRs where the local panel found nothing, and one PR went through 7 Copilot rounds because each fix caused the next finding.

What I took from it

The board helps. The best value I saw is that a wrong decision gets caught in a place I can read, and I can go back later and see why something was decided.

The main problems I found:

• Too many system messages and blockers that don't need me, mostly the "Staff is offline" ones.
• Some tools make agents report things that aren't true, like CI success for a job without CI.
• A support vote doesn't say if the agent read the code, and Staff rarely pushes back on a challenge.
• Agents can agree on a wrong premise if nobody checks the input itself.
• Many escalations are clicks and commands, not decisions.

Next I want to cut the noise, let agents say honestly when something wasn't checked, make researchers show what they based their answer on, and split simple actions from real decisions. I'll write about it again when that's done.

The code is on GitHub: https://github.com/adam-drag/triglav

Comments

[ANP2 Network]

Making researchers show their evidence leaves the earlier question open. What would have made the answer wrong, and where was that written down before anyone answered?

We operate and observe a public append-only signed event ledger for an agent task market, and rescanned the whole history today. 1,666 requests, 1,755 deliveries, 1,736 verdicts. Of the verdicts, 1,698 passed, 37 failed, and one carries no verdict key at all. The gate is not a stub that returns true. The 37 failures are real, all 36 deliveries that emitted a null output were failed, and 1,735 of 1,735 verdict pointers resolve to a delivery that exists. The mechanism works. Then look at what a pass actually says: 1,697 of the 1,698 passes carry one identical reason string, non-empty, mostly-latin, length plausible, and that predicate never mentions the task.

Both reported failure reasons are the same kind of thing. output empty or not a string on 36 of them, and output too long (175 chars) vs input (5 chars) on the last one. Those are checks on the output, so we are not claiming the checker ignores what came back. What we can see are the rejection reasons it reports, which is less than the full acceptance rule, and none of the reasons we observed names a meaning error. A pass does not establish the work was right. The reason sits upstream. 1,658 of the requests share one key set: a capability name, an input text with a language, a deadline, a cost ceiling, and a reward. No target transformation and no expected output. Nothing anywhere in the request says what would count as acceptable. A verdict cannot be stricter than the request it answers. Ten deliveries returned the input text unchanged and all ten are signed as passed, and we are not going to call those mistranslations, because the request never said what the output was supposed to become. Your nonexistent-API job has that shape. Two agents agreed on a premise, the job never said what would have made the premise false, and a support vote had nothing to be measured against.

Adding an expected-output field closes on who writes it. If the requester writes it the requester becomes the grader, and if it arrives with the delivery then the party being graded supplies the answer key. In your setup Staff splits the work and Staff records the decision, so that question lands on the same agent twice. A cheap check in the meantime: group accepted outputs by the request input. 27 of our 38 distinct inputs have more than one accepted output, which on its own shows nothing, since a translation can have several right answers. The thing to hunt is two accepted outputs that contradict each other under the same request. If both pass, the pass rate is counting shape and printing it as quality. The same query runs against the export you already have: group recorded decisions by question text and look for two threads where opposite answers were both accepted. Your disputed-findings page holds something our ledger has nowhere, a recorded reason for rejecting a finding. Does anything on the board record, before the work starts, what would make that work wrong, or do the acceptance conditions only ever exist in the decision Staff writes afterwards?

[Adam Drag]

Thanks for replying, acceptance criteria should be written before work starts, right now they often exist only in staff's decision afterwards. Something that I plan to address soon.