Two coding agents agreed—and still built incompatible APIs

#ai #agents #testing

The frontend agent proposed API A. The backend proposed API B. Later, the frontend agreed to B—and the backend agreed to A.

Both were cooperating. Their implementations still did not connect.

While building Sunny Code, we compared one coding agent working across two repositories with two agents splitting frontend and backend work. That exchange changed what we needed to measure: sending a message, receiving it in context, agreeing on a contract, and implementing that contract are separate events.

Three rounds, three different questions

Each round had four paired comparisons using GPT-6.1 Sol at medium reasoning. The first two compared one session with two. The third used two agents in both arms and changed message delivery timing. These rounds should not be pooled into a leaderboard.

1. Start with an agreed API

The task was an OAuth integration with persistence, concurrent callbacks, expiry, workspace isolation, retries, and browser navigation. Both arrangements received the complete interface contract and shared a task-level budget.

All eight saved artifacts passed 28 checks after we corrected two evaluator assertions that incorrectly required lowercase “connected” instead of accepting “Connected”. We re-evaluated unchanged artifacts; the agents did not repair them.

Two sessions were faster in three of four pairs, with mean implementation time 12.6% lower. Their cumulative reported tokens were 2.17× the single-session total. That includes repeated cache reads, so it does not mean twice the dollar cost. The timing also excluded independent acceptance and human review.

This supported a narrow finding: splitting an already-specified interface can give a parallel benefit. It did not tell us whether the agents could negotiate one.

2. Leave the interface unsettled

We added safe disconnection to a working OAuth product. An old request retried after a lost response must not disconnect a newly established connection.

Some pairs started with incompatible API drafts; others continued real maintenance histories. Both arrangements could access the relevant information, but its organization and retrieval costs differed.

Under a shared ten-minute, 1.6-million cumulative-token and 24,000-output-token budget:

Round two outcome	One session, two repos	Two sessions, queued messages
Completed within budget and passed	4/4	0/4
Saved artifact passed all 48 checks	4/4	1/4

Passing code and completed delivery were separate outcomes. One two-session artifact passed every check after exceeding its budget.

The bridge accepted 38 messages, but only 20 entered recipients’ persisted histories before the tasks stopped. The delivery mode queued messages until the recipient’s current execution settled. A decision could remain waiting while both agents implemented their own assumptions.

3. Change when messages enter context

Both arms now used two agents, reusing round two’s starting snapshots, tasks, histories and budgets. Queued follow-ups were compared with delivery at tool boundaries during execution.

Round three outcome	Queued follow-up	Mid-execution delivery
Completed within budget and passed	0/4	1/4
Saved artifact passed all 48 checks	1/4	4/4
Messages entered before stop	14/32	45/47

Earlier entry aligned the final code more often in this small comparison. It did not solve bounded completion: three of four mid-execution runs still exceeded budget. Delivery at a tool boundary also does not interrupt a running tool or guarantee arrival before the immediately following step.

The crossed-confirmation exchange occurred here too. In the mid-execution arm, the agents subsequently froze a final version and aligned their code. That artifact passed all 48 checks, but exceeded the output budget.

Check the checks, too

We later added 12 boundary checks and calibrated them against a reference implementation and four deliberately broken variants. The reference passed all 12; the corresponding checks caught each broken variant.

We then evaluated unchanged copies of all 16 saved artifacts from rounds two and three, without rerunning the agents. All ten previously passing artifacts passed the added checks. This increases confidence in those particular boundaries, not complete coverage: the audit was post-hoc and the reviewer knew the earlier outcomes.

What I would check next time

For a new task spanning two repositories, we currently lean toward explicitly attaching both to one session. That is a product judgment informed by these runs, not a universal ranking.

For separate sessions, I want the record to distinguish queued → entered context → answered → implemented, with one agreed contract checked against both implementations. Mandatory handshakes and designated coordinators were not tested here.

There were only four pairs per round. File-access boundaries were primarily instruction-based, and one two-agent run in each of rounds two and three used host Playwright outside the built-in-dependencies-only premise. Those runs remain disclosed in the results. We did not measure full human review and rework time.

The useful failure was concrete: two agents can both say yes while building different interfaces. Message counts alone would have missed it.

Adapted from our original experiment, with per-run results and method disclosures. AI-assisted writing; experiment findings and limitations checked against the published record.