I expected more AI agents to make coding faster.
They didn’t.
That was not the result I was looking for, but it ended up being the most interesting result of the project.
Over the past few weeks, I have been building Sol-Luna Orchestrator, an open-source orchestration layer for OpenAI Codex.
The idea started with a simple question:
What if one strong AI could decide when it actually needed help from other AI agents?
Instead of automatically splitting every coding task across multiple workers, I wanted the supervisor to look at the work first and decide whether delegation was actually worth the coordination cost.
That distinction ended up mattering much more than I expected.
The basic idea
Sol-Luna has two roles.
GPT-5.6 Sol acts as the supervisor. It owns the overall task, decomposition, verification, and final review.
GPT-5.6 Luna instances act as bounded workers when Sol decides delegation is useful.
Conceptually:
Task
|
v
Sol Supervisor
|
Should I delegate?
/ \
No Yes
| |
Sol Split into
handles bounded tasks
the work |
Choose worker
effort
|
Luna Luna Luna
|
Results
|
Sol verifies
and reviews
There are really two separate adaptive decisions.
1. Adaptive delegation
Sol decides whether to delegate at all.
The optimal worker count is allowed to be zero.
Small tasks, tightly coupled work, or tasks where coordination looks more expensive than simply doing the work can stay entirely with Sol.
2. Adaptive worker effort
If Sol does delegate, it separately decides how much reasoning effort each Luna worker needs.
A worker can receive:
- Medium
- High
- XHigh
- Max
A mechanical change does not necessarily need the same reasoning budget as a difficult debugging problem.
So the goal was never simply to spawn more agents.
The goal was to let the strongest model decide how the work should be executed.
Making delegation safer
Once multiple coding agents start working at the same time, practical problems appear pretty quickly.
Workers can edit overlapping files. A worker can move outside its declared task scope. Verification can disagree with what the worker reports. Parallel Git operations can interfere with each other.
Sol-Luna adds controls around those problems.
Parallel workers run in isolated Git worktrees. Tasks declare their intended file scope, and scope violations are checked after execution. Verification is independently rerun instead of trusting a worker’s own PASS result. Sol remains responsible for reviewing the final output.
Workers also cannot recursively invoke the orchestrator and create their own worker trees.
That gave me a functioning orchestration system.
But I still had a more basic question.
When does parallelism actually become faster?
My earlier benchmarks had already shown that parallel Luna workers could beat sequential Luna delegation.
But one Sol working alone was still faster.
For small tasks, that made sense. Delegation itself has overhead.
So I assumed there must be a break-even point.
Make the task large enough. Give workers genuinely independent modules. Eventually multiple agents working at the same time should catch up and win.
That became the next experiment.
Looking for the crossover
I created progressively larger deterministic engineering fixtures.
The important part was that the parallel workloads were deliberately designed with independent streams of work.
I did not want to give parallel agents an artificially coupled task and then conclude that parallelism was bad.
The larger benchmark included:
- a four-module independent workload
- a six-module independent workload
- a coupled control where decomposition was intentionally inappropriate
The six-module fixture contained roughly 530 lines of specification and 85 deterministic assertions.
Each module could be worked on independently, so six workers could theoretically make progress at the same time.
This was deliberately much larger than the earlier fixtures, but it still fit comfortably inside a single Sol session.
That limitation matters.
I was testing whether clean parallelism alone was enough to create a crossover. I was not trying to simulate a huge production repository or a task running for several hours.
For each fixture, I compared three modes.
Sol working alone
Delegation was disabled.
Free choice
Delegation was available, but Sol was free to decide whether to use it.
This is closest to how I actually want the orchestrator to behave.
Forced parallel delegation
Sol was required to delegate the independent work so I could measure what happened when the worker path was definitely used.
The scale benchmark completed 19 out of 19 runs successfully.
Then came the interesting part.
Sol kept choosing zero workers
Across all six free-choice runs, Sol declined to delegate.
Every time.
At first, that can sound like an orchestration system refusing to do its job.
But then I compared those decisions with the forced-delegation results.
Four independent modules
| Mode | Median |
|---|---|
| Sol working alone | 171.5s |
| Free choice, Sol chose 0 workers | 120s |
| Forced parallel, 4 workers | 250s |
The free-choice runs used zero Luna workers.
Six independent modules
| Mode | Median |
|---|---|
| Sol working alone | 189.5s |
| Free choice, Sol chose 0 workers | 186.5s |
| Forced parallel, 6 workers | 394.5s |
Again, Sol chose zero workers.
The interesting result was not simply that parallel workers lost.
The supervisor had been given the option to use them, declined to do so, and none of the measured workloads gave me evidence that this was the wrong call.
Going from four independent streams to six also did not move forced parallel execution closer to the solo baseline.
It moved further away.
Forced parallel was roughly 46% slower than solo at four streams and roughly 108% slower at six.
So the benchmark did not find the crossover I expected.
More agents were not free
Token usage told a similar story.
On the independent workloads, forced parallel execution used approximately:
- 5.1x the known tokens of solo execution at four streams
- 4.8x the known tokens of solo execution at six streams
There was no token crossover either.
That does not make the workers useless.
Parallel workers can still provide useful properties such as isolated workspaces, bounded tasks, separate context, independent verification, and explicit ownership of different pieces of work.
And when delegation is already required, the earlier benchmark showed that parallel workers can beat sequential delegation.
But for raw speed on the workloads I measured, forcing delegation was clearly not winning.
Where was the time going?
One possibility was that the orchestration machinery itself was expensive.
Maybe Git worktrees or integration were eating all the time.
They were not.
Measured median phases looked roughly like this:
| Phase | Median |
|---|---|
| Supervisor work before batch | 37.1s |
| Worktree setup | 0.8s |
| Slowest worker | 187.1s |
| Integration | 0.4s |
| Supervisor review | 32.4s |
The mechanical Git orchestration was tiny.
Worktree setup plus integration was around 1.2 seconds.
Most of the fixed overhead came from useful supervisor work: decomposing the task, writing bounded worker contracts, and reviewing the results afterward.
But another effect became much more visible in the six-worker runs.
The slowest worker matters
Parallel completion time depends heavily on the worker that finishes last.
In one six-worker run, five workers finished within about 95 seconds.
One worker took 333 seconds.
The observed max-to-median worker-duration ratios in the two six-worker runs were around 3.5x and 2.7x.
So in these runs, finishing five tasks quickly did not help enough because the batch still had to wait for the final worker.
I also calculated a simple counterfactual using the measured timings.
If every worker in the six-stream runs had completed around that run’s median worker duration, parallel execution would have landed around 176 seconds, compared with the 189.5-second solo median.
That counterfactual would have crossed the solo median.
But no observed run actually did.
The 176-second number is arithmetic on measured timings, not a benchmark result.
And with only two repetitions of the six-worker cell, I do not have enough data to characterize the full distribution of worker durations.
So the careful conclusion is that the slow-worker tail looks like a strong candidate for an important parallel-latency constraint.
It is not proven to be the only one.
The result changed how I think about orchestration
When I started this project, I thought a successful orchestrator would mainly be good at distributing work.
I now think that definition is incomplete.
A good orchestrator should also be good at not distributing work.
Across the workloads I measured, Sol chose zero workers in every free-choice run. Forced delegation was slower on the corresponding fixtures.
That does not prove that the free-choice policy itself caused the faster timings. These are stochastic model runs, and separate runs can behave differently.
But none of the measured workloads provided evidence that declining delegation was the wrong decision.
That leads to what is probably my favorite idea from the project so far:
The optimal number of workers can be zero.
More agents are a tool, not an objective.
Good orchestration is not about maximizing agent count.
Sometimes the strongest agent should simply do the work itself.
This does not mean one agent always wins
I want to be careful about what these results actually show.
They do not prove that one strong agent is universally better than multiple agents.
Every workload I tested still fit comfortably inside one Sol session.
A much larger production repository may behave differently.
A task running for hours may behave differently.
Work that requires several highly specialized contexts may behave differently.
And a workload large enough to push beyond what one supervisor can comfortably keep in context may be exactly where delegation starts to become much more valuable.
That remains an open question.
At what point does keeping everything inside one strong agent become more expensive than coordinating several workers?
I do not have that answer yet.
And I think that is more interesting than simply adding eight or ten agents to another synthetic fixture until I find a benchmark where parallelism wins.
The negative result is staying public
One temptation with developer-tool benchmarks is to keep changing the experiment until your tool wins.
I did not want to do that.
My original prediction was that larger independent workloads would eventually produce a latency crossover.
The benchmark falsified that prediction in the regime I tested.
So the methodology, benchmark harness, raw records, and results are staying public.
If someone wants to try the same setup on a genuinely large real-world workload, I would genuinely like to see what happens.
That is one of the benefits of making the whole thing open source.
Try it
Sol-Luna Orchestrator is open source:
https://github.com/mahadansar/sol-luna-orchestrator
Install it with:
npm install -g sol-luna-orchestrator
sol-luna-orchestrator init
The repository includes the architecture, security model, benchmark fixtures, raw results, and documentation around the delegation policy.
For now, I am deliberately holding off on major new features.
I would rather see how people actually use it, what larger real-world workloads expose, and whether the assumptions behind the project continue to hold before deciding what is worth building next.
Final thought
I started this experiment asking:
How many agents should work on a coding task?
I ended up with a question I like much more:
When should a strong agent delegate at all?
For the workloads I have measured so far, the answer was often:
It shouldn’t.
And I think knowing that is part of orchestration too.
Top comments (2)
We hit the same result on a knowledge graph pipeline. The orchestrator defaulted to single-worker mode on about 70% of tasks, and we thought it was misconfigured. The subtask boundaries we'd defined were just too tightly coupled for delegation to pay. The worktree isolation is something we still don't have, which is probably why concurrent worker conflicts are still messier than they should be.
Yeah this sounds really close to what I saw. The coupling part is probably the key, and worktrees definitely made the concurrent runs way cleaner on my side. Curious what kind of conflicts you were seeing without them?