A design for running AI agents like a company, with seats, budgets, probation, benching, and a human overseer on every seat.
Most writing about AI agents is about how to build one: the prompt, the tools, the loop. Far less is about how to run dozens of them once you have them.
Who decides how much budget each agent gets? What happens when one keeps doing bad work? Who can stop it, and how fast?
Companies already answered these questions for people: org charts, budgets, probation periods, performance reviews, managers. So I tried an experiment and designed an organization for agents, with humans in control at every level.
👉 Interactive version: probita-ai.github.io/agent-org-model
(It has a crew lead console and a control room you can click around in.)
1. Every box is a seat
The key idea is that a box on the org chart isn't an agent. It's a seat: a role with authority, a budget share and reports. A human or an agent can fill it.
seat: platform-lead
occupant: { type: agent, agent: lead-agent@v2 }
autonomy: L2 # suggest; overseer approves
overseer: { user: priya, backup: sam }
reports_to: cto
Handing a seat to an agent, or taking it back, means changing one field. The authority, budget and audit trail stay the same.
In the example org, the board and CEO are human (the CEO has an agent copilot). Some executives and crew leads are human and some are agents. The workers are agents.
2. Capacity flows down as percentages
"Capacity" means the money, tokens and concurrent job slots a unit may use. Each leader receives a percentage of its parent's capacity and splits it among its reports. Whatever it doesn't assign stays in reserve.
Here's one monthly budget traced from the top to a single agent:
| Step | Share | Monthly |
|---|---|---|
| Board sets the total | — | $10,000 |
| CEO gives Engineering | 40% | $4,000 |
| CTO gives the Platform crew | 60% | $2,400 |
| Platform Lead gives the Coder agent | 40% | $960 |
This makes cost a property of the org structure, not a surprise on the bill. No agent can raise its own budget or one above it.
3. Crews do the work
A crew is a reusable team of agents set up for one kind of job: writer → editor → fact-checker, or coder → tester → reviewer. The crew is a template. Each use of it is a separate job with its own trace, cost and outputs.
(CrewAI uses "crew" in a similar sense. What I'm adding here is everything around the crew: seats, budgets, hiring and oversight.)
4. Hire, bench and rehire
This is the part people react to most: leaders can "fire" agents. Here's the lifecycle:
- Catalog: agent definitions live in a registry. A lead picks one its authority allows.
- Probation: the new agent runs in shadow mode at 0% capacity. It does the work alongside the crew, its outputs are compared, and nothing it does takes effect.
- Active: it gets a real share of capacity.
- Review: an HR agent scores it on success rate, cost per job, checklist pass rate and how often humans override it.
After a review, it gets promoted, retrained (new prompt, skills or model, then back to probation), benched, or retired.
Benching is "firing" that you can undo. The agent loses its capacity and access but keeps its definition and history, so it can be rehired. Only retirement is permanent, and a human has to approve it.
5. Autonomy is a dial, not a switch
Each agent seat runs at one of four levels:
| Level | What happens |
|---|---|
| L1 Manual | A human fills the seat. Agents only do assigned tasks. |
| L2 Suggest | The agent proposes hires, benches and rehires. A human approves each one. |
| L3 Act and report | The agent acts within its limits. A human reviews afterwards and can undo. |
| L4 Autonomous | The agent acts within its limits. Only exceptions reach a human. |
A seat moves up a level only when its scorecard shows it's ready. You don't have to bet on AI managers on day one.
6. A human overseer on every agent seat
Every agent seat has a named human overseer and a backup. An overseer can always:
- See live status, spend, scores and recent decisions
- Pause one agent, a crew, a division, or everything
- Roll back any applied change, because each one stores its before-state
- Override any decision (overrides count against the agent's score)
- Take back the seat from the agent
- Audit the ledger for who decided what, and who approved it
Two rules I think matter most:
- No unwatched agents. An agent seat can't be activated without an overseer.
- The off switch is plain software. Pausing never depends on an agent cooperating, and agents can't lower their own oversight.
Humans can't read everything, so oversight scales with risk:
| Risk | Example | Oversight |
|---|---|---|
| Low | Drafts, internal summaries | Daily digest, 2% sampled |
| Medium | Customer replies, code changes | Alerts, 10% sampled, rollback ready |
| High | Money, production, legal, people | Human approval first |
Will this actually work?
Partly, and the split matters.
What I'm confident about: the governance layer. Budget splits, permissions that only flow down, probation, benching, approvals and a ledger are ordinary software. They're enforced by the platform, not by a model behaving well. Crews of agents doing narrow, well-defined jobs also work today.
What I'm skeptical about:
- Deep chains of agents compound mistakes. CEO agent → CTO agent → lead agent → worker means context gets lost at every handoff. Multi-agent setups often do worse than one good agent with good tools.
- An "AI CEO" making strategy is the weakest link. Models are much better at doing scoped work than at long-term judgment.
- Firing is only as good as the scorecard. Without real signals like tests, checklists and human overrides, you're benching agents based on another model's opinion. Evals are the hardest part.
- Management layers cost money and do no actual work.
So I'd build it bottom-up: humans in every management seat, one crew with a real scorecard, and the full probation → active → bench → rehire loop working there first. Raise a seat's autonomy only when the numbers support it.
What's next
This is a concept, not a product yet. The page and design are here:
I'd like to hear from people running agents in production:
- Which seat would you never give to an agent?
- What signal would you trust enough to bench an agent automatically?
Top comments (1)
A planted bug: break the code on purpose and the agent's tests have to go red. I'd only bench on it if the run also shows the mutant got compiled. In one of my runs a mutation control reported 155 tests and 0 failures while the mutant sat on disk. Gradle had marked the compile task UP-TO-DATE, so the tests ran against classes built before the change, and the XML report was fresh. Without a forced recompile, about one control run in four went bad like that. With the recompile forced, none of eleven did.