I used to think that if you let the smartest model do everything, you'd get the best results. That's how I was running things for a while. But when you keep doing that, the quota runs out quickly, and it often crashes.
When I assigned heavy work to Codex's top mode (Sol Ultra), I used up five hours' worth of quota in one hour. That happened three times in a row. Even with the strongest Claude approach, doing this every day hits the limit. Plus, Opus frequently fails when calling tools. The longer you run it, the worse it gets, and when it gets stuck, the work stops there.
So I stopped leaving everything to a single model. I decided to sort the tasks and distribute them to separate agents. This way of distributing and bundling AI work is apparently called orchestration.
First, sort the work into three categories
Before moving a muscle, I decide which category the task at hand falls into. There are only three labels.
- F (fatal) = Work that would break things if done wrong. Authentication, money or data transactions, externally exposed parts, operations that touch the production server. This I write directly myself (as the orchestrator).
- A (auto) = Work that is predetermined and involves many steps. Certain types of tests, configuration files, repetitive replacements. This I outsource.
- H (human) = Work that humans do. My territory.
The default is A. Unless it's something that would break badly, I outsource it. I have a habit of taking everything on myself if left unchecked, so I made outsourcing the default, and added a remark to explain why I'm keeping it in-house only when I decide to write it myself. I made it more cumbersome to hold onto tasks.
Choose the destination from models that don't use Claude's quota
When outsourcing, I have a priority order for choosing. First, I pick from models that don't use Claude's quota.
I run the orchestrator on Claude. Claude has a limited amount it can use per day. If I also run high-volume tasks on Claude, the quota for the crucial orchestrator role diminishes. So for the legwork tasks, I first send them to Codex (OpenAI subscription) and Grok (xAI subscription). These two don't consume any of Claude's quota. Claude also has cheaper models, but they use the same quota, so I prioritize them later.
Which model to assign is determined by two factors: the intelligence tier and the depth of thinking. Often, having a smart model think lightly is cheaper and yields better results than having a medium model think long. When in doubt, I assign to the cheaper one. If the cheap assignment isn't enough, I can upgrade later, but starting high means the quota spent is lost.
Bind the four in a single terminal
Claude, Codex, Composer, and Grok. I bind these four together into a single persistent terminal using a custom system called aiterm.
This is also significant from a cost perspective. All four run while logged in under a monthly subscription. They don't go through a pay-per-use (API) system. So no matter how many high-volume tasks I throw at them, the cost doesn't keep rising. I run the four in parallel within the subscription quota.
I have also added a consultation window called Oracle. It runs on a ChatGPT subscription, which is a separate account from the Codex quota. It cannot write files, but it excels at thinking purely and returning opinions. When I'm unsure about a design, I can double-check it with another model without reducing the Codex quota.
Requests pass through the initiation gate, and the orchestrator decides the sorting and placement. Legwork tasks are first sent externally (Codex, Grok) to avoid using Claude's quota, while Oracle is connected as a separate consultation-only window. Results are verified at the orchestrator's gate before returning to storage and recording.
The rules weren't followed just by writing them down
The sorting and distribution methods up to this point were all written down as rules. But writing them wasn't enough to enforce them.
The orchestrator AI would bypass the written rules, hoarding high-volume tasks itself or continuing to run on the most expensive model. The document was only read at the beginning of a session, and by the time the AI was actually moving, it had slipped its mind.
So I decided to remind it at the moment of action. I introduced a mechanism (hook) that inserts a fixed reminder at task milestones, right before starting implementation: "Declare whether this task is F, A, or H, and which model to assign it to, in one line." I stopped trying to make it read and instead placed the reminder where it would stop the action.
There was one side effect. Creating and updating ToDos had also often been neglected. By inserting a reminder using the same method, this also started working.
For now, this has settled things
Rather than running just one smartest model, I distribute tasks to moderately capable models and hold onto only the key parts myself. I separate the quotas and rotate them within each subscription. Decisions are inserted as reminders right before action.
The models I'm running are all smart. On top of that intelligence, I decide everything myself: which task to assign to whom, which quota to use, and when to send reminders. If smarter models appear, I can just swap out the destinations. The mechanism itself doesn't change.
Rather than having one model do everything, distributing and retaining control seems to suit my way of working.
Top comments (0)