Decide Where the Agent Runs With a Four-Axis Fit Ledger

Every week brings another round of confident model announcements, and every week my review queue looks roughly the same as it did a month earlier. That gap is not a scandal and it is not a mystery to me; it is a routing problem that most teams never name out loud. We keep asking which model is best when the operationally useful question is which compute tier this specific unit of work should run on. I have been answering that question with a four-axis ledger instead of a price tag, and what follows is the entire workflow with runnable code.

Three tiers show up in that ledger: a free hosted tier, a paid hosted tier, and self-hosted compute. A concrete example of the first is MonkeyCode's free model access, which the operator describes as including free model access plus a free server option, with a free allowance reported at ten million tokens. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I have not audited those numbers, so treat them as claims to verify inside your own account rather than constants to plan around.

Conclusion First: Price Is the Last Question, Not the First

When a free tier appears, the instinct is to route everything to it and celebrate the savings on Monday. That instinct fails for a boring reason: the expensive part of agent work was never the token bill, it was the human time spent catching bad output. A free generation that costs me forty minutes of review is more expensive than a paid generation I can trust on the first pass.

So I stopped optimizing for price and started optimizing for fit. Fit means the tier matches the data boundary, the reversibility of the output, the patience of the waiting human, and the blast of a missed error. Price is a derived quantity that falls out of those four answers, not an input you start from.

So why do so many teams still open the pricing page first? Because price is the only number that arrives pre-formatted on a marketing site. The other three axes require you to measure your own work, which is slower and far more honest.

The Four Axes That Actually Decide the Route

1. Egress class. Can the prompt and its context leave your boundary, or does it contain internal and regulated material? This axis has veto power over every other consideration.
2. Output reversibility. If the result is wrong, can you delete it with no cleanup, or does it land in a branch, a ticket, or a production path? Throwaway output tolerates a weaker tier.
3. Latency budget. How long will a human actually wait before they switch windows and lose the thread? This is a measurement, not a feeling.
4. Cost of a wrong answer. What happens if the error ships and nobody catches it before Friday? High cost buys review, not a fancier model.

Notice that nothing on that list mentions price per million tokens. That is deliberate, because price only becomes meaningful after the four axes have already narrowed the field.

Step 1: Write the Policy Before You Write the Router

I keep the policy in a file so the arguments happen in review rather than inside my head at midnight. The thresholds below are placeholders, and you should replace each one with your own measured numbers.

# policy.yaml
axes:
  egress_class: [public, internal, regulated]
  reversible: [true, false]
  latency_budget_s: [interactive_10, batch_600]
  wrong_answer_cost: [low, medium, high]

routes:
  - when: {egress_class: regulated}
    tier: self_hosted
    note: 'boundary rule, no hosted call at any price'
  - when: {egress_class: internal}
    tier: self_hosted
  - when: {egress_class: public, reversible: true, wrong_answer_cost: low}
    tier: free_hosted
    note: 'output gets discarded, nothing downstream depends on it'
  - when: {egress_class: public, latency_budget_s: interactive_10}
    tier: paid_hosted
    note: 'measure your own p95 before you trust this line'
  - default: true
    tier: paid_hosted

Step 2: Implement the Router as a Pure Function

The router should be boring, deterministic, and easy to test, which means no network calls and no hidden state. Here it is as a single file with no dependencies beyond the standard library.

# route.py
from dataclasses import dataclass
from enum import Enum
from typing import Literal

class Tier(str, Enum):
    FREE_HOSTED = 'free_hosted'
    PAID_HOSTED = 'paid_hosted'
    SELF_HOSTED = 'self_hosted'

Egress = Literal['public', 'internal', 'regulated']
Cost = Literal['low', 'medium', 'high']

@dataclass(frozen=True)
class Task:
    name: str
    egress_class: Egress
    reversible: bool
    latency_budget_s: int
    wrong_answer_cost: Cost

def route(task: Task) -> tuple[Tier, str]:
    if task.egress_class in ('regulated', 'internal'):
        return Tier.SELF_HOSTED, 'rule 1: the data boundary beats every other axis'
    if task.reversible and task.wrong_answer_cost == 'low':
        return Tier.FREE_HOSTED, 'rule 2: throwaway output over public data'
    if task.latency_budget_s <= 10:
        return Tier.PAID_HOSTED, 'rule 3: interactive budget, check your own p95'
    return Tier.PAID_HOSTED, 'rule 4: default until you have evidence to move'

Two things deserve emphasis here. First, the first matching rule wins, and I ordered them so that the unglamorous boundary rule can never be overridden by a clever saving. Second, every return carries a reason string, because a router that cannot explain itself is impossible to debug when it misroutes.

Step 3: Split Authoring From Verification

Here is the part that changed my workflow the most: authoring and verification do not need the same tier. Authoring is probabilistic and touches your context, while verification is deterministic and touches your repository.

That split means I can run generation on whichever tier the axes allow and send the test suite somewhere cheaper, as long as the repository itself is clean. Verification on a disposable workspace is exactly the kind of job a free server option fits, because the output is a pass or fail rather than a mergeable artifact.

def route_verification(task: Task, repo_has_secrets: bool) -> tuple[Tier, str]:
    if task.egress_class != 'public':
        return Tier.SELF_HOSTED, 'verification still crosses the same boundary'
    if repo_has_secrets:
        return Tier.SELF_HOSTED, 'build logs can echo environment state'
    return Tier.FREE_HOSTED, 'deterministic job, disposable workspace, no secrets'

That guard clause is not paranoia; build logs leak more credentials than anyone expects. Run a coarse preflight before you hand a repository to any hosted runner.

# coarse preflight, not a replacement for a real secret scanner
git grep -nIE '(AKIA[0-9A-Z]{16}|-----BEGIN [A-Z ]*PRIVATE KEY-----)' -- . \
  && echo 'key-like material in tracked files: stop and self-host' \
  || echo 'no obvious key material found in tracked files'

Step 4: Prove the Router With Labeled Tasks

A policy file that only lives in your head drifts within two weeks, so I pin expectations in a test. These cases represent my labeled decisions, not universal truths, and the suite protects consistency rather than correctness.

# test_route.py
import pytest
from route import Task, Tier, route, route_verification

CASES = [
    (Task('summarize internal incident', 'internal', True, 5, 'low'), Tier.SELF_HOSTED),
    (Task('rename vars in sample repo', 'public', True, 5, 'low'), Tier.FREE_HOSTED),
    (Task('rewrite prod migration', 'public', False, 900, 'high'), Tier.PAID_HOSTED),
    (Task('generate throwaway fixtures', 'public', True, 600, 'low'), Tier.FREE_HOSTED),
]

@pytest.mark.parametrize('task,expected', CASES)
def test_route(task, expected):
    tier, reason = route(task)
    assert tier is expected, reason

def test_verification_refuses_dirty_repo():
    task = Task('run unit tests', 'public', True, 600, 'low')
    assert route_verification(task, repo_has_secrets=True)[0] is Tier.SELF_HOSTED

Run it with pytest -q. When a case fails after a policy edit, you get to argue with your past self instead of guessing, which is the entire point of writing the policy down in the first place.

Step 5: Change Rules, Not Cases

Every misroute gets one line in a log, because patterns only appear after you stop explaining incidents away. The log is plain JSON Lines so the Friday review ritual stays cheap and boring.

printf '%s\n' '{"task":"fixture rewrite","chosen":"free_hosted","problem":"output was merged, not discarded","fix":"make reversible an explicit input"}' >> misfits.jsonl
jq -r '.fix' misfits.jsonl | sort | uniq -c | sort -nr | head

The rule I follow is simple: if the same fix shows up twice in a month, I amend the policy instead of adding a third exception. Exceptions rot quietly, while a changed rule is visible to everyone who reads the file. My review hour is short because most of the questions were already answered on Monday.

Where Free Access and a Free Server Fit

The free tier earns its place in route two: public data, throwaway output, low wrong-answer cost. Fixture generation, sample-data munging, docstring drafts, and "explain this stack trace" all land there comfortably, because a bad answer costs me nothing but a rerun.

The free server option fits the verification half rather than the authoring half, which surprised me at first. Deterministic test runs, lint passes, and build checks produce no mergeable artifact, so there is nothing to review and nothing to regret. Keep credentials out of the workspace and the boundary question mostly answers itself.

What I do not do is pretend the free tier is an availability guarantee. Operator-supplied allowances, quotas, and server terms change, and any workflow that breaks when they change was never a workflow; it was a bet.

The Decision Table I Use

Situation	Route	Reasoning
Regulated or internal data	Self-hosted	The boundary rule outranks every price argument
Public data, throwaway output, low error cost	Free hosted	Nothing downstream depends on the result
Public data, output gets merged, medium or high error cost	Paid hosted plus human review	You are buying accountability, not tokens
Interactive loop under ten seconds	Paid hosted	Replace this row with your own measured p95
Deterministic verification, no secrets in the repo	Free server	Disposable workspace, repeatable result
Credentials in tracked files	Self-hosted	Build logs can echo environment state

Limitations and Who Should Skip This

The thresholds in my policy are placeholders calibrated against my own task mix, so yours will differ and probably by a wide margin. The router also cannot see your repository, which means the secret preflight stays a separate step that a human has to remember.

Free-tier availability, quotas, and free server terms are operator-supplied claims here rather than promises, so I keep a manual override for the days when a tier misbehaves. Self-hosting is not free either; it trades a bill for maintenance time, patching, and someone who gets paged at an inconvenient hour.

Who should skip this? If you run one task type against one repository, the ledger is ceremony, and a single documented default will serve you better. If your team has no review process, routing will not save you, because the fourth axis is unknowable until someone is accountable for misses. And if a compliance review of hosted egress has not finished, stay self-hosted and revisit the table later.

What I Would Do Next

Start by labeling ten real tasks from last week against the four axes, then route them by hand before you write any code. You will probably find that two rules cover most of your work and that the free tier is bigger than you expected. If you want a hosted free tier to pressure-test that third route against your own task list, that is a cheap first stop before committing budget anywhere else.