Architects, Not Code Writers: Why System Design Matters More in the Age of AI

How token economics make code structure a cost, speed, and correctness problem — not just a style one.

---

If you're a software engineer working with AI coding agents, your job has fundamentally changed. You're no longer the person writing most of the diffs. You're the person designing systems that agents operate through — and how well you design those systems has measurable, compounding consequences.

This isn't an abstract argument about clean code being "nice to have." Token economics turn code structure into a cost, speed, and correctness problem with real numbers attached to it.

The Shift

Before AI agents, structure was a personal habit. Some teams enforced it, most let it slide. The code worked either way.

Now, agents write, edit, and reason across your codebase continuously. Every time an agent touches your repo, it follows the same cycle:

1. Read — Pull in files, directory structure, dependencies, past context.
2. Reason — Hold it all in the context window while planning the change.
3. Write — Generate an edit or new code, often re-stating surrounding code.
4. Verify — Re-read to check the change, sometimes across several turns.

Every one of these steps consumes tokens. A single coding task can loop through this cycle many times before it's done. And every token has a price — not just in dollars, but in real GPU compute, latency, and accuracy.

What a Token Actually Is

A token is the chunk of text a model reads or writes at a time — roughly 4 characters of English prose.

But here's the thing: code tokenizes worse than prose.

Symbols, punctuation, and indentation all cost tokens that carry little semantic meaning on their own. Long identifiers, boilerplate, and repeated imports inflate the count fast. calculateShippingCostForOrder burns roughly 8 tokens just sitting there as a function name — before it does anything. Meanwhile, fn is a single token but tells neither the agent nor the next human reader anything useful.

Verbose or duplicated code is literally more expensive to read and write. Not metaphorically — literally.

The Context Window Compounds

This is where it gets expensive. In an agentic loop, most of the context from turn 1 gets re-sent on turn 2, turn 3, turn 4. The context doesn't just add linearly — it compounds.

Turn	Approximate Context
1	~14K tokens
2	~22K tokens
3	~31K tokens
4	~40K tokens

The same file gets paid for again and again. If that file is 2,400 lines when the agent only needs 90, you're paying for the other 2,310 lines on every single turn.

Same Edit, Two Very Different Blast Radii

Let's make this concrete. Task: "fix a rounding bug in checkout pricing."

The monolith approach — a single orders.py at 2,400 lines:

# orders.py — 2,400 lines
def calculate_shipping(...): ...
def apply_discount(...): ...
def validate_inventory(...): ...
def send_email_receipt(...): ...
def log_analytics(...): ...
def checkout(cart, user):
    total = round(cart.sum * 1.0725)  # <- bug here
    # … 40 more unrelated functions

The agent reads ~9,600 tokens to make a one-line fix safely, because the whole file is one unit.

The modular approach:

checkout/
├── cart.py        (140 lines)
├── pricing.py      (90 lines)  ← bug lives here
└── checkout.py    (110 lines)

# pricing.py
def apply_tax(subtotal):
    return round(subtotal * 1.0725)  # <- fix this

Bug lives in one 90-line file. Agent reads ~1,400 tokens. Nearly 7x cheaper for the same fix.

Caching Rewards Stability

Modern model APIs offer prompt caching: reused context can be read back at roughly 90% off a fresh read. But caching only pays off when the same context is genuinely reusable turn to turn.

A 90-line pricing.py is stable and cacheable. The 2,400-line god-file that half-changes every turn? It invalidates its own cache constantly. Structure decides whether this discount is even available to you.

In the modular case, that same 5-turn session on pricing.py runs roughly 3.5x cheaper — for free, just by not re-explaining the file to the model every turn.

The Real Compute Underneath

The dollar figure is a proxy. Self-attention — the mechanism models use to relate every token to every other — gets more expensive faster than the token count grows.

• 1x context → ~1x compute
• 2x context → ~4x compute
• 4x context → ~16x compute

This isn't linear. Doubling the context quadruples the compute. That means bigger context adds real latency to every turn — and real GPU-hours that somebody is paying for.

Bloated Context Doesn't Just Cost More — It Works Worse

Research testing 18 frontier models found that accuracy degrades as input length grows, often well before the context window is even full. The pattern is consistent across every model tested:

• Info at the start: ~90% accuracy
• Info in the middle: ~58% accuracy
• Info at the end: ~87% accuracy

For a coding agent, this is a third lever alongside cost and compute. A bloated file doesn't just cost more to read — the agent is measurably more likely to miss or misuse the one relevant function buried in the middle of it.

Duplicated Logic Multiplies Every Future Read

Task: "tighten email validation rules." The same check exists in 5 files.

Copy-pasted across the codebase:

# orders.py, billing.py, signup.py, support.py, admin.py — all contain:
return '@' in addr and '.' in addr

Fixing the rule means finding and editing 5 places — 5x the tokens, 5x the chance one gets missed.

Shared through a single module:

# validators.py
def validate_email(addr):
    return '@' in addr and '.' in addr

Fix the rule once. Every call site is correct without being touched or re-read. This isn't new advice — DRY has been a principle for decades. What's new is that duplication now has a measurable per-invocation cost every time an agent traverses your codebase.

The Compounding Loop

Here's the part that should make you uncomfortable: AI-written code builds on top of what's already there. Every change an agent makes becomes the context the next change is read against. Structure is self-reinforcing in both directions.

Virtuous cycle: Clean, modular code → agent reads only the relevant piece → small, well-scoped edit that fits the existing pattern → next task starts cheaper. The codebase keeps paying dividends.

Vicious cycle: Tangled, sprawling code → agent pulls in far more than necessary to be safe → bolted-on edit that makes the pattern messier → next task starts more expensive and more error-prone than the last.

A Good Signature Can Save an Entire File Read

Task: "call this from the new refund flow." Can the agent trust the function signature, or does it have to read the entire body?

Unclear:

def do_stuff(a, b, c=None):
    # ~40 lines of logic
    # no types, no docstring
    ...

The agent must open and read the full body to know what this does. ~150 extra tokens just to trust one call.

Self-describing:

def apply_discount(
    cart_total: float,
    discount_pct: float,
    *,
    cap: float | None = None,
) -> float:
    """Applies a capped percentage discount."""

Signature plus docstring is often enough. Body stays unread.

"But Repo Maps and Retrieval Fix This"

Fair pushback. Modern coding agents increasingly use repo maps, embeddings-based search, and codebase indexing to fetch only what looks relevant — instead of reading a whole file blindly every time.

But retrieval quality depends on structure too. Clear boundaries and names make it easy for a retrieval system to identify what's relevant. Tangled code with unclear boundaries confuses automated retrieval the same way it confuses a person skimming quickly.

Better tooling raises the floor for everyone. But it performs best on exactly the codebases that are already well-structured. These tools shrink the gap. They don't erase it.

Break the Vicious Cycle

If a messy codebase makes every future agent task more expensive, the fix isn't to stop using AI on it. It's to point AI at the mess itself.

• Refactor as you go. When a feature touches messy code, clean the touched area before extending it. Don't bolt on.
• Budget for it explicitly. Treat structural cleanup as a normal line item in agent usage, not a "someday" project.
• Let agents propose the boundary. Ask for a modularization plan before the feature, then implement against that plan.

What Pays Off

These aren't new principles. What's new is that each one now has a measurable impact on every agent interaction:

• Modular boundaries — clear seams limit how much an agent must read to change one thing safely.
• Strong naming and interfaces — self-describing code reduces how much surrounding context is needed to reason correctly.
• Docs and READMEs as context — written for humans, but now also the cheapest way to orient an agent fast.
• Tests as guardrails — let an agent verify itself against tests instead of re-reading the whole surrounding system.
• No god-files — a 5,000-line file forces an all-or-nothing read; small files let an agent scope precisely.
• Explicit dependencies — hidden coupling is invisible to an agent until it breaks something. Make it visible.

The Real Takeaway

"Architect" isn't a metaphor here. Your highest-leverage work is designing a system that stays cheap, fast, and correct for every agent that touches it next — human or otherwise.

Every read and write is billed, in dollars and in real compute. Structure compounds: clean code keeps a task cheap for the next task, messy code makes every future one worse. And the choices that move the needle aren't grand architectural rewrites — they're small, measurable decisions. A file split. A clear signature. A shared module instead of a copy-paste.

These are things you can estimate in tokens before you ship them. That's the new game.