DEV Community: Ravi Rohith Savaram

Architects, Not Code Writers: Why System Design Matters More in the Age of AI

Ravi Rohith Savaram — Fri, 14 Aug 2026 05:24:16 +0000

How token economics make code structure a cost, speed, and correctness problem — not just a style one.

If you're a software engineer working with AI coding agents, your job has fundamentally changed. You're no longer the person writing most of the diffs. You're the person designing systems that agents operate through — and how well you design those systems has measurable, compounding consequences.

This isn't an abstract argument about clean code being "nice to have." Token economics turn code structure into a cost, speed, and correctness problem with real numbers attached to it.

The Shift

Before AI agents, structure was a personal habit. Some teams enforced it, most let it slide. The code worked either way.

Now, agents write, edit, and reason across your codebase continuously. Every time an agent touches your repo, it follows the same cycle:

Read — Pull in files, directory structure, dependencies, past context.
Reason — Hold it all in the context window while planning the change.
Write — Generate an edit or new code, often re-stating surrounding code.
Verify — Re-read to check the change, sometimes across several turns.

Every one of these steps consumes tokens. A single coding task can loop through this cycle many times before it's done. And every token has a price — not just in dollars, but in real GPU compute, latency, and accuracy.

What a Token Actually Is

A token is the chunk of text a model reads or writes at a time — roughly 4 characters of English prose.

But here's the thing: code tokenizes worse than prose.

Symbols, punctuation, and indentation all cost tokens that carry little semantic meaning on their own. Long identifiers, boilerplate, and repeated imports inflate the count fast. calculateShippingCostForOrder burns roughly 8 tokens just sitting there as a function name — before it does anything. Meanwhile, fn is a single token but tells neither the agent nor the next human reader anything useful.

Verbose or duplicated code is literally more expensive to read and write. Not metaphorically — literally.

The Context Window Compounds

This is where it gets expensive. In an agentic loop, most of the context from turn 1 gets re-sent on turn 2, turn 3, turn 4. The context doesn't just add linearly — it compounds.

Turn	Approximate Context
1	~14K tokens
2	~22K tokens
3	~31K tokens
4	~40K tokens

The same file gets paid for again and again. If that file is 2,400 lines when the agent only needs 90, you're paying for the other 2,310 lines on every single turn.

Same Edit, Two Very Different Blast Radii

Let's make this concrete. Task: "fix a rounding bug in checkout pricing."

The monolith approach — a single orders.py at 2,400 lines:

# orders.py — 2,400 lines
def calculate_shipping(...): ...
def apply_discount(...): ...
def validate_inventory(...): ...
def send_email_receipt(...): ...
def log_analytics(...): ...
def checkout(cart, user):
    total = round(cart.sum * 1.0725)  # <- bug here
    # … 40 more unrelated functions

The agent reads ~9,600 tokens to make a one-line fix safely, because the whole file is one unit.

The modular approach:

checkout/
├── cart.py        (140 lines)
├── pricing.py      (90 lines)  ← bug lives here
└── checkout.py    (110 lines)

# pricing.py
def apply_tax(subtotal):
    return round(subtotal * 1.0725)  # <- fix this

Bug lives in one 90-line file. Agent reads ~1,400 tokens. Nearly 7x cheaper for the same fix.

Caching Rewards Stability

Modern model APIs offer prompt caching: reused context can be read back at roughly 90% off a fresh read. But caching only pays off when the same context is genuinely reusable turn to turn.

A 90-line pricing.py is stable and cacheable. The 2,400-line god-file that half-changes every turn? It invalidates its own cache constantly. Structure decides whether this discount is even available to you.

In the modular case, that same 5-turn session on pricing.py runs roughly 3.5x cheaper — for free, just by not re-explaining the file to the model every turn.

The Real Compute Underneath

The dollar figure is a proxy. Self-attention — the mechanism models use to relate every token to every other — gets more expensive faster than the token count grows.

1x context → ~1x compute
2x context → ~4x compute
4x context → ~16x compute

This isn't linear. Doubling the context quadruples the compute. That means bigger context adds real latency to every turn — and real GPU-hours that somebody is paying for.

Bloated Context Doesn't Just Cost More — It Works Worse

Research testing 18 frontier models found that accuracy degrades as input length grows, often well before the context window is even full. The pattern is consistent across every model tested:

Info at the start: ~90% accuracy
Info in the middle: ~58% accuracy
Info at the end: ~87% accuracy

For a coding agent, this is a third lever alongside cost and compute. A bloated file doesn't just cost more to read — the agent is measurably more likely to miss or misuse the one relevant function buried in the middle of it.

Duplicated Logic Multiplies Every Future Read

Task: "tighten email validation rules." The same check exists in 5 files.

Copy-pasted across the codebase:

# orders.py, billing.py, signup.py, support.py, admin.py — all contain:
return '@' in addr and '.' in addr

Fixing the rule means finding and editing 5 places — 5x the tokens, 5x the chance one gets missed.

Shared through a single module:

# validators.py
def validate_email(addr):
    return '@' in addr and '.' in addr

Fix the rule once. Every call site is correct without being touched or re-read. This isn't new advice — DRY has been a principle for decades. What's new is that duplication now has a measurable per-invocation cost every time an agent traverses your codebase.

The Compounding Loop

Here's the part that should make you uncomfortable: AI-written code builds on top of what's already there. Every change an agent makes becomes the context the next change is read against. Structure is self-reinforcing in both directions.

Virtuous cycle: Clean, modular code → agent reads only the relevant piece → small, well-scoped edit that fits the existing pattern → next task starts cheaper. The codebase keeps paying dividends.

Vicious cycle: Tangled, sprawling code → agent pulls in far more than necessary to be safe → bolted-on edit that makes the pattern messier → next task starts more expensive and more error-prone than the last.

A Good Signature Can Save an Entire File Read

Task: "call this from the new refund flow." Can the agent trust the function signature, or does it have to read the entire body?

Unclear:

def do_stuff(a, b, c=None):
    # ~40 lines of logic
    # no types, no docstring
    ...

The agent must open and read the full body to know what this does. ~150 extra tokens just to trust one call.

Self-describing:

def apply_discount(
    cart_total: float,
    discount_pct: float,
    *,
    cap: float | None = None,
) -> float:
    """Applies a capped percentage discount."""

Signature plus docstring is often enough. Body stays unread.

"But Repo Maps and Retrieval Fix This"

Fair pushback. Modern coding agents increasingly use repo maps, embeddings-based search, and codebase indexing to fetch only what looks relevant — instead of reading a whole file blindly every time.

But retrieval quality depends on structure too. Clear boundaries and names make it easy for a retrieval system to identify what's relevant. Tangled code with unclear boundaries confuses automated retrieval the same way it confuses a person skimming quickly.

Better tooling raises the floor for everyone. But it performs best on exactly the codebases that are already well-structured. These tools shrink the gap. They don't erase it.

Break the Vicious Cycle

If a messy codebase makes every future agent task more expensive, the fix isn't to stop using AI on it. It's to point AI at the mess itself.

Refactor as you go. When a feature touches messy code, clean the touched area before extending it. Don't bolt on.
Budget for it explicitly. Treat structural cleanup as a normal line item in agent usage, not a "someday" project.
Let agents propose the boundary. Ask for a modularization plan before the feature, then implement against that plan.

What Pays Off

These aren't new principles. What's new is that each one now has a measurable impact on every agent interaction:

Modular boundaries — clear seams limit how much an agent must read to change one thing safely.
Strong naming and interfaces — self-describing code reduces how much surrounding context is needed to reason correctly.
Docs and READMEs as context — written for humans, but now also the cheapest way to orient an agent fast.
Tests as guardrails — let an agent verify itself against tests instead of re-reading the whole surrounding system.
No god-files — a 5,000-line file forces an all-or-nothing read; small files let an agent scope precisely.
Explicit dependencies — hidden coupling is invisible to an agent until it breaks something. Make it visible.

The Real Takeaway

"Architect" isn't a metaphor here. Your highest-leverage work is designing a system that stays cheap, fast, and correct for every agent that touches it next — human or otherwise.

Every read and write is billed, in dollars and in real compute. Structure compounds: clean code keeps a task cheap for the next task, messy code makes every future one worse. And the choices that move the needle aren't grand architectural rewrites — they're small, measurable decisions. A file split. A clear signature. A shared module instead of a copy-paste.

These are things you can estimate in tokens before you ship them. That's the new game.

Identity Infrastructure at Scale: The Hardest Problem Nobody Sees

Ravi Rohith Savaram — Mon, 10 Aug 2026 08:49:46 +0000

Every engineer has a mental model of what "hard problems" look like in large-scale systems. They think of recommendation engines, real-time video encoding, or distributed databases that span continents. Identity infrastructure rarely makes the list. And that's exactly why it's the hardest problem in the building — it's invisible until it breaks, and when it breaks, everything breaks.

I've built identity systems that serve billions of users across multiple products. Along the way I've learned that identity isn't a service you bolt on. It's the connective tissue of your entire platform. Get it right and nobody notices. Get it wrong and you're looking at cascading auth failures, account takeovers at scale, and the kind of trust erosion that takes years to repair.

This post is a practitioner's account of what actually works, what doesn't, and what I wish someone had told me when I started.

Why Identity Is Structurally Different from Other Infrastructure

Most backend systems can tolerate brief inconsistency. A social feed that's two seconds stale is fine. A search index that hasn't caught the last write is acceptable. Identity doesn't get that grace period.

When a user changes their password, revokes a session, or enables two-factor authentication, that change must propagate instantly and universally. If there's a ten-second window where an old session token still works after a password reset, you've created a ten-second window for an attacker who already has that token. Identity is one of the few domains where consistency isn't a nice-to-have — it's a security boundary.

At the same time, identity sits in the critical path of every single request. Every API call, every page load, every real-time message begins with "who is this user, and are they allowed to do this?" If your identity layer goes down, your entire platform goes down. That means you also need extreme availability.

This is the fundamental tension: you need strong consistency and extreme availability, which anyone who's read the CAP theorem knows is a painful place to live. The real engineering isn't choosing one over the other. It's designing around the tension so that the tradeoffs are invisible to users and safe for the platform.

The Core Design Challenges

Federated Identity Across Products

Single-product companies have it easy. One user table, one auth flow, one session model. But the moment you operate a family of products — each with its own history, its own user base, its own auth requirements — identity becomes a federation problem.

The naive approach is to build a central identity service and migrate everyone onto it. I've watched this approach fail more than once. Products have different trust levels, different session lifetimes, different regulatory environments. A messaging product handles identity differently than an e-commerce platform or a streaming service.

What works is a layered identity model: a core identity graph that establishes who a person is, with product-specific identity projections that handle how that person is represented and authenticated in each context. The core layer owns the canonical user record, credential storage, and cross-product linkage. The projection layer owns product-specific sessions, scopes, and consent surfaces.

This separation sounds clean on a whiteboard. In practice, the hard part is the boundary. Where does the core layer end and the product layer begin? The answer shifts over time, and you need to design for that migration path, not just the current state.

Token Lifecycle at Billions Scale

A typical large-scale platform might have tens of billions of active tokens at any given moment — access tokens, refresh tokens, device tokens, API tokens, OAuth grants. Each has its own lifetime, revocation semantics, and blast radius if compromised.

The mistakes I see most often:

Over-reliance on long-lived tokens. Engineers default to long expiration times because short-lived tokens mean more refresh traffic. But long-lived tokens are long-lived attack surfaces. The right pattern is short-lived access tokens (minutes, not hours) backed by longer-lived refresh tokens with rotation on every use. If a refresh token is used twice, you know it's been stolen — kill the entire token family.

Revocation as an afterthought. It's easy to issue tokens. It's incredibly hard to revoke them instantly across a globally distributed system. Maintaining a global revocation list that every service checks on every request doesn't scale. What does scale is a combination of short token lifetimes (so most revocations are "wait for expiry"), a lightweight real-time revocation channel for critical events (password changes, account compromises), and epoch-based invalidation where you can bump a user's "auth epoch" and instantly invalidate everything issued before it.

Treating all tokens as equal. A token that lets you read your own profile and a token that lets you modify account security settings should not have the same architecture. Step-up authentication — requiring a fresh, stronger auth challenge before high-risk operations — is essential. But it needs to be baked into the token model, not layered on top.

Zero Trust in a Multi-Service World

The old model was simple: authenticate at the edge, then trust everything inside the network perimeter. That model is dead, and it should be. Any system with hundreds of internal services has too large an internal attack surface to trust implicitly.

In practice, zero trust for identity means every service-to-service call carries user context and is independently verified. The user's identity and their authorization scopes travel with the request, and each service makes its own authorization decision based on that context.

The engineering challenge is doing this without destroying latency. If every service makes a fresh call to your identity service on every request, you've built a system where your identity service's p99 latency is your platform's p99 latency. The solution is cryptographic verification at the edge: signed, self-contained tokens (like JWTs, but carefully — see below) that services can verify locally without a network call, combined with an asynchronous revocation channel for the cases where you need to pull the rug out.

A word of caution on JWTs specifically: they've become the default answer, but they're not a free lunch. They can't be revoked without infrastructure to support it. They bloat quickly when you stuff too many claims in. And the ecosystem of JWT libraries has a terrible security track record — algorithm confusion attacks, none-algorithm acceptance, key confusion between signing and encryption. If you use JWTs, treat the library choice and configuration as a security-critical decision, not a dependency you pull in and forget about.

Patterns That Work (and Ones That Break)

Hierarchical identity models work. Rather than a flat user → session mapping, model identity as a tree: person → accounts → sessions → tokens. This gives you natural revocation cascades (disable an account, all its sessions die; kill a session, all its tokens die) and clean multi-account support.

Progressive authentication works. Not every action requires the same level of assurance. Let users browse with a session cookie, but require a fresh password or biometric before they change their email or download their data. This reduces friction for low-risk flows while maintaining security for high-risk ones. The key is building this into your identity model from the start — retrofitting progressive auth onto a system that treats all sessions as equal is painful.

Global session management works, eventually. Letting a user see and revoke all their active sessions across all devices sounds simple. Building it at scale, across multiple products, with real-time accuracy, is a multi-year infrastructure project. But it's table stakes for user trust and regulatory compliance (GDPR's right to withdrawal of consent, for instance). Start early.

What breaks: monolithic auth services. I've seen teams build a single auth service that handles registration, login, session management, OAuth, SAML, passwordless auth, and fraud detection. It becomes the most critical, most complex, and most feared service in the entire stack. No one wants to deploy to it. No one fully understands it. Break it up. Separate the credential verification path from the session management path from the token issuance path. They have different scaling characteristics, different failure modes, and different change velocities.

What breaks: treating identity migration as a one-time project. Identity infrastructure is never done. Credential standards change (passwords → OTP → push notifications → passkeys). Regulatory requirements shift. New products join the platform. Design for continuous migration, not a big bang cutover.

What Most Engineers Get Wrong

The biggest mistake is treating identity as an application feature rather than a platform primitive. When identity is owned by a product team, you get auth flows that are optimized for that product's onboarding funnel but impossible to extend to new products. When identity is a platform, you get consistent security properties everywhere, even in products that haven't been built yet.

The second mistake is underinvesting in the unglamorous parts. Credential rotation, token hygiene, session cleanup, audit logging — these aren't exciting. They're also the things that determine whether you survive a security incident or end up in the news.

The third is ignoring the human side. Identity systems are ultimately about people, and people do unpredictable things. They share accounts. They forget passwords on devices they no longer own. They pass away, and someone needs to access their account. They're minors who age into adulthood and need their data handling to change. Every identity system I've built has eventually needed to handle cases that no spec anticipated, and the systems that handled them gracefully were the ones designed with extensibility and human judgment baked in.

Where Identity Infrastructure Is Heading

Passkeys are the most significant shift in consumer authentication in a decade. They eliminate passwords entirely, replacing them with device-bound cryptographic credentials that are phishing-resistant by design. The migration is messy — you need to support passwords and passkeys simultaneously for years — but the security improvement is step-function, not incremental. If you're building identity infrastructure today and you're not planning for passkeys, you're building for the past.

Decentralized identity (DIDs, verifiable credentials) is interesting but not ready for mainstream consumer platforms. The standards are maturing, but the user experience is still poor, and the trust model requires users to manage cryptographic material, which history says they won't do reliably. Watch this space, but don't bet your architecture on it yet.

AI-driven adaptive authentication is already here in practice, even if it's not always labeled that way. Risk-based authentication — adjusting the auth challenge based on device fingerprint, location, behavioral signals — has been standard at scale for years. What's changing is the sophistication of the signals and the models. Expect authentication to become increasingly invisible for legitimate users and increasingly hostile for attackers, with the identity system making real-time judgments about how much friction to introduce.

The constant across all of these trends is that identity infrastructure keeps getting more complex, more critical, and more underappreciated. If you're an engineer working in this space, know that what you're building is the foundation everything else stands on. And if you're not in this space, the next time you log in seamlessly across three products on two devices without thinking about it — someone built that, and it was harder than it looked.