DEV Community: Anusha Mukka

"Don't Learn to Code" Is the Worst Career Advice of 2026

Anusha Mukka — Sat, 13 Jun 2026 03:50:12 +0000

Everyone's debating whether coding is dead. I actually do this job.. with AI writing code beside me for most of my working hours. Here's what the headlines get wrong.

Open your feed right now and you'll find the same headline in a dozen costumes:

"Why AI will replace 80% of software engineers by 2026."
"Is coding dead?"
"Should you still learn to code?"

It's the most-clicked anxiety in tech, and it's everywhere for a reason, it taps a real fear about real careers. But here's the thing about almost every one of those posts: they're written from the sidelines. Predictions about a job by people who don't do it.

I'm writing this from the other side. I'm an engineer, and I drive AI coding agents every single day. They read code, write changes, run tests, and open reviews for most of my working hours. So when someone asks "should you still learn to code in 2026?", I'm not guessing.

Here's my honest answer: Yes. Absolutely. But the job you're learning for has quietly become a different job and almost nobody is telling you which one.

The hype isn't entirely wrong

Let me start by giving the doomers their due, because pretending the shift isn't real would make me exactly the kind of person I'm criticizing.

The productivity jump is genuine, and it's not subtle. Industry surveys in 2026 put the share of new code that's AI-assisted somewhere north of 40%, and developers using these tools self-report double-digit speedups on routine work. That matches my experience. The agent now handles:

Boilerplate and glue code —-> the stuff I used to type on autopilot, gone in seconds.
First drafts —-> "scaffold something that does X" gets me 80% of the way instantly.
Syntax recall —-> I stopped breaking focus to look up things I half-remember.
Tedious refactors —-> rename-this-everywhere, migrate-this-pattern, done fast. and all the kludgy things that I dread to do.

If your mental image of "coding" is typing syntax into an editor, then yes.. a big chunk of that is being automated. The viral posts are right about that part.

They're just wrong about what it means.

What AI hasn't touched and probably won't soon

Here's what you only learn by actually using these tools all day, the part that never makes it into the scary headline:

Knowing what to build. The agent will cheerfully build the wrong thing, beautifully and quickly. Deciding what is worth building..and what isn't.. is the actual job. The model has no stake in the outcome.

Judgment and taste. Is this the right abstraction? Will it survive contact with scale? Is this the simple solution or the clever one that quietly wrecks us in six months? AI produces an answer. It does not produce an opinion I'd trust unsupervised.

Debugging the genuinely weird. When something breaks for a non-obvious reason, a race condition, a subtle interaction between two systems — the agent flails. You need a human who understands what's underneath.

Verification. This is the big one. AI generates plausible code fast, and plausible-but-wrong is the most expensive kind of wrong there is. Someone has to read every line, understand it, and catch the bug that looks fine. That someone has to know how to code deeply.

Notice the pattern: everything AI didn't replace requires you to truly understand code. You cannot direct, verify, or debug what you can't read. The tool didn't remove the need for expertise. It moved it.

The shift nobody puts in the headline

My job didn't disappear. Its center of gravity moved.

I spend less time writing code and more time reading, reviewing, and directing it. In fact, I did not write a single line of code in months..So, the skills that are appreciating in 2026 look like this:

Reading code fast and critically —-> because you're now reviewing a firehose of machine-generated output.
Context engineering —-> giving the agent the right constraints, examples, and guardrails. "Prompting" is the toy version of this. The real skill is engineering the conditions for good output.
System thinking --> architecture, tradeoffs, knowing where the bodies are buried.
Verification instinct —-> smelling the bug before the test suite finds it.

None of that is less code knowledge. It's more. The bar for "I pasted things until it worked" went up. The bar for "I understand systems" went up too. AI didn't lower the ceiling, it raised the floor and the ceiling at the same time.

So should you learn to code?

Yes! But learn it for the 2026 job, not the November 29th of 2022 one(given what happened on the 30th):

Learn fundamentals deeply, not just syntax. Data structures, how systems fit together, why one design beats another. AI gives you syntax for free. It cannot give you judgment.
Learn to read code, not just write it. Practice reviewing. Read open-source pull requests. This is the single most underrated, fastest-appreciating skill right now.
Use the agents as a sparring partner, not a crutch. Let them draft; you decide and verify. You'll learn faster and build the exact instinct that's becoming valuable.
Get precise about what you want. Specs, constraints, examples. The people who can direct an agent clearly are pulling ahead of the people who can only type.

The bottom line

"Don't bother learning to code" is the worst career advice of 2026.

AI didn't kill programming. It commoditized the typing and put a premium on the thinking. The people who win in this era are the ones who understand code deeply enough to direct, verify, and correct a machine that is confidently wrong a meaningful fraction of the time.

You still need to learn to code. You just get to skip the boring parts now and spend your energy on the part that was always the real job.

This is my take on the engineering in the age of AI agents. If this resonated, or if you think I'm dead wrong.. I'd genuinely like to hear it. What has AI actually replaced in your work, and what hasn't? Drop it in the comments.

The Illusion of Scale, Part 5: The System That Outlives the Team

Anusha Mukka — Sat, 06 Jun 2026 00:04:34 +0000

A few years ago I built an electronic search warrant system for a state law enforcement agency. Paper process, courthouse logistics, hours of waiting -- we turned it into two minutes. Designed it, built it, deployed it, handed it off.

That system is still running. Eight years later. Extended to handle warrant types that didn't exist when we built it. The original team moved on years ago(most of them retired). The requirements document is probably in a folder nobody opens anymore.

Still works.

I've thought about this a lot. Why did that system survive when so many others didn't? I've built things that were arguably more technically interesting that got rewritten within two years(in move fast and break things environment). What was different?

This is the final post in a series about assumptions that quietly break systems at scale. But this one's about something different: what makes a system last.

The second write (or: the time I broke production by being too smart)

Before I talk about what we did right, I want to talk about something I got wrong somewhere else. Because I think the mistake is more instructive.

I inherited a codebase that had a pattern that looked like a bug. A service was writing to two tables in a way that appeared redundant. The second write looked like a copy of data that already existed elsewhere. Dead code, clearly. Technical debt from someone who didn't clean up after themselves.

I removed it. Tests passed. I moved on feeling like I'd done something useful. Feeling productive.

A week later we had a data consistency incident in exactly the scenario the second write had been protecting against. No documentation explaining it. No comment in the code. No ADR anywhere. The engineer who'd written it had left the company.

The second write was load-bearing and completely invisible. And I'd ripped it out because I was clever enough to see it was "redundant" but not wise enough to wonder why it was there.

The code told me what the system did. It said nothing about why it did it that way. At scale, the gap between those two things is where SEVs live.

The question that changes how you build

Most engineering teams design systems for themselves. The decisions about structure, naming, and abstraction are implicitly optimized for the team that currently exists, because that's the team building it and living with it right now.

The problem is: that team won't be there forever. People leave. Priorities shift. The team that inherits your system doesn't have your context, can't ask you questions, and has to reconstruct your intent from whatever you left behind.

Usually that's not much.

Systems that survive long enough to matter ask a different question during design: not "does this make sense to us?" but "will this make sense to someone who wasn't here?"

That one shift changes specific things. How you name things. How much you externalize versus bake in. Whether your runbooks are written for the team that built the system or for an engineer encountering it cold at 2am with an incident open and nobody to call.

Configurable beats clever (and it's not even close)

The search warrant system was designed to be configurable almost to a fault. New document type? Configuration, not code. New approval chain? Configuration. New workflow state? Configuration. We went kind of overboard with it, honestly.

The argument wasn't technical sophistication. It was a simple belief: requirements will keep changing and we won't always be there to change the code.

Eight years later and no rewrite. That belief held.

Here's the thing about clever code: it achieves things concisely in ways that are satisfying to write. I enjoy writing clever code. It makes me feel smart. But it's also hard to understand six months later when you've forgotten the context, and nearly impossible for someone who was never there at all.

Configurable code is boring to write. It's verbose. It's repetitive sometimes. It's what you find in systems that are still running years after the team that built them moved on.

The team that inherits a clever system has to reverse-engineer intent. The team that inherits a configurable system can mostly just read it. I know which one I'd rather inherit at 2am.

The documentation that actually survives

Engineers intend to write documentation. The road to technical hell is paved with good documentation intentions. It either doesn't happen, or it gets written once and reflects the system as designed rather than the system as deployed. Six months later it's wrong and nobody updates it because they're not sure what the current accurate version should say.

The documentation that actually survives and stays useful isn't a comprehensive spec. It's the decision log.

What problems did the original team hit? What did they consider and reject? Why did they make the choices they made? What tradeoffs did they accept on purpose?

This information lives in people's heads. When those people leave, it leaves with them. The inheriting team doesn't know why the system works the way it does. They only know that it does. And they will "fix" things that were intentional and reintroduce bugs that were already solved, usually at the worst possible time.

An architecture decision record doesn't have to be formal. A short document: the context, the options considered, the decision made, the tradeoffs accepted. Written at the time the decision was made, when context is fresh. Stored somewhere the next team will actually find it.

One ADR prevented two separate teams from making the same mistake on one of my systems over a span of three years. That one document was worth more than any other single artifact in the codebase. And if I'd written one for the system with the second write, a week of incidents could have been a five-minute read.

Observability is a gift to your future strangers

When something goes wrong in a system you built, you know where to look. You know which metrics matter, which log lines carry signal, which alerts mean something real versus which ones you can ignore.

When something goes wrong in a system you inherited, you start from zero. Every metric might matter. Every log line might be the one. You have no instinct for it yet.

Systems that outlive their teams are systems that explain themselves. Meaningful metrics with names that tell you what they mean. Structured logs with enough context to reconstruct what happened. Trace IDs that follow a request through every component it touches.

These aren't features. They're the difference between a system that can be operated by someone new and one that can only be operated by the person who built it. Which is fine until that person is no longer there. And that person will always, eventually, no longer be there.

What this series has really been about

Each post in this series has been about a different version of the same problem: an assumption that was harmless when the system was small and expensive when it grew. The data model that encoded a belief about cardinality. The permission model that assumed roles would stay simple. The latency budget that was validated in isolation and wrong under real load.

The systems that survive are not usually the most architecturally impressive ones. They're the ones where someone spent time on the things that don't show up in demos. Configuration over hardcoding. Decision logs over implicit knowledge. Observability over optimism. Simplicity over cleverness, everywhere it was possible to choose.

The search warrant system is still running because the team that built it made boring decisions on purpose. Nothing in it is clever. Everything in it is as simple as we could make it while still meeting the requirements. We spent complexity only where we genuinely had to.

That's the thing hyperscale teaches you, eventually: the goal is not to build impressive systems. The goal is to build systems that keep working when you're not there anymore.

That's a wrap on "The Illusion of Scale" five posts, five assumptions, one intent. Thank you for following along!

Happy building!

The Illusion of Scale, Part 4: Latency Is a Design Decision, Not a Measurement

Anusha Mukka — Sun, 31 May 2026 19:42:31 +0000

I need to tell you about the time I confidently presented a latency budget to a stakeholder and then watched it disintegrate in production like wet tissue paper.

We had a system with a 200ms latency budget. We'd measured every component. Auth service: 15ms. Business logic: 30ms. Database query: 40ms. Total: well inside budget. I remember feeling good about this. We'd done the work. We had numbers.

We shipped it. In production, the auth call that took 15ms in testing was regularly hitting 200ms at peak. So I panicked, ran profiling tools, even on-boarded new profiling tools coz LLM agents did not exist back then and I was desperate and sure that a block of code was causing it, maybe writing to DB, reading from DB, calling an API, something.. something that would explain this, but there it is.. the bitter truth!

The auth service was slow. Because it was shared with four other services, all of which peaked at the same time, and nobody, I mean nobody, including me -- had reserved any capacity for ours. We had 15ms allocated. We were spending 200ms. The rest of the budget was irrelevant at that point.

That was the moment I stopped treating latency as a measurement exercise and started treating it as a design problem. Measure-and-optimize sounds like engineering rigor. In practice, it's usually "discover your architectural constraints too late to change them cheaply."

This is Part 4 of a series on the assumptions that quietly wreck systems at scale.

You can't optimize your way out of a bad structure

The instinct is totally reasonable. Build the thing, run load tests, measure latency, optimize what's slow. Feels like good engineering discipline. I've given this exact advice to junior engineers.

The problem is: by the time you're measuring in production, the decisions that created the latency are three layers deep in the architecture. Changing them means rewriting things that other things depend on, under load, with users waiting. That's not optimization. That's reconstruction. And it happens at the worst possible time -- when you're already under pressure to deliver and 3 days away from piloting the product.

Load tests seldom catch the real issue either. They model the traffic you imagined. Production brings shared dependencies, concurrent spikes, and usage patterns that your test suite never considered because honestly, why would it? You test what you know. Production teaches you what you didn't(at the most inconvenient time possible).

Where latency actually lives (hint: not where you think)

The obvious suspects -- slow queries, unoptimized loops, API calls with bad timeouts -- those are worth fixing. Sure. But they're usually not the interesting problem.

The interesting latency problems are structural. They're baked into how the system is organized before anyone writes a line of code.

Chattiness. A user-facing request that requires eight internal service calls to complete has a latency floor equal to the sum of those calls. You cannot optimize below that floor. No amount of caching or connection pooling or index tuning changes the fundamental math. You have to redesign the call structure. Which is a very different conversation than "let's optimize the hot path."

Unbounded fanout. A query that touches N records where N is controlled by user input is fine in development, where every test dataset is small and tidy. In production, one legitimate power user has an N that's ten thousand times your assumption, and the query that runs in 20ms for everyone else runs in three minutes for them. And -- I love this part -- they're usually your most important customer. So the conversation about "we need to add limits" becomes a very political discussion very quickly.

Synchronous waits on async work. This is the quietest killer. If your system waits synchronously for something that's fundamentally asynchronous -- a write to propagate, a downstream service to confirm, a cache to warm -- you've put a hard ceiling on your response time. No optimization lifts that ceiling. You have to change the boundary between sync and async, which is one of those decisions I mentioned in Part 1 that's genuinely hard to reverse.

Latency budgets: think before you build, not after

Here's what actually works: decide your latency budget before you build, not after and give yourself some buffer, coz trust me., you are going to need it.

Take your target response time. Allocate it across each component in the critical path. Every component has its own number. Write it down. Put it somewhere people will see it.

What this surfaces immediately: shared dependencies. When two components share a downstream resource, their budgets aren't independent. The budget math that looks fine for each component in isolation falls apart when they both spike at the same time. That's exactly what happened with our auth service. If we'd done this exercise before building, we would have caught that the auth service was shared and had no capacity isolation. We would have had a conversation about it. Maybe we would have made the same choice, but at least it would have been a choice and not a surprise.

Writing the budget down also forces tradeoffs into the open before anyone's committed code. Maybe something expensive moves off the critical path and gets computed asynchronously. Maybe you denormalize data you'd rather not. Those are real conversations worth having before the code exists.

I know this sounds like process for process's sake. It's not. It's the difference between "we chose to accept this tradeoff" and "we discovered this tradeoff during an incident at 2am."

The number that should scare you

10ms of unnecessary latency at 100,000 requests per second is 1,000 seconds of user wait time per second of operation.

Let that sink in for a second. One thousand seconds of wasted human time, every second your system is running.

That's not a performance problem. That's a customer problem. It's why teams at real volume spend weeks on single-digit millisecond improvements and can justify every hour of it. When someone asks "is 10ms really worth optimizing?" the answer depends entirely on your volume. At low traffic, no. At high traffic, it's one of the highest-leverage things you can do.

The conversation I couldn't answer

There was a point where a stakeholder asked us why the system was "sometimes fast and sometimes slow with no obvious pattern." We couldn't answer cleanly. Not because we didn't understand the code -- we did. We just hadn't modeled what the components did to each other under concurrent load.

The answer turned out to be resource contention between two services that looked completely independent on the architecture diagram. They shared a database. Nobody had documented that as a latency dependency. It had just been built that way, probably seemed fine at the time, and nobody had flagged it.

I spent an embarrassing amount of time looking at application code when the problem was infrastructure topology. Once I found it, the fix was straightforward. But the finding took days because I was looking in the wrong places.

After that experience, every shared dependency in a critical path gets an explicit owner and an explicit budget in any system I work on. Not because it's an elegant process. Because the alternative is standing in front of a stakeholder at 2pm on a Tuesday unable to explain why the system is slow in ways you can't describe.

Final post next week: the systems that outlive the teams that built them, and what the ones that survive actually have in common. (Spoiler: it's not architectural cleverness.)

Where did latency surprise you? What was the shared dependency nobody had mapped? I've started keeping a list and it's getting disturbingly long.

The Illusion of Scale, Part 3: Access Control Doesn't Scale Linearly

Anusha Mukka — Tue, 26 May 2026 03:54:19 +0000

One day you look up and realize your permissions model is something only two people on the team can explain. One of them just put in their notice.

Nobody planned to be in that position. It happened one exception at a time. One "just add a role for this" at a time. One "we'll clean this up later" at a time. Later never comes. It never comes.

This is Part 3 of a series about assumptions that quietly break systems at scale.

How 15 roles become 340 (a horror story in slow motion)

When we built out the permission model for one of the systems I worked on, we had 15 roles. Clean, well-defined, each with a clear purpose. You could explain the whole model in ten minutes to anyone new on the team. I was proud of it, honestly.

Two years later there were 340 roles. Three. Hundred. And forty.

Nobody planned for that. Nobody woke up one morning and said "you know what this system needs? 340 roles." It happened like this: a team needed access to one resource but not another, so a new role was created. A contractor role was almost identical to the standard role but needed one extra permission, so another role was created. An emergency access role was supposed to be temporary but was kept "just in case" and never revisited.

Each decision made perfect sense at the time. Collectively they produced a permission model that no single person could fully explain, audit, or reason about confidently. Including me, and I'd been there since the beginning.

That is role explosion. It's not a failure of discipline. It's what happens when a model designed for a clean set of cases gets pushed, one reasonable exception at a time, into a reality more complex than it was designed for.

Why simple RBAC always eventually breaks

Role-based access control works great when access decisions are binary: you either have the role or you don't. Clean, auditable, easy to reason about.

The problem is that real-world access decisions are almost never that clean.

You need a user who can access their own records but not others. You need access that expires after a project ends. You need a decision that depends on the current state of the resource, not just who's asking. Each of these requirements pushes you either toward more roles (which gets unwieldy fast) or toward a richer model that can express context-aware decisions.

Most teams take the path of more roles because it's faster in the moment. I've done this. You've probably done this. The second path -- attribute-based or policy-based access control -- is more work upfront and dramatically less work over time. But "more work upfront" loses to "we need this shipped by Friday" approximately 100% of the time.

The 10-minute incident (or: why caching permissions is terrifying)

Even a well-designed permission model has to be evaluated, and at scale the evaluation cost matters.

The usual answer is caching. Cache the authorization decision with a TTL. Fast, cheap, easy to implement. But during that TTL window, you're making decisions based on permissions that may no longer be current. This is fine. This is a reasonable tradeoff. Until it isn't.

We had a 10-minute TTL on cached permission decisions. The security team had asked what would happen if they needed to revoke access immediately. We said: up to 10 minutes. They accepted that.

Then a credential was compromised.

The security team revoked access and watched the logs. The system kept serving that user's requests for another eight minutes. Eight minutes is not long in most contexts. Standing in front of a security team watching real-time access logs during an active incident, trying to explain why the revocation hasn't taken effect yet, is a very different experience of those eight minutes. How did I eventually got around that problem? Take a guess in the comments.

Anyways..I have never forgotten what that room felt like. I will never set a cache TTL on permissions without thinking about that room.

That tradeoff -- cache TTL versus revocation speed -- exists whether or not your team has discussed it. The only variable is whether you made it consciously or discovered it during an incident.

Audit trails at volume (the compliance conversation from hell)

Every access decision needs to be attributable: who requested it, what they were authorized to do, what decision was made, and why. At 100,000 decisions per second, that's substantial write volume to your audit store.

Synchronous writes add latency. Asynchronous writes mean you have to handle the failure case where a decision is made but the audit entry is lost -- which is a compliance conversation nobody wants to have. I've been in that conversation. It's not fun.

I've worked on systems where the requirement was "log first, then execute." That constraint reshapes your entire architecture -- your latency budget, your failure handling, your storage design. It's buildable, but it needs to be in the design from the start. Retrofitting "log before execute" onto an existing system is expensive and almost never goes cleanly. Ask me how I know.

Granting is easy. Revocation is the real test.

Granting access is trivial. Write a row somewhere. Done. Ship it.

Revocation is where the design quality shows.

Access needs to be revoked across every cache, every replica, every long-running process that may have loaded a stale copy of that permission. A batch job that started before the revocation happened, loaded permissions at startup, and is still running an hour later -- technically, every individual check it made was valid at the time. But the aggregate behavior is wrong.

Explaining that gap to a compliance team is not a conversation you want. "Well, technically, at the time of each individual check..." doesn't land the way you hope it will.

Designing revocation that actually works means deciding explicitly what "immediately" means in your system and then building infrastructure to deliver it. Not assuming it'll sort itself out. It won't sort itself out.

What I'd do differently

Authorize close to the data, not just at the API boundary. Edge authorization is necessary and not sufficient.

Design the hot-path permission check to require no joins. It should be cheap by construction, not by optimization. Optimization after the fact is harder and less reliable than just designing it right.

Treat the cache staleness window as a product decision, not a technical one. Write it down. Make sure the people responsible for security incidents know what it is before the incident happens.

Build the audit trail into the design before anyone writes application code. Retrofitting it under compliance pressure is one of the more unpleasant engineering experiences I can describe. And I've had some unpleasant ones.

Next week: LATENCY - we all have seen the websites go "loading..." before they respond, what was that experience like? Not great, right? So, let's talk about the culprit behind that.

What access control decision do you wish you'd made differently? The 340 roles story is mine. I want to hear yours. The worse, the better.

The Illusion of Scale, Part 2: When Your Data Model Becomes Your Bottleneck

Anusha Mukka — Sun, 17 May 2026 06:19:41 +0000

I want to talk about the cruelest kind of technical debt. Not the kind where someone wrote bad code, and you can see it. The kind where the code is clean, the tests pass, the results are correct, and you're still screwed.

Data model debt.

It hides. For months, sometimes years. It doesn't announce itself. It just sits inside perfectly functional code, returning correct results, passing every test. And then one day, you realize everything else is built on top of it, and you cannot move it without moving everything.

This is Part 2 of a series about assumptions that quietly break systems at scale.

The customer who broke our schema

A few years into working on a multi-tenant system, we onboarded a large enterprise customer. System had been running great for over a year at that point. Hundreds of tenants, smooth operations, no major incidents. We were feeling pretty good about ourselves.

This customer had fifty million records in a table where our typical tenant had maybe fifty thousand.

Same schema. Same queries. Same everything. But queries that ran in 200ms for every other tenant were running in 45 seconds for them.

Nobody had designed a bad system. The schema had just quietly encoded a belief: that tenants would be roughly similar in size. That belief had never been written down anywhere. Never tested. Never questioned. It was just... assumed, the way you assume things that have always been true until they suddenly aren't.

The fix was conceptually simple -- partition the data, route large tenants differently. The implementation took months, because everything else had been built around that original schema. Every query, every index, every join had opinions about how the data was structured. We ended up running two schemas simultaneously for six weeks to migrate without downtime.

It was the most expensive technical debt I've ever watched get paid off. And I'm including the time someone accidentally dropped a production table (different story, different company, different bottle of wine).

Why "good design" has an expiration date

Here's the thing about data models: they're designed for the use cases the team can see right now. That's almost always the wrong frame, because the use cases that matter at scale are the ones nobody anticipated when the schema was first written.

The pattern is incredibly consistent. System starts with a well-normalized schema. Foreign keys everywhere. Third normal form. At moderate load, it's fine. Correct, even. Textbook stuff.

Then volume grows. Queries that touched thousands of rows now touch millions. Joins that were fast become table scans. The query planner starts making choices that surprise you, and suddenly you're reading execution plans at midnight -- midnight -- trying to understand why a query that used to take 80ms now takes 12 seconds.

Normalization optimizes for write correctness and storage efficiency. Not read performance at volume. When your read load is enormous relative to your write load -- which it is in basically every user-facing system -- those goals pull in opposite directions. You find out which one your schema actually prioritized the hard way. Usually on a Friday.

The cardinality time bomb

Okay, this one's personal because I've made this exact mistake.

A permissions table with one row per user-resource pair. Fine when users have tens of permissions. Completely reasonable design. Then fine-grained access becomes a product requirement and users can have thousands of them. Table gets enormous fast.

Every permission check is now a large query. Every access decision slows down. And because authorization sits in the critical path of almost everything, a slow permissions table makes the whole system feel sluggish in ways that are incredibly hard to diagnose. You end up chasing phantom performance issues across half the codebase before someone finally traces it all the way back to a table that's just too big to query efficiently anymore.

The schema wasn't badly designed. It was designed for a world where users had 10-20 permissions. Then the product team said "actually, we need thousands" and the schema didn't get the memo.

When you design a schema, there are two questions: "what cardinality do I expect?" and "what cardinality could this legitimately reach?" They're not the same question. The first one is optimistic. The second one saves you.

When being correct gets too expensive

If producing the accurate answer requires joining five tables and aggregating across millions of rows... correctness has a real cost. A cost you pay on every single request.

Your options at that point are denormalization, pre-computation, materialized views, or derived tables. They all work. They all introduce consistency challenges that the normalized schema never had. That's the actual tradeoff, and it's worth naming clearly: not "normalization vs. performance" but "easy to get right" vs. "fast under real load."

Choosing consciously is very different from discovering the tradeoff at 3am during an incident. Trust me on this.

Migrations: where you pay the real price

A migration that takes 30 seconds in development can take three weeks in production. Not because the operation changed. Because the table grew from thousands of rows to billions, and suddenly every part of the process has consequences you never thought about.

Locking is the first problem. DDL operations on large tables can block reads or writes even briefly. "Briefly" on a hot table cascades into timeouts across the entire system within seconds.

Backfill is the second. Writing a new column's default value to a billion rows is a lot of I/O competing directly with live traffic.

And then there's the dual-write period -- running old and new schemas simultaneously so you can migrate without downtime. This is the right approach. It's also the approach that reveals every single implicit assumption in your application code. Things you didn't know your code believed about the schema. Fun.

It almost always happens under pressure too. Nobody says "let's do a major schema migration" when things are going well. They say it when things are on fire. Plan for it before you're in that situation. You won't, but you should.

What I'd tell my past self

Design for your read patterns, not just your write patterns. Know which queries are on your critical path and whether your schema serves them cheaply or with heroics.

Write down your cardinality assumptions explicitly before you ship. Explicitly. In a document. "This table is expected to have X rows per tenant. At Y rows, query Z will degrade." If you can't fill in those numbers, the answer to "will this hold at scale?" is also unclear.

Separate your operational and analytical models early. The schema optimized for transactional correctness is rarely the schema optimized for reporting. Trying to serve both from one schema is a compromise that satisfies neither at volume.

And treat major schema changes as an operational project, not a technical task. They need a plan, a rollback strategy, a communication plan, and ideally someone who has done it before and can warn you about the part you haven't thought of. There's always a part you haven't thought of.

Next up: why access control is one of the most quietly expensive places for schema assumptions to go wrong at scale. Spoiler: 15 roles became 340.

What data model decision have you had to undo the hard way? I want the painful stories. The "we ran two schemas for six weeks" stories. The more awful, the more I want to hear it.

The Illusion of Scale, Part 1: When Your "Scalable" System Isn't

Anusha Mukka — Mon, 11 May 2026 00:48:03 +0000

I want to talk about something that's been bugging me for a while.

There's this moment -- and if you've been in this industry long enough you know exactly what I mean -- where a system that looked rock solid just... stops working. Not dramatically. Not with a big crash and a SEV page at 3am (though sometimes that too). It's more like a slow suffocation. Latencies creep up. Queues get deeper. Someone opens a ticket that says "it feels slow" and you roll your eyes because everything feels slow to users, but then you look at the graphs and oh. Oh no.

I've been on both sides of this. I spent years working on public-sector infrastructure -- criminal justice workflows that had to work across 87 counties in a state, which sounds boring until you realize that "87 counties" means 87 different usage patterns, 87 different peak hours, and at least 12 counties who will absolutely hammer your API in ways you never anticipated. More recently I've been in enterprise AI infrastructure, where the fun game is "this API call costs $0.003 and we make it 40 million times a month, do the math."

Both times, the system didn't fail because we forgot to add servers. It failed because of something dumber.
This is the first in a series I'm writing about scale assumptions. I don't have a clever acronym for it. It's basically: the decisions that seem fine when you're small and make you want to quit your job when you're big.

Linear thinking will absolutely wreck you

Here's the thing nobody tells you early in your career: scaling is not a linear problem, and your intuition about it is almost certainly wrong.

A system handles 1,000 req/s. So 100,000 is just... more machines, right? Tune some indexes, maybe bump the connection pool, call it a day?

Sometimes, honestly, yes. I've had that experience and it's great. You feel like a genius. "We just horizontally scaled it." High fives all around.

But more often -- and this is the part that took me embarrassingly long to internalize -- the bottleneck isn't compute. It's a design choice someone made in week 2 of the project that seemed totally reasonable at the time.

I'll give you a specific example because I think abstractions are useless here.

We had a system running in pilot with one county agency. Worked beautifully. Fast, stable, everyone's happy. We expand to three agencies. Same code. Literally the same code, no changes. System slows down noticeably.

I remember staring at the metrics genuinely confused. Nothing changed! What is it then?

What changed was width. Three agencies meant three times the concurrent load on shared workflow components. Database access patterns that were totally fine with one agency's usage started colliding. Integration points that had been sized for one agency's volume were now contested. It wasn't a bug. It was an assumption -- that the system would scale linearly with tenants -- that nobody had written down because nobody had thought to question it.

That was the week I started losing sleep about the statewide rollout. Not because the architecture was bad -- it was actually pretty solid for what it was designed for -- but because "what it was designed for" and "what it was about to face" were diverging fast.

The synchronous call in the hot path (a.k.a. my nemesis)

Okay, pet peeve time.
A 50ms synchronous call to a downstream service. Totally fine at low traffic. You barely notice it. It's in the critical path but hey, 50ms, who cares.

Then traffic goes 10x and suddenly that 50ms dependency is your ceiling. Every request is waiting on it. When it has a bad day, you have a bad day. When it times out, you time out. And the really fun part: by the time you realize this is the problem, it's woven into everything. You can't just "make it async" without rearchitecting half the request flow.

I don't have a clean solution here. I just have scar tissue.

Data models: where optimism goes to die ####

I need to rant about schemas for a second.
Every bad scaling story I have eventually comes back to the data model. Not because anyone designed a bad schema -- usually the schema was perfectly sensible for the requirements as understood at the time. The problem is that schemas encode beliefs about the future, and we are terrible at predicting the future.

Beliefs like:
●      "We'll only have a handful of roles" (we now have 47)
●      "This workflow has 4 states" (it has 11, plus 3 that are technically illegal but exist in prod)
●      "This lookup will always be fast" (it was, until someone added a tenant with 2M records)
These aren't mistakes. They're reasonable bets that didn't pan out. But the wreckage is the same either way.

The logging bill

This one still makes me laugh in a pained way.
You start a project. Good engineering culture. "Let's log everything, we'll need it for debugging." Absolutely correct instinct! Gold star.
Fast forward 14 months. Someone pulls up the infrastructure bill and goes "uh, why is our logging pipeline costing more than our actual application?" And everyone looks at each other. Nobody planned for this. Nobody put "the audit trail will eventually need its own architecture team" on any roadmap. It just... happened. Slowly, and then all at once.

p99 is not a rounding error

I used to think about p99 the way most people do: as an edge case. The unlucky 1%.

Then I did the math on a system doing 100k req/s and realized that 1% is a thousand requests every second getting a bad experience. Those aren't theoretical users. They're filing support tickets. They're hitting retry. Their retries are making other requests slower. The p99 tail is generating its own secondary workload that feeds back into the system.

Your unhappy path, at scale, is a system unto itself. That realization changed how I think about optimization priorities pretty fundamentally.

What actually breaks (spoiler: it's never what you tested)

Look. I have never -- not once in my career -- seen a system fail in production the same way it failed in load testing. The tests always pass because test traffic is polite. Real traffic is feral.
Real traffic is: retries stacking on retries. One tenant with 10x everyone else's data volume. A permissions edge case that only fires for one specific role combination that nobody on the QA team had. Duplicate events from an upstream that swore they'd deduplicate on their end. Events arriving out of order because someone's clock is wrong.

The thing I got most wrong, personally: I assumed a decision-making component would maintain consistent latency as we onboarded more systems. In isolation, it was fast. Really fast. What I didn't think about was what happens when multiple systems are doing concurrent writes to the shared database underneath it. The component was fine. The contention was the problem. And you can't see contention in a single-system test. By definition.

I think the broader lesson -- and sorry if this sounds hand-wavy but I genuinely believe it -- is that at scale, failures happen in the interactions between components. Not in the components. A retry policy that's totally safe in isolation starts amplifying failures when combined with another service's retry policy. Cache invalidation creates cascading churn nobody modeled. A permission check that's microseconds alone shows up on flame graphs when it's called 50,000 times per second.

There's one debugging session that broke my brain a little. Access control issue. Could not figure out where to even look. Turned out we had multiple sources of truth for permissions and they'd drifted apart. The system was just... checking whichever source it hit first. There was no canonical answer to "does this user have access." I had to reconstruct the state of three different systems at a specific timestamp to understand one decision the system had made.
That was when I realized: past a certain scale, you stop debugging code and start debugging emergent behavior. And that's a fundamentally different skill.

So what do you do about it?

I'm not going to tell you to design for massive scale on day one. That's almost always wrong. YAGNI is real. Premature optimization makes systems worse, not better.

But.
Some decisions are genuinely hard to reverse. And you should at least know which ones they are:
●      Your data model (migration under load is hell)
●      Sync vs. async boundaries (you can't easily untangle these later)
●      Consistency vs. availability tradeoffs (distributed systems don't let you change your mind cheaply)
●      Authorization architecture (this one always comes back to haunt you)
●      Audit and retention strategy (see: logging bill, above)

Get any of these wrong and the rewrite happens under pressure, in production, while users are affected, with half the team arguing about the approach and the other half on PTO. It's never the calm six-month project you pitch to leadership.

Next time I'll write about the one that's cost me the most career stress: data modeling decisions that look totally fine on day one and become load-bearing walls by year three. I have stories.

Genuinely curious -- what's the scaling assumption that burned you worst? The one where you looked at the system and went "oh no, this was baked in from the start"? Drop it in the comments, I collect these like trading cards at this point.

When the Cloud is Too Slow: Enter Fog Computing

Anusha Mukka — Sat, 11 Apr 2026 17:38:01 +0000

You know that feeling when you're waiting for a response from your cloud service, and it feels like forever? Now imagine that same delay happening for a self-driving car making a split-second decision, or a smart factory robot on an assembly line. Yeah, not great.

I've been digging into this problem lately, and I wanted to share what I've learned about a pretty cool approach that's gaining traction: hierarchical fog computing combined with some clever optimization tricks.

The Problem: Everything Lives in the Cloud (And That's a Problem)

Here's the thing. We've gotten really good at building cloud infrastructure. AWS, Azure, GCP—they're incredible. But as we add more IoT devices everywhere (smart homes, industrial sensors, autonomous vehicles), we're running into a fundamental issue:

The cloud is physically far away.

When your smart thermostat needs to process data, that packet has to travel potentially hundreds or thousands of miles to a data center and back. For simple tasks, that round-trip can take 50-200 milliseconds. For real-time applications? That's an eternity.

Plus, you're sending everything to the cloud:

Burning through bandwidth 💸
Draining device batteries 🔋
Creating potential privacy issues 🔒
Wasting cloud resources on trivial tasks

There has to be a better way.

Enter Fog Computing: The Middle Ground

Fog computing is basically the answer to "what if we put mini data centers closer to where the action is happening?"

Think of it like this:

Traditional Model:
IoT Device → (hundreds of miles) → Cloud → (hundreds of miles back) → Response

Fog Model:
IoT Device → (few feet) → Fog Node → Decision made locally
→ Only important stuff goes to cloud
The fog layer sits between your devices and the cloud—on routers, gateways, local servers. It handles the time-sensitive stuff locally and only sends the heavy lifting or long-term storage to the cloud.

But Here's Where It Gets Tricky

Okay, so fog computing sounds great. But now you have a new problem: how do you decide what runs where?

Imagine you're managing thousands of IoT devices, and each one is generating tasks that need to be processed. Some tasks are urgent (like collision detection), others are less critical (like uploading historical temperature data). You have:

Edge devices with limited CPU and battery
Fog nodes with medium computing power
Cloud with unlimited power but high latency

The million-dollar question: For each task, where should it run?

This is called the task offloading problem, and it's harder than it sounds because you're trying to optimize multiple things at once:

Minimize latency (keep things fast)
Minimize energy consumption (save battery)
Minimize costs (use resources efficiently)
Respect deadlines (urgent tasks can't wait)

Hierarchical Architecture: Think in Layers

What I've been researching is a three-tier hierarchical approach:

Layer 1: The Edge (Your Devices)

Smartphones, sensors, smart cameras
Super limited resources
Makes quick decisions: "Can I handle this myself?"

Layer 2: The Fog (Local Processing)

Routers, gateways, local servers
Moderate computing power
Handles most of the real-time processing
Coordinates with nearby fog nodes
Only escalates to cloud when necessary

Layer 3: The Cloud (The Big Guns)

Massive data centers
Heavy computations, machine learning training
Long-term storage and analytics
The beauty is that each layer knows its role and passes work up only when needed. It's like having a good manager who doesn't escalate every little thing to the CEO.

The Optimization Challenge: Grey Wolf to the Rescue

So how do you actually decide where tasks should run? You need an algorithm that can:

Make decisions fast (no time for complex calculations)
Handle changing conditions (devices come and go)
Optimize multiple objectives at once
Scale to thousands of devices This is where Grey Wolf Optimization (GWO) comes in. And yes, it's literally inspired by how wolves hunt.

How Wolves Hunt (Seriously)

Grey wolves have a pack hierarchy:

Alpha (α): The leader, makes final decisions
Beta (β): The advisor, second in command
Delta (δ): Scouts, soldiers, elders
Omega (ω): The rest of the pack
When hunting, the pack uses a coordinated strategy:

Track and approach the prey (exploring solutions)
Surround the prey (narrowing down options)
Attack when the time is right (converge on optimal solution)
The algorithm mimics this: you start with a bunch of random solutions (the pack), identify the best ones (alpha, beta, delta), and have the rest follow their lead while still exploring. Over time, everyone converges on the best solution.

Why This Works for Fog Computing

In our case:

Prey = Optimal task distribution across edge/fog/cloud
Pack = Different possible ways to allocate resources
Hunting = Iteratively finding the best solution
The algorithm runs fast (critical for real-time decisions), avoids getting stuck in local optima, and handles the complexity of balancing latency, energy, and cost.

Adding Deep Learning to the Mix

Here's where it gets even better. We can combine GWO with deep learning to make smarter predictions:

Step 1: Predict the Future (kinda)

Use LSTM networks to predict incoming workload patterns:

"Oh, it's 5 PM, traffic pattern analysis requests are about to spike"
"Battery on this device is at 20%, we should offload more"

Step 2: Classify Tasks

Use a feedforward neural network to classify tasks:

Compute-heavy vs. latency-sensitive
High-priority vs. can-wait
Local-capable vs. needs-cloud-power

Step 3: Optimize with GWO

Feed all this info into the GWO algorithm to find the best task distribution in real-time.

Step 4: Learn and Adapt

Use reinforcement learning to improve over time based on actual results.

The Results (Why This Matters)

Early research shows some pretty impressive numbers:

Latency reduction: 40-70% compared to cloud-only approaches
Energy savings: Up to 80% by processing locally when possible
Throughput increase: 80%+ by distributing load efficiently
Faster convergence: 20-30% quicker than traditional genetic algorithms
Real-World Applications

Where does this actually help?

Smart Cities:

Traffic light coordination (can't wait for cloud round-trip)
Emergency response systems
Public safety monitoring

Industrial IoT:

Manufacturing robots (milliseconds matter)
Predictive maintenance
Quality control systems

Healthcare:

Patient monitoring (life-critical response times)
Wearable health devices
Remote surgery assistance

Autonomous Vehicles:

Real-time obstacle detection
Cooperative driving (vehicle-to-vehicle)
Edge-based navigation
Why I Find This Fascinating

I've spent the last decade building distributed systems at scale—from nation-wide law enforcement infrastructure to Meta's monetization platform handling billions of requests. Here's what strikes me about this approach:

It's Practical: This isn't just academic theory. These are real problems I've encountered: how do you reduce latency from hours to minutes? How do you optimize resource allocation when you have millions of users?
It Scales: The hierarchical model mirrors how we build microservices—each layer has a specific job, clear boundaries, and knows when to escalate.
It's Adaptive: Systems that can learn and optimize themselves are way more resilient than static configurations. I've seen this firsthand—adaptive systems survive conditions you never planned for.
It Solves: Multi-Objective Problems In production systems, you're never optimizing just one thing. It's always latency AND cost AND reliability AND user experience. GWO handles this gracefully.

The Challenges (Let's Be Real)

Nothing's perfect. Here are the hard parts:

Complexity: Managing three tiers is harder than managing one. You need coordination, monitoring, fallback strategies.

Edge Heterogeneity: Your edge devices aren't uniform. Different CPUs, memory, network capabilities. The algorithm has to handle this diversity.

Network Reliability: What happens when a fog node goes down? You need fast failover and re-optimization.

Privacy & Security: Distributing processing means distributing attack surface. Need end-to-end security across all layers.

Debugging: Ever try debugging a distributed system? Now add "distributed across thousands of devices in the real world." Fun times.

What I'm Working On Next:

I'm currently diving deeper into:

Reinforcement learning integration: Making the system continuously improve from real traffic patterns

Multi-agent coordination: How fog nodes can collaborate without central control

Fault tolerance: Graceful degradation when nodes fail

Real-world deployment considerations: Because simulations are one thing, production is another

I'm also exploring how this applies to edge AI scenarios—running ML models across the hierarchy, where each layer handles what it can and passes up only what it must.

Try It Yourself - If you want to experiment with fog computing concepts:

Simulation Tools:

iFogSim: Java-based fog computing simulator
EdgeCloudSim: Simulates edge computing scenarios
Python + NetworkX: Build your own simple model

Start Small:

Model a simple 3-tier architecture
Create synthetic tasks with different requirements
Implement a basic task scheduler
Compare random vs. optimized offloading

Read More: The research in this space is moving fast. Look for papers on:

Task offloading strategies
Deep reinforcement learning in edge computing
Optimization algorithms for distributed systems

Final Thoughts

We're at an interesting inflection point. IoT devices are everywhere and getting smarter, but the old "send everything to the cloud" model is hitting physical limits.

Fog computing isn't going to replace the cloud—it's going to make it better by handling what it does best and letting the cloud focus on what it does best.

And optimization algorithms like GWO combined with deep learning? They're giving us tools to manage this complexity at scale, in real-time, with multiple competing objectives.

If you're building IoT systems, industrial automation, edge AI, or anything where latency really matters—it's worth understanding these concepts. The architecture patterns and optimization techniques apply to a lot more than just academic papers.

What do you think? Are you working with fog/edge computing? Running into latency issues with your IoT systems? I'd love to hear your experiences in the comments.

And if you're interested in the full technical details, I'm working on a research paper diving deep into the hierarchical GWO approach. Happy to chat about it!

P.S. - Yes, I did just spend several paragraphs explaining computer science concepts using wolf hunting analogies.. 🐺