We Migrated 6 Years of Legacy Code. Production Chose Violence.

Krishnankamatchi — Tue, 22 Sep 2026 19:25:40 +0000

I joined a wedding-tech company in December 2025.

Like every developer joining a new team, my first few weeks were spent figuring out three things:

How does the application work?

How does the team work?

And why does everyone suddenly become religious before a production deployment?

The platform had been running for years. Real customers, real money, a large CMS, serious search traffic, and enough business logic that deleting one innocent-looking if statement felt like a reasonable way to ruin everyone's weekend.

It was a legacy system.

I don't use legacy as an insult.

The code had survived years in production. Death would've been kinder.

Years of hotfixes, workarounds, and forgotten requirements had become part of the application.

Eventually, history becomes architecture.

And I had joined somewhere around season six.

Then I Asked About Production

Development happened in GitLab.

Dev branch. Staging. Normal stuff.

Then I asked:

"Which branch is production?"

There wasn't one.

Production lived in SVN.

Yes.

Subversion.

GitLab handled development and staging. SVN handled production.

And the deployment pipeline connecting them was...

a human being.

Developer
    │
    ▼
  GitLab
    │
    ▼
 Staging
    │ QA ✓
    ▼
┌─────────┐
│ Human™  │
│ compare │
│ copy    │
│ pray    │
└────┬────┘
     ▼
    SVN
     │
     ▼
Production

Sometimes we compared files.

Sometimes we reconstructed changes.

Sometimes code moved line by line.

You haven't experienced continuous delivery until the continuous part is a developer continuously checking whether they copied the right function.

Worse, Git and SVN had diverged.

Production had hotfixes staging didn't. Git had changes production didn't.

They weren't really two environments anymore.

They were distant relatives who still shared a surname.

Deployments Were... Ceremonial

Production releases needed tickets, approvals, and a scheduled window.

The application wasn't containerized, so deployment also meant downtime.

Naturally, releases happened early in the morning.

Because apparently software is more stable before breakfast.

05:30 — Deploy
06:35 — Developers verify
07:40 — QA verifies
08:00 — Suspicious silence
08:15 — "One small issue"
08:16 — Nobody agrees what "small" means
08:30 — Deployment window closes

If something broke early, rollback.

If something broke later...

Congratulations. Your bug has been promoted to management.

The scary part wasn't even downtime.

It was that the code QA approved wasn't necessarily identical to what reached production.

There was always a gap between:

"QA approved it."

and:

"Production will behave exactly the same."

That gap is where developers discover new forms of prayer.

Eventually, though, I got used to it.

Which is probably the most dangerous stage of technical debt:

"That's just how we do it."

Technical debt rarely arrives saying technical debt.

It usually says:

"Just for this release."

Then it gets a chair.

Then a desk.

Six years later, it has admin access and its own bedroom.

Then Came One Thursday

About two months after joining, we finished another production release.

QA signed off.

Nothing caught fire.

Friday was a holiday.

Basically a five-star deployment.

I walked into one of our VP's cabins to say production was done.

We started talking about the deployment process.

Fifteen minutes later, we had somehow progressed from:

"This is annoying."

to:

"Let's migrate it to AWS."

Migrate to AWS.

Beautiful sentence.

Three words.

Software has a strange habit of making terrifying amounts of work fit into very short sentences.

Behind those three words were six-plus years of code, data, infrastructure, integrations, and production behavior — all connected in ways we understood, and probably a few we didn't.

This wasn't:

docker build .
docker run

followed by applause.

Some parts of the application ran because we understood them.

Other parts appeared to run because nobody had informed them they were allowed to stop.

The backend would be handled separately.

I was asked to own the frontend.

The intelligent response would've been:

"Let me analyze it first."

Instead, my brain spent approximately five seconds consulting my ego.

"I can do it."

I even promised a frontend foundation over the three-day weekend.

My confidence lasted until I started my bike.

The Ride Home Improved My Intelligence

Every kilometer introduced a new question.

SEO-critical pages?

Complex routing logic?

Caching?

Legacy dependencies?

Undocumented production behavior?

How much of this thing actually works locally?

And finally:

Can I call him and pretend the three-day estimate was a network issue?

By the time I reached home, I'd mentally migrated the application six times.

Five failed.

The sixth required technology that probably doesn't exist.

Friday: Zero Useful Commits

Friday, I barely coded.

I read.

Traced dependencies.

Mapped boundaries.

Tried to understand what we were actually moving.

The goal became simple:

Don't replace today's legacy system with next year's legacy system.

AI was already useful, but pointing an agent at six years of production code and saying:

"Modernize this."

felt like a reliable way to generate 40,000 lines of beautifully documented regret.

I eventually settled on a frontend architecture inspired by Hexagonal Architecture.

Not the textbook version — the useful part: keep business logic away from framework and infrastructure details, and put APIs and external services behind clear boundaries.

The point wasn't architecture cosplay.

It was making sure changing an API or infrastructure detail didn't require open-heart surgery on the entire frontend.

Saturday disappeared.

Then Sunday.

By Monday, I had a foundation.

Not the migrated application.

A place where one could actually be built.

Then Everyone Got Involved

I started migrating alongside normal sprint work.

This quickly revealed an advanced software-engineering principle:

Oh. This is going to need more people.

So I did what any responsible engineer would do.

I trapped a backend engineer who knew the legacy system into joining me.

By the time he realized what he'd signed up for, he was already in the Git history. No refunds.

His knowledge was more valuable than any documentation.

Because the most dangerous legacy code isn't ugly code.

It's ugly code that looks unnecessary.

Delete it, feel intelligent, and three days later discover it was quietly holding a revenue flow together.

Eventually, the team joined.

At first, the architecture caused friction.

Fair enough.

Something that looks beautiful after two sleepless nights designing it can look like an elaborate tax scheme to everybody else.

Then people challenged it.

Changed it.

Improved it.

It stopped being my architecture.

It became our application.

The New World

We moved toward a containerized setup:

              User
                │
                ▼
         ┌─────────────┐
         │ Akamai Edge │
         │ CDN / Cache │
         └──────┬──────┘
                ▼
         ┌─────────────┐
         │   AWS ALB   │
         └──────┬──────┘
                ▼
         ┌─────────────┐
         │ ECS Fargate │
         │ Containers  │
         └──────┬──────┘
                │
        ┌───────┴───────┐
        ▼               ▼
     Next.js         Backend APIs
    SSR / ISR             │
                          ▼
                      Data Layer

Next.js handled SSR and ISR.

ECS Fargate ran containers.

ALB routed traffic.

Akamai handled the edge.

For the first time, production could be built from source instead of being lovingly reconstructed by humans.

Beautiful.

Suspiciously beautiful.

Because this was also an SEO-heavy platform, we had to preserve URLs, redirects, metadata, rendering behavior, and caching.

Turns out search engines don't care that your architecture diagram looks cleaner.

Migration Day

Before cutover, we prepared backups, a rollback plan, database movement, DNS, ALB health checks, monitoring, and the WIP page.

We also temporarily reduced the Akamai edge TTL so changes would propagate faster.

This seemed sensible.

Remember that.

Friday, around 6 PM:

Code freeze.

WIP page up.

Database moved.

Containers online.

Domains switched.

ALB targets healthy.

Logs streaming.

Most conversations became:

"Done?"

"Checked."

"Check again."

Eventually, WIP came down.

Homepage loaded.

Authentication worked.

Core flows worked.

SEO pages rendered.

APIs responded.

We had actually migrated it.

This would've been an excellent place for the story to end.

Production disagreed.

Why Is Memory Still Going Up?

Real traffic arrived.

Memory climbed.

In lower environments, the frontend might idle around 150–200 MB.

Production crossed 2 GiB.

Then we looked at request metrics.

Some dashboards were reporting traffic in the tens of thousands of requests per second.

Well.

That's probably not ideal.

Crawler storm?

Scraper?

DDoS?

So began the bot hunt.

Everyone Is a Suspect

We had:

Akamai for edge traffic.

ALB metrics for origin traffic.

Loki for application logs.

CloudWatch for infrastructure.

And Grafana for converting numbers into anxiety.

We correlated:

IP addresses
User agents
Routes
Cache hits and misses
Response codes
Origin request rates
Container memory

We checked IP reputation.

Blocked obvious bots.

Filtered suspicious user agents.

Checked for origin bypass.

Inspected cache headers.

Streamed logs.

Watched Grafana.

Find suspicious traffic
        │
        ▼
Block it
        │
        ▼
Watch Grafana
        │
        ▼
Find another pattern
        │
        ▼
Watch Grafana
        │
        ▼
Question career choices

It felt like operating an extremely boring air-defense system.

Except the missiles were HTTP GET requests.

Midnight passed.

Then 1 AM.

Then 2 AM.

At some point, our VP looked at us and said:

"Let's stop. We'll continue tomorrow."

Fair.

He had a family to go home to.

Well, I had Grafana.

A few of us continued until around 6:30 AM.

I went home, freshened up...

and came back.

This was not resilience.

I simply couldn't mentally close the browser tab.

Please don't take this as career advice.

Plot Twist: We Were Attacking Ourselves

Saturday morning.

Almost empty office.

Grafana. Loki. Akamai. AWS. Logs.

Too many tabs.

Three nearly-dead neurons left. All three were debugging production.

Then we stopped asking:

"Who's attacking us?"

and asked:

"What if nobody is?"

The frontend used Next.js Incremental Static Regeneration (ISR).

During migration, we'd shortened cache durations because we wanted changes to propagate quickly.

At the CDN layer:

Short Akamai TTL → more requests reaching origin.

At the application layer:

Short ISR revalidation → popular pages regenerating more often.

Both decisions looked reasonable individually.

Together:

User
 │
 ▼
Short Edge TTL
 │
 ▼
More Origin Requests
 │
 ▼
Low ISR Revalidation
 │
 ▼
Frequent Regeneration
 │
 ▼
SSR + Data Fetching
 │
 ▼
CPU + Memory Pressure
 │
 ▼
Grafana
 │
 ▼
"WE'RE UNDER ATTACK!"
 │
 ▼
No.

It's us.

There it was.

Next.js wasn't broken.

Akamai wasn't broken.

AWS wasn't broken.

The load balancer wasn't secretly plotting against us.

Every component was doing exactly what we'd configured it to do.

Our assumptions were broken.

Those are the bugs that hurt the ego just enough to become permanent memories.

We adjusted ISR revalidation.

Restored saner edge caching.

Reduced unnecessary origin work.

Then we stared at Grafana again.

Traffic dropped.

Memory settled.

Origin pressure fell.

The application stopped trying to kill itself.

Stupid.

But beautiful.

What Actually Changed

The migration didn't magically make everything perfect.

Production immediately made sure we understood that.

But we had changed something fundamental.

Production was no longer manually reconstructed from a separate source-control world.

Containers gave us reproducible runtimes.

AWS gave us observable infrastructure.

Akamai became part of an intentional caching strategy.

And we finally understood much more of what happened between:

git push

and:

200 OK

That alone felt enormous.

Four Years Earlier, I Wasn't a Software Engineer

My original background is mechanical engineering.

I moved into software because I liked building things and understanding how they worked.

Kidding.

Short-term dopamine.

Four years later, I was sitting in an almost empty office debugging CDN caching, ISR, container memory, and load-balancer traffic on a production system we'd just moved to AWS.

I still have an absurd amount to learn.

Which is disappointing. I thought that was it.

That weekend made this painfully obvious.

But it also reminded me why I chose this job.

We started with something we didn't completely understand.

We designed.

We argued.

We broke assumptions.

We asked for help.

We shipped.

Production punched us in the face.

And we fixed that too.

Also:

Sleep is an excellent technology. I strongly recommend adopting it.

I Hope AI Never Takes This Part Away

AI was there.

GPT/Claude sat beside Grafana, Loki, AWS consoles, terminals, and documentation.

I use AI every day.

It makes me faster.

But I remember when being stuck meant reading documentation, digging through source code, finding a seven-year-old Stack Overflow answer from some random user who somehow knew exactly what was wrong, breaking something, fixing it, and slowly building the mental model yourself. Mental model

I don't want all of that back.

Nobody needs to manually center another <div> for character development.

But I hope we don't optimize away understanding.

Let AI write boilerplate.

Let it explain logs.

Let it help debug.

But give me the architecture decisions.

Give me the production incidents.

Give me the bug where every component is working correctly and the entire system is still wrong.

Give me that moment when Grafana finally starts moving in the right direction and everyone realizes:

Oh. We were the DDoS.

Because somewhere between SVN, containers, AWS, Akamai, ISR, an almost-melted origin, and a Saturday morning with far too little sleep...

I remembered why I chose software engineering.

And I'd still choose it again.

Names, domains, and identifying project details have intentionally been generalized. The technical events and lessons are based on a real production migration.

DEV Community: Krishnankamatchi