DEV Community

Nobody wants to own the innovation from three leaders ago

There's a specific kind of silence that hits Microsoft Teams about ten minutes after a high-profile departure is announced in the Engineering General channel. It's not exactly sadness. It's more like the sound of a dozen different teams realizing they're now the permanent owners of a "strategic pilot" that nobody actually likes.

I've been sitting through our quarterly planning sessions for the fulfillment and catalog pillars, and the pattern is becoming impossible to ignore. We aren't just fighting the legacy order management monolith anymore. We're fighting the ghost of every architectural opinion held by people who don't work here anymore.

The industry is currently obsessed with the idea that CTOs and heads of engineering are burning out and taking long career breaks. I saw a post from Gergely Orosz about this recently, noting how hard it is to fill these roles because the people at the top are just done. But we don't talk enough about what they leave behind. When a leader who pushed for "architectural diversity" or "best-of-breed tooling" exits the building, they leave a sprawl that's essentially a massive, unhedged loan. Executive turnover is the final interest payment on that sprawl.

I'm probably going to annoy some of my peers with this, but the "innovation" we celebrated two years ago has become the anchor dragging us down today.

About a year and a half ago, we had a senior leader who was convinced our SQL Server and a caching tool setup wasn't going to cut it for a specific regional expansion in Europe. They pushed hard for a very niche, distributed document store for our product metadata. At the time, I was distracted by a messy rollout in the checkout flow and I just approved it. I should have pushed back but didn't. It felt easier to let the "visionary" have their win so I could focus on keeping the site up.

That leader left six months ago. Today, that distributed document store is a nightmare. It has its own custom deployment logic, it doesn't surface logs to our log viewer properly, and only two engineers even remember how the consistency model works. Whenever the dashboard starts showing red for that service, everyone in the Incident Response channel just looks at each other. We're paying the "sprawl tax" every single day, but the person who signed the contract is long gone.

The thing about technical sprawl is that it's rarely born out of malice. It's usually born out of a desire to move fast or a belief that the current tools are "limiting." But tools are rarely the limit. Complexity is the limit.

I was looking at our tech debt board yesterday and noticed that nearly 40 percent of the items aren't actually about old code. They're about "rationalizing" systems that were introduced as shortcuts or experiments. We have three different ways to handle event-driven messaging because three different leaders at different times wanted to try something new.

When you have a stable leadership team, you can sometimes get away with this. There's enough institutional memory to navigate the mess. But as soon as that turnover starts, the sprawl turns into a liability. The new person coming in doesn't want to maintain the "innovation" of their predecessor. They want to simplify. Or, even worse, they want to introduce their own favorite tools, adding a new layer to the sedimentary rock of our architecture.

We're currently in the third quarter of our orders-v3 migration, which was supposed to solve the monolith problem. But it's stalling because we keep hitting these pockets of sprawl. We'll try to move a piece of the fulfillment logic and realize it's tied to a "special" service built during a hackathon three years ago that somehow became production-critical.

I'm simplifying this, but it feels like we're running an archaeological dig instead of an engineering org. We spend more time figuring out why someone decided to use a specific library than we do shipping features for the catalog.

There's a temptation to blame the engineers, but that's a cop-out. Engineers will build what the culture rewards. If leadership rewards "shipping the new thing" but ignores the cost of "maintaining the five old things," you get sprawl. And when those leaders leave, they take the context with them, leaving the rest of us to handle the on-call escalations for a system we never wanted.

I keep thinking about a meeting I had last week with a senior on the checkout team. They were showing me a particularly gnarly part of the session handling logic (which, to be fair, is already a mess). They asked if we could just rip it out and move to a standard library. I had to tell them no, because that specific logic is tied to a fraud detection vendor contract we signed during a panic two years ago. We're stuck with it until the contract renewal, regardless of whether it's the right technical choice.

That's the reality of the job at this scale. You're rarely making the "best" technical decision. You're making the decision that keeps the most people happy while staying within the constraints of past mistakes.

I've noticed that our most productive teams are the ones that have stayed boring. The catalog team has stuck with a fairly standard framework and SQL Server pattern for years. Their velocity is consistent. Their on-call rotation isn't a death march. Meanwhile, the teams that were allowed to "experiment" with every new framework that hit the top of the dev news cycles are the ones constantly stuck in the Deploys channel trying to figure out why their build pipeline is failing for the fourth time in a week.

This connects to something I was chewing on earlier about how we look for "savior" engineers. We hope a brilliant hire will come in and fix the mess, but they usually just add to it. They bring their own favorite patterns and the sprawl grows.

I'm not saying we should never try new things. That would be a different kind of death. But we need to start measuring the "departure cost" of our technical choices. If the person proposing this new database or this new architectural pattern left tomorrow, would we still want to own it? If the answer is no, we probably shouldn't be building it.

The most unpopular opinion I hold right now is that we should probably stop 80 percent of our "innovation" projects and spend a year just deleting things. I've suggested this in a few planning meetings, and the look on people's faces is usually one of pure confusion. They think I'm being cynical or that I'm trying to kill the "culture."

I'm not trying to kill the culture. I'm trying to make it possible for people to work here for more than two years without losing their minds. We keep talking about developer happiness, but we ignore the fact that nothing makes an engineer unhappier than being forced to maintain a "clever" solution for a problem that was solved five years ago.

Actually, that's not quite right. It's not just that the solution is clever. It's that it's orphaned. It's a piece of the stack that has no father and no mother, just a series of git blames leading to "User Not Found."

I don't have a tidy solution for this. We're still hiring, we're still growing, and the pressure to ship new features for the holiday season is already starting to ramp up. The temptation to let a team "just use this one new tool" to get a feature out the door is going to be massive.

But I'm tired of being the one who has to explain to a new senior dev why they have to learn a proprietary templating language that we only use in one corner of the fulfillment system. I'm tired of the interest payments.

The honest version of this is: we're currently incentivized to create complexity because it looks like progress, and by the time the bill comes due, most of the people who signed for it will be at a different.

Top comments (0)