You can find other posts on my blog and follow me on X/Twitter for more.
It's easy to fall in love with complexity as a software engineer.
Put a queue here. Some cache over there. Make this class abstract so you can add implementations later on, even though you only have one now. Break this service into five. Sprinkle some ML too. Spin two databases, one for relational and one for unstructured data.
The best parallel I can draw of the range of options you have in software engineering is to arts.
It's like your palette has colours that will bring out a specific feeling. Or a combination of notes that's perfect for a particular chord progression.
It's true that there's a techonlogy choice, colour or note for every situation. But too much is no good.
In many contexts, more is just dilution. With too many words, a crisp sentence becomes a weak narrative. More can also quickly turn into excess. It doesn't take much to turn a soothing breeze into a slightly annoying wind.
Don't get me wrong: I have nothing against complexity. I just believe it should be earned through necessity.
A story might better illustrate what I mean. In 2009-2010, when Node.js, MongoDB and the WebSocket protocol were starting to become more popular, I was working on a product which had as a requirement to track user behaviour on a specific screen and make it "real-time".
The "real-time" need was loosely held. Data needed to be relatively fresh but even a 30-minute lag wouldn't dramatically impact the product, given what it was used for.
The solution needed to handle a few hundred thousand users. But it would start with much fewer users. Let's say a few hundred QPS. The type of scale that maybe we could get away with live with polling, a few async jobs and get it done using Rails and MySQL, which we were already using anyway.
But, naively, it felt like the perfect opportunity to try something new.
At this point, this story might feel familiar, and you know how it goes. This specific piece of software became a hotspot for that team. While we learned something new, we spent weeks figuring out all the details to get it live. It slowed us down every time we had to touch it.
While inconvenient, we didn't mind, as we were intoxicated by the love for complexity.
We convinced ourselves doing it that way was "big-picture thinking". In a backwards way, we justified the investment through what it could be, not by what it was. But that "big picture" scale really never came.
One may argue that situations like that only happen when you're starting your career. But that's not true. Even engineering leaders and the seniormost individual contributors sometimes also fall into the complexity trap.
If you need something done, make it a cross-functional team OKR. If you've seen an issue a few times, write a strategy. If you repeat words over and over, make it an acronym. If there are way too many teams involved in a project, make it program. Not enough capacity? Headcount is the answer.
All useful tools in the some contexts, but wasteful in many others.
Even in the context of big tech companies – where complexity was earned to justify some pretty esoteric solutions to problems – the best solution usually has the fewest moving parts:
- The best goals probably articulate what you care about in simple, specific words.
- The best architecture is probably one that works and you can extend with most ease.
- The best process is probably the minimum you need to repeat what works.
- The best org is of just the right size to enable you to achieve all of the above.
In other words, just enough.
When you find yourself wondering whether you fell victim to the complexity trap:
Focus on business outcomes. Be honest about what matters to your customer and for the business. A lot of the times you can create more value through reduction. Not because doing less is necessarily better, but doing "just enough" gives you appreciation for what's missing – if it really is.
Bring it back to the "small picture". If you notice circular conversations where you end where you started, you probably have a scope problem. Fix the time and budget, and cut back the scope to put something tangible up for discussion. Set the appetite upfront and get complex if you have to from there.
Answer why now. Most people have an endless list of important things to get to, be it a product that you're building or an organisation you are managing. Very few honestly answer why you should do it all, and even fewer question why now. Very often, the world won't fall apart if you leave it to the side.
Do just enough not because you're lazy, but because you care about how much more you can do of something else and give yourself, your team, and your product some breathing room.
After all, "it's the pause or the rest that gives the piece its beauty and its shape".
Top comments (0)