There’s something deeply ingrained in many developers, including myself, that creates a tendency to over-engineer. Maybe it’s how we’re taught or maybe it’s a natural desire to “future proof” our code. Regardless, this tendency is so strong that even being aware of it is not enough to prevent the behavior.
Earlier this year, I was working on a system for helping users filter their emails. While I love Gmail filters, the experience of creating them and debugging them is a bit lacking. I wanted something much simpler for Maleega. I started off with just two very broad options:
- Mark emails from every contact as important
- Mark emails from important contacts as important
Storing and applying this setting isn’t rocket science, but that didn’t stop me from trying to make it so.
In my head were all these grand plans for everything I would build in this new system. I had brainstormed a few dozen and had only selected this as the first “rule” in my rule engine. Because of this, I couldn’t just store the setting! I needed to have a data model that could store the setting for any possible rule I could think of. This would make it much easier to add new rules in the future.
This flexibility promises many benefits, but it doesn’t come free. I wrote hundreds of lines of complex code to handle all my theoretical cases. I ended up spending 2 weeks building this framework. It would have taken a day to build this one feature with its one use case. That’s alright though! I enjoyed every minute of it and I would soon reap the benefits of all this extra work!
Fast forward 7 months.
While I had plenty of brainstormed ideas for rules when I first built my rules engine, I realized that most of them were of questionable value. Instead I spent most of my time working on more important things based on user feedback and my own usage of the product. It took 7 months before I had come up with another rule worth building.
The problem with most email filtering is that the rules are usually generated after an email comes in from a certain person or about a certain subject. However, a lot of important email comes from people we have just met or are contacting us for the first time about buying our product, offering us a job, or just networking in general. There’s a novelty in this email that makes it important, which is why the next filter I implemented was to mark all novelty emails as important.
That alone isn’t so bad and would have been covered easily by my rules engine on the backend. The problem is that this is a very distinct experience from the previous rule on the frontend. The instructions/tutorial/onboarding is going to be different. The warnings for certain actions based on these settings will be different. The interactions available based on these settings will be different.
The extra layer of abstraction I had built to make my life easier was actually going to make things harder when dealing with all these specific cases. While I had enjoyed writing this code, it was abstraction for the sake of abstraction. This was something I had vowed never to do again years ago and here I was repeating the mistake.
What I had built felt like sophisticated engineering, but the unneeded complexity makes things harder to test, harder to track production bugs, and harder to build new things in. This is especially true when doing user research because the point of doing user research is that I’m unsure of what may be needed 6 months down the line.
I honestly can’t fully explain this fallacy. Maybe it’s because 90% of software isn’t rocket science and good products don’t need complex code, but complex code is much more interesting to write. I could have some deep programming need that’s not necessarily being met just by building a great product. Or maybe the habits I had developed in my younger years are still causing me to make certain decisions reflexively.
Regardless, I hope I’ve finally kicked the habit. I deleted my rules engine and rebuilt my old rule and my new rule in less than two days. There’s still some abstraction, but it’s a lot simpler and it provides an actual benefit to the codebase. The abstraction I have now has a purpose beyond being just abstract.
Top comments (5)
This is so true. I've dealt with codebases where almost every design pattern is applied, just for the sake of having a design pattern. Interfaces that have only one implementation... We are taught that more abstraction is better, but that is not always true. If, instead, we were encouraged to make simpler code and refactor when needed then we would make better solutions
My favorite satire of the "more abstraction is better" mindset is enterprise fizzbuzz.
This is a very good representation, indeed
Always beware of the "rabbit hole". I can't count the number of times I've overengineered something, and ~50% of the time it's never used (Luck/Skill? who knows :)). It's even more embarassing when you are showing a new team member through the codebase and have to explain why it's overengineered and realize during the explanation that the case you were allowing for can never ever happen ... and never could.
My personal "ah-ha" moment was during one of these cases with a contractor and I had to honestly tell them "the entire premise that this code is written upon is complete crap". We had a good laugh and then we re-coded it in 1 day removing 80% of the old code.
This was something I did when I first started development. I tend to over-engineer everything and apply whatever I have learnt even when not necessary. I developed future features in products which end up not being used. I did so, sometimes to show off,future proof the app or just write complex algorithms because it made me feel confident 😂.