That was one reply to my article about testing:
So I thought, well... 🤔
Image credits:
https://blog.gardeviance.org/2008/04/three-stages-of-expertise.html
Actually, I've been thinking about this and my personal development for a couple of years.
First, I recognized that I was changing a lot of things, about how I code and how I look at technology and problems.
I started to code very early in 5th grade. By the time I was done with secondary school, I decided not to study computer science (and went for electrical engineering instead) because I thought there's nothing more I could learn about programming. I thought, I was just the best programmer in the world.
As you already may guess by now, I was totally wrong. 🤦🏻♂️
I had to learn what software engineering is about: Architecture, testing, performance, maintainability and much more that I have not really considered being an important part, in fact being the distinction between a programmer and a software developer.
So I read a lot of articles, books and tried to turn all the knobs to 11 which led to very complex software that was build to cover all possible cases that could ever happen.
There are a couple of similar articles like this. In fact, there are plenty of suggestions about possible phases, you can have 4, or five or even more (see References. All tackling slightly different aspects of being a programmer.
From my perspective, I distilled my personal evolution into three phases and its transitions:
Rapid Prototyping
- Writing code quickly and fast, the more code, the better!
- No or very few automated tests (See tweet above)
- No clean code
- No sensible source code structure
- Large files & methods
- No proper naming ("There are plenty of letters in the alphabet!")
- Hard-coding values and file names all over the place
- No underlying design principles or architecture
- No one else can read or understand the code
- "I can write a RegEx to parse the XML"
- "Look, that source file finally has over 1000 lines of code! Only professional programmers write that much code!"
- Being a programmer instead of a software engineer
Over-Engineering
After failing in a couple of projects, I read articles and books about what I could do and try to apply all of that which leads to
- Implementing features for the future without being requirement
- Making everything configurable an pluggable
- Designing for scalability and performance
- Complicated design & architecture ("I've applied all GoF patterns!")
- High abstraction but low domain knowledge
- Using the most complicated and esoteric language features
- 100% unit test code coverage
- Starting with clean code
- Learning about new technologies and trying to immediately use them
- DRY to the max
- "Let's put that code into a library!"
- No one else can understand the code (but at least they can read it)
Pragmatism
After also failing with the previous approach, I tried to:
- Only build what's necessary (YAGNI)
- Find a good testing balance (testing pyramid)
- Clean code
- Domain specific code
--
CarWheelinstead ofAbstractVehicleRotatingDataItemFactory-- Don't think about how to implement something but also why - Accept that low redundancy (DRY) leads to high coupling and balance the two
- Optimizing for performance or scalability only when it's actually necessary (either by dedicated requirement or when the application runs measurably bad)
- Learning about new technologies but look behind the hype (Testability, documentation, contributions, security etc) and probably wait until it's battle-tested
- Recognize code smells
Conclusion
Those three phases are not scientifically proven. In fact, it's just what I've personally learned by thinking about myself on the one hand. There even might be other phases, I haven't experienced yet. On the other hand, it seems to fit to a lot of developers I've met during the last 15 years.
You can't necessary assign years of experience to those phases, some might as well skip a phase or it's been reduced to a short period depending on the personality, the way of learning or mentoring.
The most important thing is: You never never stop to learn. Make sure there is always someone smarter than you.
I learn new things every day and I am very grateful for my colleagues and friends.
Do you find any similarities in yourself?
Top comments (25)
Great post.
Your observations are strictly related to the Dunning-Kruger Effect.
Thanks for the feedback! Yes, I read about it some time ago but forgot it :) That is definitely related!
This definitely applies to just about all areas of life! I remember when I was learning to play the game of go, there was a point where I kind of thought I knew what I was doing (I was about 5 kyu, which is pretty bad). Now I realize how totally ridiculous that idea was. Plato wrote (quoting Socrates), "I know that I know nothing." That's so true. The more we learn, the more we realize what a gulf there is between what we know and what is out there to learn.
Well put! Do you think that when you have experienced it once in a field of Knowledge, it will prevent you fall into that trap in another field?
The tricky thing is, I think, that it's easy to get stuck at that level in the middle. For me, go definitely opened up my eyes, and I look at everything, including programming, very differently as a result. Having that epiphany in one field should be transformative, but not everyone actually gets there.
IMO, all three phases have value. That first stage might feel like "what a fool I was", but in a lot of ways, that shit is enlightened! I think the problem comes when one of these approaches is applied, without context, to all problems.
I think it's a mistake to start a project in over-engineered phase, better to try 10 versions of rapid prototyping, and then as you start to get a sense of the problems, potential solutions, and where the risk is and what's worth investing in, bring in that pragmatism to tame the beast before it grows horns. And over time, as the requirements call for it, and only ever after it's needed, bring in that over-engineering (I guess at that point, it'd be just engineering?) And it's also super valuable to be able to scope which approach you take to which feature you're working on, all within the same code base. Risk, volatility, and uncertainty can vary from feature to feature and method to method.
They're all good, the flaw comes when you don't scope them and they get applied to problems they're a poor fit for. It's a gradient, don't invest in volatile code you might want to throw away tomorrow, and don't make a mess in code that's proven it's worth.
Not a prescription here, because I'm confident this suggestion is effective for many. For me, though, it was a bit paralyzing (I wasn't free to experiment or explore ideas if I wasn't incredibly confident in them, because I was too worried I'd look like a fool). What usually works for me is to always make sure I'm better than I was yesterday. It is valuable to check in and see where everyone else is from time to time, but it's better to rank myself against where I was than against where someone else appears to be. Also, pushing myself into my zone of discomfort. Basically, whatever it takes to stave off complacence and stagnancy.
You made some very good points here that first underline that people are different. That why I didn’t put “every developer” in the headline.
Regarding the phases: I still value them as experience but since I work in a large enterprise, 95% of my work needs to be high-quality maintainable code. So there is less room for experiments. However, we still do it from time to time intentionally and time-boxed. And what you describe is actually agile development. You start a small feature, put it out in the field quickly to get feedback and then change and add stuff iteratively.
Regarding the “find someone smarter”: that’s an exceptionally good thing if you are able to compare yourself objectively to older versions of you. But most of the time, I think the “frog slow cooked in a pot” metaphor applies where you are simply not aware of how it if you change and what might be an achievable goal.
I not sure if the bad experience you describe might be the result of a hazardous working environment fueled by seniors and managers that do not understand the value of making
mistakes?
It's hard to convince even experienced engineers of this.
Can you say more about this one? I think I recognize when code smells bad, but I'm not very good at figuring out why. Also I don't know if readers who are new to engineering are familiar with the term.
Yes. I also know experiences developers that occasionally want to introduce a new common library. And might even be OK in some cases but most of the time the coupling issues weight much higher in the mid-term than the DRY benefits.
Regarding the Code Smells: I would recommend you to read „Clean Code“ by Robert C Martin. If you have a system like sonarqube for metrics, it will show you smells and recommend a fix. Pretty neat.
And to make it absolutely clear: Code duplication in one Code base is really a code smell and must be refactored. I specifically talking about DRY at service/systems level.
Well that depends... Having no abstraction is better than having the wrong abstraction. Sometimes it's better to write something two or even three times before refactoring to make sure you get the implementation right. Although yeah... I have rarely seen this in the same codebase.
Yeah, but I would not write something three times in parallel but change and refactor until it’s right. Use git to revert to previous versions.
Great article, thanks. I'm in the first phase, swimming through a lot of things to learn :)
I think every phase has its value. So enjoy it! My first phase was definitely fun 😎
Your a really cool dude Jan
That’s an unexpected but most welcome compliment to a nerd 🤓
At this point in my career, I think that I have forgotten almost everything that was useful.
Ah, yes. Good article, I was going to write something like this, but you beat me to it :D
This is a really interesting post and topic.
I think this is a really good talk that touches on certain of these elements:
(Not strictly related, I just think it loosely addresses some concepts examined here and is generally a good talk)
Thanks Ben, indeed a great talk!
Fun fact: Chad now works for Microsoft which acquired Wunderlist to become Microsoft Notes. And he indeed rewrote the backend again using Exchange Server :)
Listen to him on SEDaily podacst: softwareengineeringdaily.com/2018/...
Thanks for the video link. I had not seen it.
That is a great talk. Thank you Ben.
I have also hear the phases of a developer learning curve described this way:
How true it is I am not sure.
After "making own framework", most probably "framework" comes again because developers realize how difficult it is to write and maintain a well-designed framework.