Software engineer path.

Table of contents.

• Introduction.
• A journey in software development.
• Training and practice.
• (My) Correct order of things.
• Fundamentals / A Philosophy of Software Design (John Ousterhout).
• Fundamentals / UNIX philosophy (Doug Mcllroy et al.).
• Fundamentals / Pragmatic Programmer (David Thomas and Andrew Hunt).
• Fundamentals / Clean Code (Robert C. Martin).
• Medium / Release It! (Michael T. Nygard).
• Medium / Domain Driven Design (E. Evans).
• Medium / Designing Data-Intensive Applications (Martin Kleppmann).
• Medium / Infrastructure as Code (Kief Morris).
• Modularization problem.
• And OOP? What about OOP?
• Contributions.

Introduction.

This is an ongoing article. This article will be updated overtime to gather my thoughts and highlights about software development.
This article is useful for all professionals involved in software products development (developers, architects, engineering managers and even product owners) and, to be honest, it is useful for me to recap my ideas.

During the years I've read many books about software development, I guess around 50. I did this without any advice, just picking books that seemed interesting for my career. After tenths of books and years, I now have a clear idea about what would have been the best order to follow. Some of them are basic, others are specific. Some should be read when facing a particular issue.

Most (quite all) contents of this article are personal opinions.

A journey in software development.

One of the questions, maybe THE question, a young software engineer has is: what can I do to become a good and great software engineer?

The obvious answer is: training and practice.

So, how can we train and practice?

Basing on my experience, I can say that scholar and academic formation is basilar. This doesn't mean that a software engineer must attend high school or college. Absolutely not. This means that a software engineer should embrace the basilar topics of software engineering, no matter about how this is achieved.

Relational model, complexity, binary logic, computer architecture are just some of the basilar topics that are angular bricks on how to get broad knowledge. Just to give an example, it is useless to read "Designing Data-Intensive Applications" without the basic knowledge about databases, transactions, ACID properties and how storage works.

Now, let's assume a just graduated (or self-taught) person wants to start its journey in professional software development. What are the next steps?

Training and practice.

It's easy to understand that the training part will continue all along the entire career. And it's also easy to understand that the practice strongly depends on where one will work. It is also possible to practice in spare time, but this experience is marginal, either because of limited time to dedicate, either because the limited scope of projects. If you're in a company with no possibilities to practice, and you want to, then ask for it. If this doesn't happen, then leave.

For this reason, in this article we will focus on the training part.

(My) Correct order of things.

In the first part of my career (early 2000's), when internet was not a thing, I read more programming languages books: PHP, Java, Visual Basic and so on. It was not yet possible to train on specific languages through internet tutorials. Knowledge was almost exclusively based on books.

In this article, I'm going to focus more about late 2010's and 2020's books. Or books that, despite their age, are still topical.

It's interesting to note how books content evolved from early 2000s to 2020. At first they were more focused on programming languages. Then they moved onto software engineering and (more or less) illuminated authors' ideas.

The following list is in the order I would have liked to read them. The same order is used for the remaining paragraphs of this article.

Fundamentals.

1. A Philosophy of Software Design.
2. Unix Design Principles.
3. Pragmatic Programmer.
4. Clean Code.

Medium.

1. Release It!
2. Domain Driven Design (Only the first part).
3. Designing Data-Intensive Applications.
4. Infrastructure as Code.

Microservices.

1. Building Microservices.
2. Microservice Architecture.
3. Building Evolutionary Architectures.
4. Monolith to Microservices.

I will report some personal general considerations and interesting topics worth to note.

I will also report topics that I've guessed before finding them on books. This means that the specific topic has a sort of common sense meaning. For this reason, it is worth to mention them to put specific focus.

The list is divided into Fundamentals, Medium and Microservices sources. Fundamentals should be known deeply and should be refreshed every time we have a doubt in our code. I suggest to read them twice or more. Medium are worth to be kept in mind and pull out for specific tasks. Microservices are a set of sources that address... well, microservices. The trend of the moment. I decided to list them here because I think multi-level modularity expressed by microservices is a concept that will be actual for a long time.

I am quite sure that in the near future the paradigm will shift to something different from microservices. But the lesson taught by microservices about how to manage modularity from single unit-test element to broad integration-test system will last for a long time.

In this article you'll not find the Microservices books paragraphs. This choice is because they are very advanced topics and are worth reading just if you'll work with microservices, or you want to have a direct touch with modularization at scale.

Being said that, let's start with the first book first.

Fundamentals / A Philosophy of Software Design (John Ousterhout).

In this book the author reported a list of lessons he gathered during his academic career as Software Engineering course professor at Stanford University.

The best words that describe this book are in the Preface:

"[In college] We teach for loops and object-oriented programming, but not software design"

The result was his Software Engineering course.

I put this book first because I think it is the missing step between academic theories and professional software development. Worth to mention that a course similar to the one taught by the author should be mandatory in each college and high school.

This "missing step" is often left to the effort of the fresh graduated engineers. After years of struggling, some good practices come to the surface. Having a book that gathers most of them is gold.

Of course, the book content follows the opinion of the author, and this is valid for quite all the books, but having a more experienced guide is better than having nothing.

A Philosophy of Software Design. Worth to mention topics.

• Pass through variables.
• Configuration parameters. How to manage configs.
• Exception aggregation.
• Software trends (agile, unit test, TDD).

Fundamentals / UNIX philosophy (Doug Mcllroy et al.).

Actually there are plenty of books about UNIX philosophy.

I did my journey through set of principles/frameworks: TDD, SOLID, GRASP, 12-factors and others. I am strongly convinced that all of them are direct consequence of UNIX philosophy. Most of them are "just" a rewriting in a different field of software development. Having in mind these principles, and the UNIX development approach, makes natural to embrace the others principles and frameworks.

UNIX philosophy born back in the '70s. But time have no effect on it. It is still actual and this is its power.

UNIX philosophy has been summarized several times during the years by notorious authors. These summaries are worth to follow and brings innovation to the table. In my opinion, the best thing to do is to deeply know the original definition depicted back in 1978 by Doug McIlroy:

1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".
2. Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.
3. Design and build software, even operating systems, to be tried early, ideally within weeks. Don't hesitate to throw away the clumsy parts and rebuild them.
4. Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them.

These points are the tools to build a good feeling about what is wrong in our code. With that in mind, it is easy to find alignment with good practices reported in quite all the books about software programming and frameworks.

UNIX philosophy. Worth to mention topics.

I think that UNIX philosophy is too stately and insightful to be "guessed". I can say that my guesses in other reads are just the consequence of understanding UNIX philosophy. UNIX philosophy is worth to mention in its entirety.

Fundamentals / Pragmatic Programmer (David Thomas and Andrew Hunt).

As it is pointed in the preface this book is an everlasting in the programming community. Several editions have been published during the years.

I think this book is the natural extension of the UNIX philosophy. In short: UNIX philosophy principles applied to real development world.

This book seems to come out from the future even though it comes from the (far) past. Because in software development, and life, what you saw in the past is what you'll see in the future. For that reason the authors decided to gather all the good and bad practices they've encountered during their (amazingly long) careers.

The book brings also very detailed code examples and describes side effects and drawbacks of choices.

I've enjoyed a lot the fact that the patterns and behaviours described are really close to the real (programmer) life. There are no theoretical digression about abstract patterns that you'll never see in your life. Everything is sticky to real problems in real life. The fact that the patterns described here are so different from most of the OOP patterns should suggest a lot about academic classes. See the OOP paragraph below.

Pragmatic Programmer. Worth to mention topics.

For this book the same rule about UNIX philosophy matters. I think the entire book is worth and insightful. I've read it after decades of experience and I got an epiphany about revelations in all the pages.

It was like someone that was confirming and revealing a lot of good habit and guesses that it was really hard to reveal in a management-driven world. This because (bad) management is not really opened to listen to blockers and struggles behind technical development.

Fundamentals / Clean Code (Robert C. Martin).

One yelling you might have is:"why Clean Code is not in the first position? Writing clean code should be the first issue for a developer!"

The answer is simple: before even have had the chance to read about fundamentals ideas on software programming which kind of code a developer is supposed to write? What's the matter of writing shaky code but in a clean way?

I think the three sources above are fundamentals to write meaningful code. Now it's the time to write it in a clean way. Of course, you should, you must, start writing code before finishing reading books. Even if not clean code.

In Clean Code there is an extensive amount of good practices that are worth to follow. These practices spread from code syntax to concurrency management. A good exercise is to pass through the Code Smells appendix and check if we're able to spot the problem or mind a case where we faced the same issue.

Being able to "feel" clean or dirty code is one skill that every developer should have. This book explains the basis to start building that feeling in a guided way. This saves a lot of time.

Clean Code. Worth to mention topics.

• Meaningful Names.
• Small! (Do one thing).
• Reading Code from Top to Bottom: The Stepdown Rule.
• Don't Repeat Yourself (DRY).
• Data/Object Anti-Symmetry.
• Law of Demeter.
• Write You Try-Catch-Finally Statement First.
• Exploring and Learning Boundaries.
• Single Responsibility Principle. In my opinion this book provides the best definition of this: one reason to change.
• Simple Design Rules.

Medium / Release It! (Michael T. Nygard).

Now that we handle the fundamentals we're ready to deliver our code to the real world. Yes, but how?

Release It! provides a great guidance on how to deal with the production. What I've found very insightful is the broad set of examples and "close to the metal" dissertations. It is a very pragmatic source and talks directly with the code.

The topics covered into the book represent almost the entirety of the cases of building and operating a production system. It starts listing some stability patterns and anti-patterns, the following part is about the prerequisites for a production-ready application. The remainders are about how to deliver (deploy) the system and how to face with systemic problems.

This read is highly suggested and useful to properly operate a production system.

Release It! Worth to mention topics.

• Getting Thread dumps.
• Socket-based Protocols.
• HTTP Protocols.
• Session Tracking.
• Unbounded Results Sets.
• Stability Patterns.
  • Timeouts.
  • Circuit Breaker.
  • Bulkheads.
  • Fail Fast.
  • Let It Crash.
  • Handshaking.
  • Test Harnesses.
  • Decoupling Middleware.
  • Shed Load.
  • Create Back Pressure.
  • Governor.
• The OWASP Top 10.
• Schemaless Databases,
• Painless Releases (blue/green deployments).
• Create Options.
• The Simian Army.

Medium / Domain Driven Design (E. Evans).

This book is one of the most popular amongst developers. Quite all of us have read it.

I have strong opinion on it, as I have strong opinion about OOP. The content of this paragraph is strongly opinionated. I expect many of you will disagree about what is written here.

Domain Driven Design brings into the table one of the most important topics in software development: the ubiquitous language. In this book it is reported for the first time the importance of having a well-known and shared language amongst the team and the company. The vocabulary shall be used by tech, non-tech and all the people in the company.

I think this concept is really important. Never forget it.

The rest of the book is focused in refining knowledge, refactor the code and apply specific patterns related with OOP.

My point is that the book is too focused on OOP. When the first version came out in 2004 it was the golden age of OOP and this setup was normal. In my opinion it is worth reading the whole book to get the touch with the process of refactoring and extending our software. In doing this we should keep in mind that the book works with the particular case of domain modeling with OOP. This might not be a valid prerequisite anymore.

The reported patterns will be useless if you use others modelization approaches (functional, modular) but the general dissertation is still valid.

Domain modeling is still a super-valid concept but shall be applied in different manner outside OOP. In my opinion the best approach is to follow a light representation with types and the UNIX principles.

Medium / Designing Data-Intensive Applications (Martin Kleppmann).

A must-have book.

Within this essay is reported the State of Art about data engineering. Starting from the basis to advanced topics in data management.

It is worth nothing that this book explains a topic and suggests some existing market solutions that provides the data management feature exposed.

I use it as "goto" book when I need to address a data engineering problem. I am not a data engineer, but, since every real system needs its data management solution, having this book in the shelf lets me free my mind.

Designing Data-Intensive Applications. Worth to mention topics.

• NoSQL.
• Relational Versus Document Databases Today.
• Graph-Like Data Models.
• Hash Indexes.
• B-Trees.
• Column-Oriented Storage.
• The Unix Philosophy. (Not to say)
• Idempotence.

Medium / Infrastructure as Code (Kief Morris).

This book is the first source that addresses the problem of modularization in a practical way.

Infrastructure as Code (IAC) is a basilar topic. Having in mind that the infrastructure upon we build our services can be expressed with code (manage everything with text, as a good Pragmatic Programmer) before reading microservice pattern contents is a boost.

The book deep dive into advanced topics of IAC and presents several products and technologies. In my opinion, given the broad topic, mastering IAC and cloud is a full career path.

In the different Agile derived organizational models developers are often accountable to manage the software end-to-end. A developer should work on devops tasks and related skills are required. This book is a must-have. Even though, in my opinion, it is impossible for developers to maintain enterprise-grade systems, it is a full-time duty and the required competencies and skills fulfill the single person capacity.

Infrastructure as Code. Worth to mention topics.

• Goals of Infrastructure as Code.
• Challenges with Dynamic Infrastructure.
• Choosing Tools for Infrastructure as Code.
• Practice: Manage Application Code and Infrastructure Code Together.
• Continuous Integration (CI).
• Continuous Delivery (CD).
• Low-Level testing.
• Managing External Dependencies.
• Using a Local Sandbox.
• 12-Factor Applications.
• Zero-Downtime Changes.
• Measuring Effectiveness.

Modularization problem.

Modularization is the main concept a software engineer should have in mind. Modularization problem is the common trait of quite all the software development sources (articles, book). Topics from programming patterns to microservices architectures are all addressing the same problem: modularization.

Going back to the UNIX principles, modularization problem is expressed in the first point:

1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".

Modularization is fundamental because it represents the barebone of divide-et-impera strategy: reduce problems to simple concepts. Mastering modularization let us solve real-world problems.

The modularization problem in the State of Art of software development has been addressed, and still be, using microservices. But, in more recent time, with cloud technologies (lambda functions).

And OOP? What about OOP?

I have strong opinion on OOP. I know that many of you have been taught about OOP in college, high school and courses. Me too.

I think OOP is meaningless for professional software development.

(If you're not convinced with this statement please comment below AND put a Github/Bitbucket link with an example where OOP simplified your life)

I will publish an article about this topic and OOP design patterns: the evil of evils.

Contributions.

Do you have insightful books to suggest? Post it in the comments, I'd like to read them and feedback with an opinion.

Comments

[Julie Heard]

Have you read the Phoenix Project? I thought that was an interesting view on switching to CI/CD in a company where there are lots of integrated complex systems. If you are new to CI/CD well worth a read.

[ccarcaci]

I've just finished reading it.
Absolutely insightful. The story part was like "oh yes, me too".

What' I've found most worth was the "The Three Ways" part. Which I think is the sugar of the book and ready to be applied in a practical context.

[ccarcaci]

Do you mean the Phoenix replacement pattern?
It is the evolution of Blue/Green deployment. I think it's very interesting although the data migration and zero-downtime on data topic is still not very well covered and is the big blocker for such patterns.

[Julie Heard]

This one:
amazon.co.uk/Phoenix-Project-DevOp...

I have read the Unicorn project (which is the 2nd book) but I really didn't rate the second one. It was not as interesting and re-hashed the same material. First one is a decent read though.

[ccarcaci]

Thank you for your suggestion. I've added it to my reader :)