Communication is key (to good software)

What I came to realize lately is this: Communication is the single key factor of creating high quality software. Why? Let me try to explain.

Stages of software development

The very first thing to acknowledge when building a software is: Software Development is not just about writing code (maybe for many this was obvious, but it wasn't for me as a coder). If your team is solely build from "Software Developers" (as in people who write code) the project is bound to fail. Instead each software project must go through the following stages, which each require a unique set of skills, that are pretty much impossible for a single person to aquire. These stages in their most basic form are:

1. Planning
2. Building
3. Testing
4. Deploying
5. Monitoring/Maintenance

In a corporate environment it is pretty much inevitable for any software to avoid any of these stages, which all build upon and are interdependent on each other. You can't build software without planning and you can't test, deploy or monitor software that has not been build. At the same time any built software that is not tested, deployed or maintained is obviously very unlikely to generate any long-term value.

To emphasize the importance and the vast set of skills necessary for each stage I would like to explain each in greater detail:

Planning

The planning stage sets the foundation for the software project and is therefore from utmost importance. Just like any house built on a loose foundation will sooner or later collapse, any software project is destined to fail without proper planning. No code should be written before planning.

The planning stage requires specifying the software project from a user perspective. There must be a clear vision for the software, which is understood by all team members and that - when executed through code - solves a clear problem and generates value to the company.

This stage requires strong communicational skills in order to understand the needs of the customer, how those needs can be served by the software product and communicate that to the other team members. Basic technical skills from each other stage can be hugely beneficial in order to balance the needs of the customer and the rest of the development team, while team leading skills are needed in order to gain the trust and backing of the other team members when making decisions.

Building

With a proper plan, and clear vision of the product, it's time to start writing code. This is done most of the time by people called "software developers", although (as should be clear by now) much more people and roles with different skill sets are involved during the development of software.

Writing code is a complex, lengthy and expensive task. In order to create a code-base that is maintainable, scalable, perfomant, testable and adaptable many skills must be aquired, including:

• Programming language(s) for front-end, back-end
• Database design and query languages
• Software design and patterns
• IDEs and version control (Visual Studio, IntelliJ, Git, etc.)
• A multitude of ever-changing frameworks
• Understanding of computers work in general
• Clean code principles

Since technology and frameworks keep evolving and changing, the art of writing code is a life long task of learning, adapting, trying and refactoring.

Testing

To prevent bugs, potentially business-critical failures, slow performance and ensure a high quality in general, each change to the product must be tested thoroughly. With a growing code base, this can only be done by automating as much of the test suite as possible. The software must be planned and designed for making automated tests easy, which requires knowledge in:

• Software design
• Unit, integration and system testing
• Test automation frameworks (like JUnit, Selenium, etc.)
• TDD, BDD and many others

Since a lot of time can be spend on testing, it's doubly important to have clear specifications for the software and features in development that are as likely as possible to deliver value to the customer. Otherwise a lot of time can be wasted on creating a automated test suite for software features that are unneccesary, unwanted or even contraproductive.

Deploying

No software can generate any value to the company, if isn't deployed aka rolled out to the customer. Sadly this stage is more often than not overlooked during planning and therefore unnecessarily causes stress, frustration, overtime for the employees and downtime for the customers.

In order to prevent this, the software must be planned and developed with an easy and automated deployment in mind. This requires knowledge of technologies such as:

• Server administration (Linux, or if you're crazy: Windows Server)
• Shell scripting (bash, Powershell, etc.)
• Computer Networks
• Containers and Orchestrators (Docker, Kubernetes, etc.)
• Virtual Machines
• Configuration Managment Tools (Ansible, Chef, Puppet, etc.)
• Cloud Services (AWS, Azure, Google Cloud, etc.)
• Build Server (Jenkins, Azure Pipelines, Gitlab CI, etc.)
• Continuous Integration/Delivery/Deployment in general

Monitoring and Maintenance

To keep software running, identify problems with the product early and keep in touch of the needs of the customer, every deployed piece of software must maintained and monitored. Otherwise performance problems might cause downtimes with a growing customer base, or customer might become increasingly unhappy with the product and stop using it.

Mechanism for effective monitoring and maintance must planned and developed during the early stages. In addition - as with any other stage - there a lot of technologies to learn and skills to aquire:

• Server administration
• Software monitoring tools
• General customer support and relations

The key to build a successful product

With all stages of software development in mind, what is the key to build a successful product? In my mind the answer can be clearly summed up in a single word: communication. In order to build a successful product, you must build a orgisational structure that makes it as easy as possbile for information to flow between each and every the stage of developemnt at all times.

The reason is simple: All stages depend and influence each other, while requiring unique sets of skills, talents and knowledge, which are impossible to be obtained by a single person. So learnings and information from each stage must be continuously be synched between the team members through communication.

It is important to keep the communication as immediate as possible. Each detour from where the information is actually needed, bears the risk of losing or altering that information, simply because we - as humans - are imperfect. Each form of communication between humans is error-prone and always carries the possibilty of misunderstandings, delays or just forgotten memories.

One easy thing to do to improve existing teams is to track how information is transported between the stages and where that information is actually transformed into actual work. Track the source of the information, count the amount of people involved in transporting the information and the amount of time needed.

In typical, siloed work environments its not untypical for information to make its way through 2 or 3 delegates (often in the form of managers) over the course of many days or even weeks ("let's wait for next Jour Fix!"), before it can make its way to the person, that actually transforms that information (or a very likely altered version of it) into work. Often the work is also delayed further, since each silo plans work independently and is very unlikely to have time for the problem at hand.

If you can cut down on the time and persons necessary to transform the information into actual work, the software product will improve.

Sadly, more often than not, organizational structure discourages frecent communication by building building barriers, which make it very hard and frustrating to build software. In my experience bad communication tends to be the main cause for frustrated employees and the single root cause for bad products.

Bad communication as a system: The waterfall model

A very infamous process framework, that systematized bad communication, is the waterfall model (as beautifully drawn by me):

At first glance it doesn't look too bad since every stage is accounted for, right? But the big problem is how communication is supposed to flow by the system: One-directional from one stage to another. Communicating "back up" or from any stage that is not next to each other is not encouraged. This means that each error made in the previous stage gets worse by the following, while its made artifically hard for information about problems and their solutions to make their way back in order to be prevented in the first place, e.g. during planing or code writing.

A little bit better is the extended waterfall model, which allows information to flow back between the stages.

Still, any problem encountered during deployment or monitoring has to make its way back through each and every preceeding stage. This is bad because the information has to make "detours" and does not get where its needed immediatly. The ("Chinese whispers")[https://en.wikipedia.org/wiki/Chinese_whispers] effect takes place and with each detour through a unneeded stage the information is delayed, falsified or might even be lost due to shifting priorities.

What to do instead

That the waterfall model is flawed isn't a new insight by any means. And people have been starting to adopt more "agile" frameworks like Kanban or Scrum, which - when implemented properly - can eaze the communication by making work visible (through kanban boards for example) and suggesting teams consisting of specialists for different stages (Product Owners for the planning stage and Software Developers for writing code).

But instead of blindly following "Scrum" or "DevOps" while spending many dollars for expensive courses, you could simply ask yourself the following question when building your software development team:

What can I do to improve the continuous communication between all stages of development?

Things to do include:

• Build a autonomous and interdisciplinary team - starting with the planning stage - that covers all stages and is focussed on a single, well-defined project
• Have daily team meetings to keep everyone in sync. The meeting ensures that every new piece of information or problem will be discussed no later than the following day
• Cut meetings that serve no clear purpose or consist of disfunctional groups (meaning that not all people needed to solve a problem are present)
• Keep at least 50% idle time (as in time without meetings) for everyone involved in general. It must be possible for everyone to ask questions and get feedback from any stage. Busy calendars discourage questions - even though there might be some idle time here and there.
• Keep work visible by using a single kanban board for all team members
• Have a single set of tools used by all team members. Have at least readable access for everyone in the team.
• Release frequently for a tight feedback loop (which is communication) from the customer. Keep in mind that stages repeat.
• Automate as much as possible, e.g. tests, deployments. This will allow you to release often and reduces the amount of handoffs and communication needed. Communicate how the automatic release process works instead.
• Plan with technical debt and learning in mind

I feel that, as long improvements are focused on communication, the software quality and value will improve. It is not necessary to do X just because a process framework told you so. But Kanban, Scrum, DevOps, the agile movement in general all try to formulate frameworks, that make communication easier and more immediate on an organizational level. And that makes them a good starting point, although improvement is not achieved by following any named methodology, but by focussing on communication instead.