The Four Horsemen of Software Complexity — Architecture Decision Records to the Rescue

When we try to reduce complexity in software development — we always address accidental complexity.

Accidental complexity refers to the complexity that arises from the way a system is designed or implemented, rather than from the inherent nature of the problem being solved. This type of complexity can often be avoided by making design decisions that are simpler and more straightforward.

Essential complexity, on the other hand, is inherent in the problem being solved. It cannot be avoided and is a necessary part of the solution.

Let’s take a look at several potential sources of accidental complexity in software development:

1. Over-engineering: when a system is designed with more features or functionality than are actually needed, it can introduce unnecessary complexity.
2. Poorly chosen abstractions: abstractions, such as design patterns or software architectures, can help simplify complex systems. However, choosing the wrong abstractions can actually add complexity, rather than reducing it.
3. Unnecessary dependencies: adding code dependencies, such as external libraries or frameworks, can provide useful functionality, but it can also introduce complexity. When dependencies are not carefully managed, they can become a source of accidental complexity.
4. Inconsistent or conflicting design choices: when different parts of a system are designed in incompatible or inconsistent ways, it can introduce complexity. For example, using different programming languages or frameworks in different parts of the same system can make it more difficult to understand and maintain.

By understanding the sources of accidental complexity and taking steps to minimize it, it is possible to build software systems that are more maintainable, more prepared for the changes, and easier to work on.

---

Let’s first take a closer look at these sources of the problem and then get familiar with the practices that might help to mitigate accidental complexity caused by these reasons.

Over-engineering

When a system is over-engineered, it may include features that are not needed or are only used in rare or edge cases. This can make the system more difficult to understand and maintain, as developers must spend time and effort understanding and working with unnecessary features. In addition, over-engineering can lead to code bloat, where the size of the system grows unnecessarily, which can make it more difficult to navigate and debug.

Poorly chosen abstractions

Abstractions, such as design patterns or software architectures, can be a useful tool for simplifying complex systems by providing a common language and framework for understanding and organizing the various parts of a system. However, choosing the wrong abstractions or using them in an inappropriate way can actually add complexity to a system, rather than reducing it.

For example, using an overly complicated or poorly understood design pattern can make it more difficult for developers to understand and work with the system, as they must spend time and effort learning and working with the abstractions. Similarly, using an architecture that is not well suited to the needs of the system can introduce unnecessary complexity and make it more difficult to maintain and scale the system.

Unnecessary dependencies

Code dependencies, such as external libraries or frameworks, can be a valuable resource in software development, as they can provide useful functionality and streamline the development process. However, when dependencies are not carefully managed, they can also introduce complexity to a system. This is considered accidental complexity, as it arises from the way the system is designed or implemented, rather than from the inherent nature of the problem being solved.

Unnecessary dependencies can contribute to complexity in a number of ways. For example:

1. Size and complexity: adding unnecessary dependencies can increase the size and complexity of the system, as developers must include and manage additional code in the project.
2. Compatibility issues: dependencies that are not needed may not be compatible with the rest of the system, which can introduce complexity when integrating them into the project.
3. Maintenance and updates: unnecessary dependencies may require regular updates and maintenance, which can add additional overhead to the project.
4. Risk of security vulnerabilities: unnecessary dependencies may introduce security vulnerabilities to the system, which can increase the risk of a data breach or other security incident.

Inconsistent design choices

Inconsistent or conflicting design choices can contribute to complexity in software development by making it more difficult to understand and work with the system. When different parts of a system are designed in incompatible or inconsistent ways, it can make it more challenging for developers to understand how the system fits together and how to make changes to it.

For example, using different programming languages or frameworks in different parts of the same system can introduce complexity, as developers must switch between different languages and frameworks and adapt to different conventions and APIs. Similarly, using different design patterns or architectures in different parts of the system can make it more difficult to understand and work with the system as a whole.

So how do we fight these four horsemen of software complexity?
All these problems have one thing in common — they are the result of chaotic, ungoverned decisions and "accidental" software architecture process.

Architecture Decision Records

One tool that I found really useful in my experience to help with it is Architecture Decision Records.

Architecture Decision Records (ADRs) are a way to document important design decisions made during the development of a software system. They are used to capture the reasoning behind these decisions and the trade-offs that were considered, as well as the consequences of the decisions. This documentation can be useful for a number of purposes, including:

1. Providing context: ADRs can provide context for developers working on the system, helping them to understand the reasoning behind certain design choices and the constraints that were considered.
2. Facilitating communication: ADRs can help to facilitate communication between team members and stakeholders by providing a clear and concise record of the decisions that have been made.
3. Facilitating future decision-making: ADRs can serve as a reference for future decision-making, helping to ensure that new design decisions are aligned with the overall architecture of the system.
4. Improving transparency: by documenting design decisions, ADRs can increase transparency and help to ensure that all relevant parties are aware of the decisions that have been made.

To create an ADR, it is important to include a clear and concise description of the decision that was made, the context in which the decision was made, the alternatives that were considered, and the consequences of the decision. It is also important to include any relevant supporting documentation, such as diagrams or code examples, to help provide context and clarify the decision.

Summary

A huge advantage of Architecture Decision Records is that one ADR document can be really small, and doesn’t take much time to write down or review. Yet it gives you a whole versioned history of how your project architecture developed over time and gives an insight into all places where accidental complexity was introduced and what was the reasoning behind doing this. It helps you to keep consistency with your decisions and critically revisit them in face of new requirements emerging.

There are several interesting links on the topic of ADR:

• https://adr.github.io/ — homepage of a community popularising ADR technique, improving and introducing tooling for it and sharing knowledge and experience.
• AWS, Google, RedHat — articles where the companies share their experience and tips in ADRs.
• A very interesting serverless cloud architecture example project where Google Cloud team shows not only the tech, but the development process as well, and a part of it are some really good real-world Architecture Decision Records examples.

Comments

[Marshall]

Thank you for this article.

I can relate to facing accidental complexity in my past experiences (being one of the culprit more than once). As @bukharovsi pointed it in his comment, I think documentation often points out how rather than why. In my opinion, this is a mistake for technical decisions since you can find out how by yourself looking at the code but never understand why it is done this way. This leads to rearchitecturing things and multiply design choices leading to even more complexity as your article says.

I often read technical specifications that focuses on what will be done (applying this pattern, creating those classes, even sometimes modifying line 22 in file xxx). This misses the most important thing : why did you decide to do this ? What alternatives have you considered ? What tradeoff did you feel as acceptable ? I'd rather read ten ADRs than a hundred technical specifications like those.

[Sergey Bukharov]

I like the fact that the biggest part of the article explains why, and only a little one explains how.
It correlates to the nature os ADR!