by Joseph Hillier-Davis – February 5, 2020
Intro
I’m a fan of short practical IT books, packing relevant information into a concise digestible form, without losing readability (Martin Fowler’s UML Distilled is one fine example). Having recently purchased a physical copy of Building Evolutionary Architectures it was a pleasant surprise to be delivered a slim book, at less than 200 pages. A flight from London to our company HQ in Seattle offered the ideal opportunity to discover what it had to offer.
"Building Evolutionary Architectures" book
Building Evolutionary Architectures was published in Oct. 2017 and is the work of three employees (some now ex-employees) of ThoughtWorks; Neal Ford, Rebecca Parsons & Patrick Kua. Despite being a collaborative effort the book remains fairly consistent in tone, albeit with some repetition, with each author contributing their own distinct perspective around a common core set of novel ideas.
Need
Evolutionary Architecture outlines a different approach to software architecture better suited to the Agile era. Rather than impose a design, or simply try to let it emerge, a guided approach is advocated instead. Some useful contextual understanding & history of other approaches is also provided in the book. This history contrasts traditional approaches with the recommended evolutionary approach, establishing the underlying need. In addition, it explains why an evolutionary architecture approach was not previously possible. In the last millennium, prior to the prevalence of open-source options & commodity hardware, software architects & developers were much more constrained by business cost (e.g. of proprietary licenced vendor solutions & expensive specialised hardware driving applications to share precious resources such as proprietary relational databases)
At one architectural extreme lies the once traditional approach of BDUF (Big Design Up Front). This takes the architectural metaphor borrowed from the building construction industry most literally, expecting software architects to create a near complete initial blueprint that provides an accurate solution prediction for subsequent implementation work to then slavishly follow. However, for most software endeavours the need to support ongoing change (e.g. respond to business threats & opportunities), makes this approach highly unreliable. Many of the predictions will simply be wrong, since software development is a creative endeavour of design & discovery. A number of significant challenges & changes simply cannot be reliably envisaged from the outset (and the cost of trying to uncover them is prohibitive or a foolish quest, since they aren’t static). BDUF offering a false sense of certainty with often quickly evaporates as reality impinges. Plus, the shortcomings of design choices imposed too early often become apparent late on, when they are difficult, costly or untimely to reverse or amend.
At the other extreme lies a complete lack of perceptible architecture, such as from a “cowboy (and/or cowgirl?) coding” approach, that culminates in the BBoM (Big Ball of Mud) anti-pattern. Rampant coupling between software units (e.g. classes) & data (e.g. via shared database tables) accumulates over time. Inexorably this leads to a system that becomes harder to reason about & change. Even small changes can have far reaching ripples of (often unforeseen) impact throughout the system. Developers become reticent to make all but essential alterations as a consequence as fear of change set-ins. Ultimately this culminates in a brittle system, where change becomes actively avoided & the softness of software becomes lost. Change becomes too expensive & slow to accommodate efficiently. Rigidity replaces the initial flexibility and the business pays the cost in terms of loss of competitive edge.
Both these extremes are detrimental to any business fundamentally underpinned by software, plus disheartening to the software teams that must contend with them.
In between these contrasting approaches the Agile Manifesto’s Principles mentions architectural emergence. Specifically principle #11 seem to advocate reliance on emergent design:
The best architectures, requirements, and designs emerge from self-organizing teams.
However, as author Neal Ford points out, if you accept the notion of architecture as the important & hard to change choices, then you don’t really want to rely purely on emergence as a driver for these. Delayed choices are pragmatic, due to the cone of uncertainty & unknown unknowns, to reduce early accumulation of hard to reverse choices. However, some choices are practically unavoidable initially (e.g. choice of programming language, frameworks/libraries). Some skeletal outline from which to start is near unavoidable.
Donald Rumsfeld on "unknown unknowns"
In most current software implementation scenarios this book advocates Evolutionary Architecture. This involves guided selection of key architectural characteristics (e.g. good design, scalability, reliability), pertinent to the specific system under consideration, distinguishing this approach from the emergent alternative. Automated verification of these characteristics ensures their constant consideration & protection during ongoing development, in contrast to imposing specific implementation choices upon development teams & architectural governance from afar (without easy verification).
Core Idea – Fitness Functions
At the heart of the book is the idea of using a suite of quantifiable tests to drive the evolution of a software system in a desirable direction. Furthermore these tests protect those aspects from deterioration once set in place. The book dubs these fitness functions, in line with the evolutionary theme, borrowing more terminology from Biology. However, this is more akin to Artificial Selection / Selective Breeding (e.g. dog breeding or farming selection of livestock & crops over successive generations) rather than Natural Selection (non-human guided e.g. via environmental factors such as changing habitat) with variation (via mutation).
Architectural fitness functions can be applied systemwide via a variety of approaches, such as automated tests (unit, integration, functional etc.), software development metrics (e.g. cycle time), contract tests (e.g. Consumer-Driven Contracts for API) and experiments running in production (chaos engineering).
Different types of fitness function are presented, covering four categories, as outlined in the diagram below:
Types of architectural fitness function as a grid
Enablers
Evolutionary architecture is enabled by a set of supporting existing practices:
Deployment pipelines
The Inverse Conway Manoeuvre
Appropriate coupling
Bounded contexts
The ability to react to change requires frequent feedback. Continuous delivery (or alternatively continuous deployment) provides the opportunity for regular feedback via fitness functions within its deployment pipeline. Deployment pipelines provide the opportunity to ‘slot in’ automated check via architectural fitness functions. For instance automated tests to ensure cyclomatic complexity remains beneath a threshold, with this number gradually lowered to a desirable level, to incrementally reduce complexity as one measure of software quality. Alternatively legal checks for changes to open-source library licences, with automated detection of changes (e.g. via a checksum alteration) providing alerts for manual inspection. A more extreme example of a continual holistic architectural fitness function is Netflix’s Chaos Monkey to drive resilience and fault tolerance from experiments performed in production environments.
Other wider practices help enable appropriate coupling of software components. One way to avoid organisation structure dictating system architecture is to instead intentionally structure teams around a desirable software architecture. ThoughtWorks have called this approach the Inverse Conway Manoeuvre. Cross-functional product teams, rather than project teams, are generally advocated here, specifically created around the desired architectural components or key customer driven business processes. This encourages aligned autonomous teams, with few interdependencies, whose own team goals align with desirable business outcomes. (I recommend another very short O’Reilly book here, Designing Autonomous Teams and Services , which delves deeper into this emerging area of Socio-Technical Architecture).
Thinking about relevant architectural elements in terms of deployable units is also encouraged. To work effectively the Evolutionary Architecture approach requires the possibility for guided incremental change. Unnecessary or exuberant coupling (such as between classes or shared data sources) inhibit evolvability due to the impact & risk of changes. Excessive dependencies need to be tackled, e.g. tempered by modularisation (via libraries, layers, components & services), to provide an opportunity for the evolutionary approach advocated to be applied.
Beyond appropriate component level coupling, functional cohesion is also desirable (e.g. deployable units change for the same type of business reason). Following the concept of Bounded Contexts from DDD (Domain-Driven Design) also aids architectural evolvability, by forming relevant grouping of software as deployable units that can then evolve independently, at different rates, targeting different desirable architectural characteristics (which otherwise may potentially conflict e.g. high performance & extreme scalability. Or, to find the right trade-off where these must cohabit via gradual fine tuning).
Scientific Metaphors
Heavy use of scientific metaphor continues beyond just the book’s title into its content.
Correlation with biological evolution is much weaker there than with Genetic Algorithms, for example, where fitness functions select from mutating populations of candidate solutions based on their applicability for a specific goal. In contrast, fitness functions as defined in this book do not choose between different potential solutions. In the Evolutionary Architecture case we don’t have the luxury of sufficient available business time or funding to explore the mutating generations that more closely model biological evolution.
Rather than simulating the mutation with natural selection found in biological evolution, this book’s fitness functions instead act as architectural acceptance criteria applied to a current snapshot of the system under development.
Scientific metaphor becomes even more strained with use of the term Quanta to express a independently deployable component (an architectural quantum is an independently deployable component with high functional cohesion e.g. microservices have small quanta compared to monoliths). Architects are advised to find the appropriate trade-off between architectural quanta & their cost, with the right balance depending on the particular business context. Although a seemingly useful concept for comparing different architectural approaches & how readily Evolutionary Architecture techniques can be applied to them, this label initially seems too forced to me (very loose in association to its appropriation from Physics), so it will be interesting to see whether it catches on.
Challenges to adoption & success
The book is largely theoretical, providing an overview with justification and suggestions rather than lots of readily applicable examples. Relatively few practical scenarios are presented. Real world challenges, such as approaches to convince a risk averse business (e.g. of using Chaos Monkey like approaches in production) are skimmed over or left to coverage elsewhere. Although the last chapter provides some guidance on putting things into practice the coverage is rather shallow. I was left wondering about overcoming common initial obstacles to adoption (such as inappropriate team structures or practices – although admittedly solutions to address these are likely found elsewhere). I also wondered about challenges to success of the Evolutionary Architecture approach (such as potential misinterpretation leading to inadvertent alteration/removal or contention in more complex cases of shared ownership of fitness functions). However, it is perhaps unfair to expect comprehensive coverage in an innovative area, plus maintain a nice slim volume.
Conclusion
I found Building Evolutionary Architectures an engaging and thought provoking read, synthesising a persuasive narrative from current themes (e.g. software’s empirical trend).
I look forward to ‘Evolutionary Architecture Applied’ or similar future cookbook that provides deeper practical detail on putting the innovative ideas covered in this book into action, and offers best practice advice drawn from more widespread industry use. In the interim I hope to explore some more of the concepts at OpenMarket & share what we discover.
Find out more for yourself via the book or at http://evolutionaryarchitecture.com/ .
Top comments (0)