DEV Community: Steven Read

Better Software Design: Changing how we talk about software

Steven Read — Wed, 04 Dec 2024 09:33:30 +0000

Changing how we talk about software is the beginning of designing better software. Through clear definitions we have better design conversations.

To learn the foundations of better software design and architectural thinking read Better Software Design: Changing how we talk about software, in which we present the ACED Model - Architecture, Context, Engineering and Design. This builds on previous principles we outlined in Software Design: What went wrong?.

In the ACED model we focus our thinking using question words: why and how, and by considering whether they are focused on functional or technical aspects of the system. Read more about designing better software.

©️ Read the Architecture Ltd.

Software Design: What went wrong?

Steven Read — Wed, 06 Nov 2024 18:26:32 +0000

Design should be a developer’s bread and butter. Changing how we talk about software under development is the start of building better software systems. We must stop calling everything architecture, make design more accessible, and resist the urge of focusing on technology.

Design is how, who and when…

Design is how we use the system. This can be users or even systems via APIs. We can call this who. Think of this as the behaviour of the system. It can also be when - in terms of order of a process, or schedules.

Typically, though when is part of a design, it can be constrained by the architecture or engineering. For example due to data rate limits or costs.

Design isn't a set of requirements or a schematic, it is an iterative process where what is possible and thought best is improved, iterated, and captured through discussion. Not only is design a shortcut to the system that stakeholders want, design is a magic door to the system that stakeholders didn't know they wanted.

Now that we are in a post-digitisation world, more and more of what used to be design has become simply analysis, used to verbatim reproduce an old process or interface within a replacement system.

Engineering is what and how...

Engineering is the what and how (inside the system) - things like technology, algorithms, and design patterns. Also, generic resources such as databases, proxies, and compute platforms. These are implementation details. Deep dive topics which ARE useful for developers or operations in the right context, but can easily be separated from your architecture.

Engineering details matter for some stakeholders such as security or DevOps engineers, but they are a distraction in more abstract contexts such as when you are developing your application. Flagging a few of the highest level engineering concerns is a good compromise, and is something the C4 Model has handled well for structural diagrams.

Architecture is where…

Architecture is where things live (akin to city planning). This is the structure of the system, at a higher level of abstraction than engineering, thus why I describe the database as engineering. It is encapsulated within the architecture. It organises the engineering so that it effectively implements the design.

We are also making the overall picture coherent - across lots of different axes. Architecture references detail from the design, engineering, and the application's context.

Architecture is also why…

Architecture is also why we are doing things. Knowing why enables us to make decisions to change things. If we don’t know why something is in place we need to discover and assess what it was doing. For some things you will find there is no reason why, it just happened like that.

The question of why often involves the non-functional requirements of the application - the “ility” words. Reusability, modularity, testability, scalability, etc. These lead to choices in terms of engineering. Architecture is the decision, not the definition of those choices.

In architecture, we are solving problems at scale, determining approaches that usually minimise engineering while maximising value.

We are also affecting or enforcing policies, which can indeed add complexity or cost, but the policies have desirable properties such as security or availability. When architecture is successful these benefits are coming through automatically. When we are lucky, simply choosing the right tools will offer some of this value.

Iterating and capture…

Architecture benefits from fast feedback and conversations, just like design, and we capture those designs with artefacts. The same artefacts are the foundation to both design and architecture, and additional stages of enrichment can then provide additional insights in both areas. Our efforts avoid rework and allow better choices. Iterating (redrafting) and capturing (writing down to share and recall) has the same improving and accelerating effect on solutions.

We are talking about choices, and if you have fast and expressive conversations you can make a lot more choices than if you don't.

State of Play

Over the years I have seen many teams producing a lot of engineering artefacts, but much less design and architecture. People use engineering to bring favourable traits to their system, but they also use it as lipstick for a bad architecture, or to bamboozle stakeholders who never got to be involved in a design.

You can't make informed decisions around architecture without a design. When somebody asks you to “show me the architecture” of your system you need to start with the high-level design.

Teams have taken the view that by regularly releasing software they don't need a design. Regular releases provide all the feedback they need.

I hate to be the breaker of bad news, but there are much faster and less expensive ways to get a good idea of what people want, you need to move to validating through releases rather than designing through releases. So many times I have seen a development team implement a feature or functionality, and then the stakeholders just accept or tweak it due to it being "too late" to change anything. I have even seen when a feature is implemented and then after it is deployed to production the feature is then rolled back, due to it disabling important functionality that users relied on. We need more design conversations with users.

We can develop ideas faster than releases

Agile isn't about stripping the development process down to coding in small chunks, lean of everything but code. It is about feedback loops - and those small loops go all the way back to stakeholder conversations. We are once again seeing the weight engineering has acquired over design and architecture, where the majority of the feedback loops end up accumulating in operations.

Analysing the trend

First of all, there are many factors at play here. One of which is the agile manifesto, which has a blind spot in the "over" statements regarding the importance of design. There are lots of clues there - "Individuals and interactions", "Customer collaboration" - but it isn't explicit. There is also a problematic statement:

Working software over comprehensive documentation

Teams regularly misinterpret this as meaning design and documentation are not part of the process, we should only code - working software INSTEAD OF comprehensive documentation. It was further exacerbated by "Clean Code" evangelism, which explicitly demonised commenting code - it is clear how this reinforces the INSTEAD OF misinterpretation above.

A further problem is cloud certifications. We have already touched on the common misunderstanding that infrastructure is architecture. There are now many people who think a solutions architect is somebody who helps you integrate with a large cloud provider's platform, rather than integrate an application or system into your broader enterprise.

Add to that Most Valuable Player and Community Builder awards/hats and this brings further focus to engineering-focused patterns and practices. If a software engineer wants to learn for free, they will likely be attending a product-focused community, which may well have developer relations or developer advocate speakers delivering the content.

We have barely touched the surface, in that all of the above context leads to further knock-on factors within organisations, the jobs market, and even social media and conferences.

Fixing the trend

We need to get to the point where people are happy to design software, and that means to collaboratively design software, having rich conversations with stakeholders about how the software should work. The term user story does not describe a work item, or even an "as a" statement. The idea originates from the user telling you what excites them, and you exploring the design together. Collaborative design.

The only way to make a seismic change in our industry would be something of the same magnitude as the original agile manifesto, but there is a reluctance to redefine what is already there. Many people have already tried, and are just seen as trying to self-promote. This may well be the case!

The closest thing we have to a solution is domain-driven design, which emphasises the importance of stakeholder conversations, and how that feeds through into design, code and architecture.

But domain-driven design is a prickly pear, with difficult terminology, and practices that have evolved significantly over the last 21 years beyond their origins. These points came up in conversation on the fringes of Kandddinsky '24 and is a real blocker to us solving the broader software design problem in our industry.

You don't need a cutting-edge system to deliver most software requirements

Right now, until something with more magnitude is possible, my best advice for you is to invest in quality using proven products and tools. You don't need a cutting-edge system to deliver most software requirements. Use low volatility and reliable components to avoid engineering distractions. By focusing on your design and architecture instead you will create better business processes, improve your agility, and increase your solution longevity.

I have seen designed systems last 20 or more years with relatively lightweight maintenance, and they still remain evolvable. I have seen undesigned systems cancelled or declared legacy within years.

Like we have had #nosql and #nouml we need #yesdesign. We are not talking about heavy-weight model-driven development. We are talking about a sensible middle position where we evolve a solution and talk to our customers. #yesdesign

©️ Read the Architecture Ltd.

When to bring in a software architect?

Steven Read — Sun, 13 Oct 2024 13:59:21 +0000

Bringing in a software architect at the end of a project isn't architecture. It's archeology. 🤦

That’s not saying there is no value in the process, but much more value is offered from involving an architect at the early stages. It is much more expensive changing existing decisions than making new ones.

If you bring an architect into a project late on they will need to acquire context, and this will often involve learning what decisions have been made, and most importantly why. If this context isn’t something that has already been gathered and maintained then it can be a very time-consuming process. Context could come from conversations with the team, but the fog of time and staff moving on will often mean this becomes more of an archeological dig, gleaning information from chat histories, minutes, emails, git commits, source code, maybe even fin-ops reports. You will use any data sources you can lay your hands on.

The contribution is still valuable - it may take you from analysis paralysis to having light at the end of a tunnel - but the overall team's journey was more arduous.

Likewise, bringing in a software architect for only the start of a project is similarly challenging. At the start of a project we know the least about the problem. An architect will help you ask the right questions, and help you solve the right problems, but actually finding the right answers could take time. A bright and optimistic plan will likely need evolution or reconsideration in light of new information and learning. Will you have an architect involved in interpreting that?

So when should you bring in a software architect? In effect, you want architectural thinking at all stages of your product’s lifecycle. Architectural thinking is what helps us make better decisions. Decisions can easily be taken from within a silo. Architectural thinking is breaking that silo - understanding implications outside any single development team, taking into account multiple perspectives and the things that matter to stakeholders. Architectural thinking is also structuring and quantifying that thinking.

Does architectural thinking need to be from an architect? No. You can have no architect, but those perspectives and understanding of the things that matter don’t come for free, they come from breaking those silos. Can it come from a team leader? Yes, if you can balance delivery vs system quality drivers. Should it centralise decision-making? No. That’s not the point, you are looking to make the team feel more accountable for their decisions, and equipped to make better decisions.

Teaching systems thinking through computer games

Steven Read — Sun, 11 Aug 2024 18:32:31 +0000

There have been a number of times in my life when I have found that I have learned things through doing, and then later discovered empowering vocabularies and models which further enhance and improved what I was already doing. Domain-driven design was one such area, but systems thinking recently became another.

To illustrate my point we have Theme Park; a game I played in my early school years, which was overflowing in systems experiences. In the game you manage staff, set prices and salaries, build a park, stock shops, and more. Behind every computer programmer you often get a background story about computer games, so I guess I am no different.

Today we will be using Theme Park to introduce a systems thinking modelling technique – the Stocks and Flows diagram. The purpose? Well besides to apply stock and flow diagrams, it was to explore an explosive phenomenon that can occur when you have a busy system, which I will refer to as the vomit loop🤮. Some actions within this system are optimisations – staff training for instance. Others such as staff strikes could place the system in a critical position. Many of those optimisations that enable operation at scale then only make the system more susceptible to failure.

The diagram is made up of:

🛁 𝗦𝘁𝗼𝗰𝗸𝘀: Where some entity is accumulated over time via inflows and outflows. This could also be considered like a water level. A bath can be described as a stock of water

🌊 𝗙𝗹𝗼𝘄𝘀: Where a stock is being changed over time, typically measured as a rate. Water going down the plug hole is a flow

🧮 𝗖𝗼𝗻𝘃𝗲𝗿𝘁𝗲𝗿𝘀: Calculated values based on other parts of the system, or boundary values entering the model. Despite stocks and flow diagramsnot including converters in the name they are an import part of the notation

➡️ 𝗔𝗿𝗿𝗼𝘄𝘀: Representing the effects of each instance on the others

The following relationships are also illegal:

🚫 Flows cannot influence other flows

🚫 Stocks cannot influence other stocks

🚫 Converters cannot influence stocks

In my diagram I have used all of these components, and used the terms 𝘳𝘢𝘵𝘦 to label 𝗳𝗹𝗼𝘄𝘀 and 𝘴𝘵𝘰𝘤𝘬 to label 𝘀𝘁𝗼𝗰𝗸𝘀. All remaining components are 𝗰𝗼𝗻𝘃𝗲𝗿𝘁𝗲𝗿𝘀. As you can see the model is pretty large already, so for brevity, in an attempt to avoid text becoming too small, I have simplified the modelling of litter and toilet stocks.

If all of your cleaning staff go on strike, you are quickly left with no option but to close the park, or suffer significant reputational damage. Are there other solutions? Yes! Multiple cleaning contracts, or having sickbags at the exits to exciting rides and bins around the park. This is where your stocks and flows diagram comes in. You can use the diagram to explore how new measures would affect the system. The diagram isn’t to document a system, it is to explore changes to the system.

Hands-on software architecture

Steven Read — Tue, 26 Mar 2024 23:14:36 +0000

Many times I've heard it said that software architects should be hands-on - they shouldn't be in an ivory tower. They should still be coding, one foot in the game. There is so much to unwrap from this!

Hands-on

First of all, yes, architects should be hands-on. To me this means engaged, reactive to and even in front of the challenges that the business is confronted with. They should also be helping the business evade those challenges. Not asking much, eh? 🤔

Hands-on is often referred to as being a synonym for coding. This couldn't be further from the truth, as actually when you have a multi-disciplinary team, or are working across different levels of abstraction, the scope is much broader. If you use the term engaged instead, you are considering who the architect is working with, rather than what they are doing. The broader the scope, the better your awareness of context, and the better the influence of everybody in the organisation and product decision-making processes.

Examples of strong engagement include:
🔍 Finding problems in the captured requirements
🧪 Uncovering test strategy limitations or challenges
🚨 Reviewing and assessing operational tickets
🏗️ Understanding team CICD pipeline workflows
🚩 Pioneering and educating around the team's use of platforms

Examples of engagement vary up and down the architectural stack, as the artefacts, business processes and stakeholders will change, but the same principles of linking people, perspectives, and most importantly experiences remains.

Ivory Towers

Let's now address the aspect of ivory tower architecture. To me this brings about imagery of an architect in a rapunzel-like tower, absorbed in their ideas for what could be eternity. It also suggests altitude, suggesting that there is a long distance and barriers to accessibility for those around them in the business - the architect's ideas are never tested, and do not fit with a broader business strategy.

Does being a coding architect resolve these problems? Not at all, if the architect were to stay within their bubble! Furthermore, if the architect spends too much time coding, rather than providing and understanding perspectives, then the development teams and business are without an advocate. Experiences and perspectives remain unshared; they will code quite happily from their ivory tower uninterrupted.

The analogy I would use instead is to have your architects visiting and involved at the construction site, not laying bricks.

Caveat

It's worth saying I do code, work on IaC, data, pipelines, etc as an architect, and the amount of insight you gain from an hour or two in new contexts is remarkable, but that degree of insight gained is exactly the same from the same time spent across all of the other activities I outlined previously.

Coding isn't the cure - it is one of many sources of vitamin D.

Using large language models in software architecture

Steven Read — Mon, 18 Mar 2024 21:59:21 +0000

It's been maybe 6 months since I started using corporate-approved large-language models, and I thought it would be worth writing a summary of how it has affected my work.

The good:

👍 LLMs have enabled me to change how I approach some low-level software analysis tasks. I have been able to reduce the keyboard and mouse overhead of extracting requirements from existing systems, eg from SQL statements or DAOs, by using one-shot prompting. A high quality review and refactor is faster than doing the whole thing from scratch
👍 Creating API-level examples such as SQL statements or CSS layouts has worked well
👍 Image generation makes it easier to create compelling analogies for stakeholders

The bad:

👎 LLMs suffer from the same common misunderstandings or limitations as the general internet - for example, recommending performance when actually the situation requires responsiveness, and not knowing when high availability or fault tolerance are required, etc
👎 Sources/references matter. They are a non-negotiable when you are working on important decisions and enterprise strategy
👎 The more complex your context the less helpful the AI becomes. It's fine for throwing together some quick proof of concept but right now it won't help you uncover years of unintended complexity in poorly documented architectures.
👎 The last place I want to see the scaffolding of an application coming from is an LLM. Reference architectures provide much more value to your landscape
👎 Nascent concepts aren't well supported (expected, but given API-level aspects seem a strongpoint this is important if you think it will help you learn a NEW language - as opposed to a NEW-TO-YOU language!)

The ugly:

👺 Architecture is as much about why you're doing something as it is about what to do. LLMs score very badly at this
👺 The amount of low-quality material on the internet is increasing due to AI, meaning we are all having to resort to using heuristics to find good quality reference materials
👺 The internet and communities are at risk when corporations offering AI services hide the collaborative environments that have created the state of the art in our industry. Scraping laws and terms of use are presently inadequate for balancing the trade offs. We will see more paywalls, such as medium and substack, if these trends continue

What observations do you have from applying LLMs in your workplace?

Modern Approaches to Enterprise Architecture

Steven Read — Mon, 11 Mar 2024 22:30:00 +0000

Enterprise architects need to use a broad variety of techniques and perspectives, delivering a varied set of artefacts and collaborating with other parts of the organisation to provide insights which cannot be attained alone. Many activities crucially important to modern enterprise architecture are not labelled as such. Some practitioners of the new techniques don't want to be "tainted" with the EA label, and identify as other roles. Meanwhile, traditional architecture frameworks want to control and own their ecosystems. This shouldn't put off enterprise architects, or anyone working at enterprise level identifying as a similar role, from using a broader toolset.

In recent years enterprise architecture has become a term that has fallen out of fashion. The enterprise architect job title has become inherently linked with heavyweight analysis by a small number of individuals. What many people perceive as enterprise architecture has stayed relatively static over the last 25 years - the rest of the IT industry has been transformed in that time, not only in technology but in ways of working. To most, an enterprise architect is somebody who practices TOGAF.

A similar thing has happened in other professions too. What do you picture when thinking of a lumberjack? A search of the internet for pictures of a lumberjack comes back with romanticised images of bearded men in checkered flannel shirts, holding axes. Do modern lumberjacks use axes? Chainsaws, harvesters, and feller bunchers are used for most work now. Do modern lumberjacks wear flannel shirts? To be honest I don't know, I only see romantic illustrations of them. They probably wear hard hats and high vis jackets!

Organisation

Let's start with looking at how enterprise architecture fits into an organisation. The terminology has evolved since the early days of enterprise architecture - other terms have also become established. Solutions architect (or sometimes solution architect) is used for "solution-level" design; outlines of systems aligned with the interests of the business, regularly interacting with other systems. There are also technical architects, who focus on supporting implementation and delivery of those solutions. People still use the term software architect as well, but a software architect could be doing either technical or solutions architect roles - it could even be a mixture of both, depending on where projects are within their lifecycle. There are also plenty of software architects who practice enterprise architecture, some of whom even refer to themselves as developers.

In a simple world, the enterprise architect works with the above architects by identifying and prioritising the problems within the enterprise that require solutions. We aren't in a simple world though, and thus the scope of the role is much broader. We will come back to this later. The below diagram illustrates the model, but doesn't imply a team or organisational structure.

A recent trend for technical leadership titles is the staff+ path, which are technical leadership roles from staff engineer and above. These are influential roles, but not associated with traditional enterprise architecture outputs. Despite it being described by many as "the technical leadership track", architecture too is a technical leadership track. Will Larson suggests that architect is a specialised staff+ role, but this doesn't acknowledge that the experience of an individual supports the different roles. You can't parachute a Right Hand into an Architect role. You can assume those behaviours in the way that teams assume interaction modes in Team Topologies, but the experiences needed to support the roles are different.

Getting enterprise architecture out of its rut requires architects to work closely with other staff+ roles. It doesn't replace the need for enterprise architecture but makes the architects more accountable and broadens the opportunities for collaboration. staff+ engineers will carry different perspectives, and this is the power that needs to be harnessed.

An alternative to defining enterprise architecture

We have noticed that defining enterprise architecture is a controversial subject, causing heated disagreement or vehement denial of their role among practitioners. Forrester has seen this too and has identified an alternative approach, by analysing how enterprise architects deliver value. That analysis identified that enterprise architects can deliver value along a strategic and project value axis (X), and a business and technical value axis (Y). I'll refer to project as delivery, as that's the underly priority. Over time the work shifts around the different quadrants, depending upon the needs and priorities of the enterprise. Some examples have been included for illustration in the diagram below, but even the same activity is likely to differ in its position depending upon its context and lifecycle.

We have identified that different types of value address different needs and priorities. Delivering different types of value requires different capabilities and skills from the enterprise architects, interacting with different parts of the enterprise. So now is the right time to ask, in a sea of differences, should the enterprise architect follow the same approach across the board?

Of course not, and this is the problem. We need to loosen up, and stop treating enterprise architecture like it's a rigid discipline, with framework gatekeepers constraining how people work. There are too many solutions that promise to unify the approach, and they have claimed enterprise architecture as their own.

Using the correct tool for the job

Given the broad variety of problems that an enterprise architect could be called upon to solve, they need to use the most effective tool to get the job done. This is where we need to bring together a variety of perspectives and techniques.

Choose your tools based upon your context. A painter doesn't use a paintbrush for the whole of a wall - but they would for cutting in around edges. When they do use a brush they will chose a type and size to match their activity. They will even use two or more tools simultaneously - masking tape, brushes, benches, extensions, or even thinners. Good architects should be equally as flexible.
Would you expect to be able to keep all of your tools in a single toolbox? Enterprise architecture frameworks have a vested interested in providing you with certifications and new releases. They want you to use techniques that are within their ecosystem. You're better off if you don't constrain your toolbox. Not all methods that are useful in enterprise architecture are labelled as enterprise architecture techniques - you can use tools in lots of different contexts.
If you can't see where to hammer, what do you do? You will often benefit from using multiple perspectives. It is widely acknowledged that you cannot model a whole enterprise, they evolve too quickly and are very complex. It is more important that you look at aspects of your enterprise from different perspectives instead, as this will reveal information which would otherwise have been hidden.
If you look on your problem from a different perspective, you may find that actually you will find root cause solutions, rather than applying whack-a-mole patches.

There are plenty of techniques that are being used across IT landscapes which are not labelled as enterprise architecture activities. To illustrate, we have drawn up the below table that provides examples. The table is far from exhaustive, and the values are draft, but you can hopefully see that you get radically different perspectives, very quickly, from embracing modern patterns. Some of the approaches were originally identified by Mark Richards - see Lessons 50, 51 and 52 of Software Architecture Mondays. Model-driven and initiative-driven are also approaches recorded by Mark.

Each approach has been provided with quality attributes to demonstrate some of the ways that an an enterprise architect may choose between approaches. Definitions of those listed are as follows:

Velocity Speed at which an initiative can provide value.
Effectiveness Whether the initiative results in the desired business outcome.
Accuracy How accurately the approach represents its intended outputs.
Accessibility Size of the audience, and whether the audience would understand the artefacts.

It is worth keeping in mind that as in all parts of architecture across IT, "it depends". The approaches could be combined - Inventorying implemented as part of a platform engineering effort would be using a continuous approach, and so characteristics would generally align with the continuous approach, but would require the high level of systems maturity to enable it.

You may need to scroll horizontally to see all of the columns in the table.

Approach	Example	Perspective	Style	Velocity	Effectiveness	Accuracy	Accessibility
Model-driven	Zachman, SABSA	Top-down	Analytical	⭐	⭐	⭐	⭐
Initiative-driven	TOGAF	Mixed	Analytical	⭐⭐	⭐⭐	⭐⭐	⭐
Inventorying	Populating registers to improve landscape visibility	Bottom up	Analytical	⭐⭐⭐	⭐⭐⭐	⭐⭐⭐	⭐⭐⭐
Iterative	Small incremental transformations	Technology	Hypothetical	⭐⭐⭐	⭐⭐⭐⭐	⭐⭐	⭐⭐
Value-driven	Prioritised based on outcomes	Business	Analytical	⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐
Adaptive	Expects change. Typically event-driven	Mixed	Operational	⭐⭐⭐	⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐
Continuous	Automated, self-scaling, self-healing, dashboards	Platforms	Operational	⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐
Collaborative	Big Picture EventStorming, workshops	Bottom up	Interactive	⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐	⭐⭐⭐
Socio-Technical	Team Topologies, stream-aligned teams, Conway's law	Mixed	Interactive	⭐⭐	⭐⭐⭐⭐	⭐⭐⭐	⭐⭐⭐
Value chain	Wardley Mapping	Mixed	Interactive	⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐	⭐⭐⭐

To summarise, there are lots of excellent approaches available, which within their own narrow niches have very positive characteristics. Interactive and Operational approaches give better informed decisions, and change how enterprise architecture is defined. Analytical approaches are also still useful, but are typically applied in very focused contexts. All approaches have their trade-offs, and those trade-offs will vary by context.

What are your experiences of these techniques, and what other approaches would you consider? If you would like to discuss any of the concepts we have written about get in touch.