Power of Emergent Behaviour in a Complex System

This blog is about understanding how we can tap into the potential of emergent behaviour in a Complex System and what we (IT Org) can learn from Ant Colony

The ultimate purpose of enterprise architecture is not to design systems; it is to help an enterprise reach specific business goals and solve the enterprise’s core problems, and the designing system is a part of it.

Ok…If designing a system is part of it, what else the EA is as a whole?

Before I try to answer that, let me give a tour of complex systems and the emergent behaviour.

Complex System

The common assumption is that one person can fully understand the complex interactions among all the elements that make up a modern enterprise. Though we wanted that to be true, we all know the reality.

In general, the first step in approaching or solving a problem is acknowledging that there is a problem and understanding what the problem is. The problem: even a reasonably large collection of systems interacts in ways that no one person can fully understand all details or accurately predict — there is a name for this system: “Complex System”.

Enterprise is a complex system; There are many studies and researches to characterize complex systems. One of the influential studies is ‘emergent behaviour’ in a complex adaptive system. Let’s briefly see what CAS is.

Complex Adaptive System (CAS)

“The complex adaptive system is a collection of independent entities where the individual elements each behave according to their own rules and interests, but when viewed as a whole, this collection of elements functions as a collaborative group whose behaviours benefit the entire group and rapidly adapt to changing environmental conditions as if they were centrally coordinated.”

Ok…I got it, I think…But, what are we saying again?

A simple example of a complex adaptive system is a flock of birds. Moving from one place to another, turning in midflight, climbing, and diving all appear to be under the control of one — No single flock leader is instructing every step here.

Are we saying, ‘we don’t need a leader ?’

No, that is not what it means; it means — Let me give another example to make the notion apparent in the context of emergent behaviour.

Emergent Behaviour

“Usually, the individual elements of a complex adaptive system are relatively simple and operate according to a set of similarly simple rules. But when these simple elements are combined into a system, the system exhibits complex behaviours that are not readily predictable by examining the behaviours of the individual pieces. The system as a whole is greater than the sum of its parts and behaves differently.”

This phenomenon is known as emergent behaviour and is one of the critical characteristics of a CAS.

Put more simply, emergent behaviours result from each element of a system pursuing its own interest. Tapping into emergent behaviour is a compelling way to strengthening an organization. We can use the insight developed from CAS system research to improve enterprise architecture practice.

Ant Colony & IT Org

An ant colony is an example of something scientists call a superorganism.

“A superorganism emerges when individuals of the same species become so highly specialized that they cannot survive without being part of a larger group composed of many individuals with complementary specializations.”

I know what you are thinking; Yes, the powerful IT organization sounds a lot like a superorganism.

The key attributes of the superorganism, “complementary specializations” and “moving towards a common goal”, are what makes an Organization powerful. What else can we learn from superorganisms? Let’s walk back to the ant colony.

The ant colony does many complex tasks, including forage for food, maintaining the nest, caring for eggs and young ants, fighting off predators, and many others tasks. For example, when one ant finds a food source, the ants will collectively find the shortest path to the source.

Here is how it happens; worker ants leave the colony each day to forage for food. They initially set out without a specific destination, and each wanders the area until it finds a food source. Several ants may find the same food source simultaneously from different routes, and each of the ants will return to the nest independently. On this return journey, the ant releases pheromones, a chemical trail that helps other ants find the source. Those ants that return most quickly will inspire more ants to follow their trail. Over time, the worker ants converge on the shortest routes, and the longer tracks are abandoned. Voilà!

Let’s dissect and understand what just happened here.

The goal of the ant colony is not just to find food. Instead, the colony’s goal is to maintain its health and propagate the species; finding food is just one of the tasks necessary to reach that goal. They have to be back in the nest sooner to protect the colony. This is one of the constraints that create the instinct to bring food back more quickly. It makes finding the shortest path as emergent behaviour.

Let’s Conclude (for now)…

The leaders/enterprise architects can focus on the following aspects to yield results through emergent behaviours; making the common goals apparent and help make the complex system move towards that through defining a development framework, process, guidelines, and constraints. Instead of focusing more on laying out very detailed instructions for doing everything, take advantage of the CAS system by recognizing the importance of the emergent behaviours that come from these systems. Everything about an enterprise architecture effort should contribute to reaching the enterprise’s goals; tasks that do not support the enterprise’s goals are a waste of resources.

In short, Building an efficient system requires both top-down (clear vision, goals, guidelines & constraints ) and bottom-up approaches (room for exploration, adaptiveness, autonomy on implementing low-level details).

Foot Note: Characteristics of CAS systems and their relevance to system design have been recognized for many years. This blog is a summary of my learning in this field of study. In part two of this article, I will expand more on what we concluded in this post. Stay tuned.