As a developer, one of the most fun and intriguing phenomena that I regularly come across is called “The Depth Issue”. Now, this is a term I coined myself; consequently, there might be better and more recognized names for it out there. It’s just that I am none the wiser.
So what is the depth issue? Contrary to how it sounds, it is not actually an issue; instead, it functions as a feature that aids in understanding the said issue. So here is how it goes.
The Mechanism
As the lead developer in my project, backend architecture design discussions are something I regularly get involved in. And as expected, the goal of such discussions are to figure out an architecture that caters the best to the requirements in hand, while also making sure that you build your system in a scalable, extensible and reusable manner. This is even more pronounced in my work because we build open-sourced digital public goods, which are made available to independent adopters. What we build has to be developed in such a manner that it can be easily adopted by anybody, used at any scale and even promote contributions back.
So naturally, our architecture design discussions can go on for hours, across several weeks, with regular reviews to make sure that we are accounting for the maximum number of scenarios as possible. At some point during the discussion, we might propose an idea which might help with one particular facet of our requirements. But the idea itself might have several levels of abstractions that it isn’t straightaway well-understood. There could be concerns like if it will integrate well with the existing system, or that it isn’t readily apparent that we are not running into some obscure anti-pattern.
Each node represents a decision point during the simulation. The goal is to reach an optimal solution i.e. the green node, as quickly as possible.
The obvious solution is to run the idea through a simulation to see how it gets applied, how it interfaces with the existing system, and what are the issues that come up with it. You start at the entry point of your current implementation, run the system mentally or on paper and see how the system behaves in response. And this is a time-consuming process, often involving a few moments of silence in our meetings where nobody speaks, and everybody is engaged in mental simulations.
An ideal simulation where a great deal of time was spend accounting for every permutations and combinations. Depending on the requirements & complexity, always isn’t a realistic option.
The Depth Issue
If everything is good, the idea or pattern is shortlisted and proposed for a prototypical proof of concept (POC) implementation. If it isn’t, it is promptly discarded. Now this is where the depth issues come in. It is the phenomenon of knowing right away that an idea or a pattern will not actually work, without running an explicit simulation. Or more importantly it is an early warning, that the decision being taken might lead to a suboptimal solution/state in the future. The distinguishing characteristic of this phenomenon is that you don’t know why it doesn’t work, only that it wouldn’t. The reason is buried at a depth which isn’t reachable without actually simulating it (as shown above). But you somehow already know something is awry at some depth. Hence the issue at depth, “depth-issue”.
The risk here is committing to a decision which will lead to a suboptimal solution (or state of the system) down the line. A state which isn’t readily identified as being suboptimal at that moment in time.
The way I imagine it, is like a black box in your head, which takes in the said ideas or patterns and spits out either a cautious ‘This seems to be okay’ or a definitive ‘Something is wrong’, without providing any more context to it. It is basically the intuition which you can’t explain at that point in time. The fun part comes in when you have the confidence of your team in trusting your depth-issue triggers and work together to figure it out. And that almost always results in that rewarding ‘Ah ha! That’s why’ moment where you have figured out what the issue is.
As you chance upon a path that leads to an obscure suboptimal solution/state, depth-issue warnings are triggered, calling for caution.
Now personally for me, over the years I have learnt to trust the depth issue triggers as a credible source of information to act upon. And the quality of the exercise of designing something greatly depends on how much bandwidth we can assign to figuring out why there was a trigger in the first place. Moving too fast without heeding the cues of the black box always feels like a recipe for trouble later on. And that is a pitfall we try our best to avoid and mostly always succeed in.
The Source
I have often wondered where this intuition comes from and why it isn’t context heavy. And my guess is that, this black box is supposed to be a highly efficient real-time cognitive tool, built on top of all the knowledge you have gained from various sources, including books, conferences, YouTube videos, discussions with your peers, and most importantly, everyday experience of doing the same set of things again and again for years, while getting slightly better at it each time. Any more context could potentially defeat the purpose of what this cognitive tool is supposed to be. A real-time warning system, which is built to save you valuable time in the long run. Going back to the old adage, ‘Prevention is better than the cure’, it makes a lot of sense.
In the Wild
So what could be a good example of the depth issue triggers? Consider that we are building a microservice that would enable a modified API-gateway pattern for a dynamically pluggable set of microservices. The goal is to build a system that can survive the replacement of one-or-more microservices with foreign ones, without needing explicit code changes at gateway service. Basically allow microservices which aren’t explicitly designed to be interfaced with each other, work together as if they were. This is a valuable feature if you are building open-source services which might need to work with the adopter’s existing services.
Extending further on the classical API Gateway pattern (where there is a single point of entry from client-side), here we are interfacing the backend services with each other through the gateway service. Here it is fair to call the gateway as the interface service as well.
The Centralized Pattern
Now as broad as the gateway pattern could be, we do have some clear cut requirements. Core gateway service shouldn’t have to be modified based on the microservices it caters to. Meaning, there has to be some sort of plug-in based implementation that would make the gateway recognize the downstream microservice correctly. Then the following questions may arise; how much of the feature-set or workflows should be centralized to the gateway rather than within individual plug-ins? Can gateway hold all the infra-dependencies like message-passing systems, interfaces, caching workflows and provide these to each plugin as required through dependency injection? This invariably means standardization of the structure and workflows within the plug-ins.
This pattern incorporates a thin plug-in layer where we have standardized plug-ins which act as the dynamic element where service-specific code is implemented. Gateway holds all the dependencies and provide it to plug-ins as necessary through dependency injection.
The Decentralized Pattern & The Triggers
The opposite of that would be for the plug-ins themselves to have implementations that tap into the infra-dependencies, along with workflows compatible with the downstream services. This means less standardization and more decentralization. Now functionally, lack of standardization might be a glaring concern that is readily understood right away. But this pattern can give rise to other depth-issue triggers, which might require time to figure out with simulations.
And at the end of that process, the issue might be that with decentralization, there is distributed complexity within the system, making it difficult to express the system in a simple manner. Plug-ins aren’t following a set of well-defined standards and that would make it difficult to convey the idea of the plug-ins themselves within a team. That’s the issue at depth, going beyond the mere functional sphere and delving deep into the aspect of communication within and outside of a team working on the system. Basically, How self-expressive the system is.
Here, the plug-in layer is quite wide, allowing for non-standardized plug-ins to exist and act as the translation blocks between gateway and services. Each plug-in holds their own infra-dependency instances and can greatly vary in size and complexity.
The Conclusion
Now of course, it is all a matter of finding the right balance and the answer is not always black and white in any complex system. But with the help of trustable depth issue triggers, it’s amazing how enjoyable the whole process of designing and breaking down the complexities can be. Once again, it is most effective when the entire team recognizes the validity in the concept and decides to act on such triggers.
In effect, depth-issue or depth-issue trigger is an intuitive cognitive tool which acts as an early warning system triggering the need for caution. And the quality of this tool is cumulative on various factors like knowledge from various sources and your own experiences. I subscribe to this idea so much that, even a tidbit of passing information is a valuable source of knowledge for me. And the quality of what we build relies partly and greatly on how seriously we can take the warnings from such triggers. From my experience, this requires something more than just the technical expertise. It delves into the synergy and the social integration within a team. How seriously do you consider your colleague's and your own intuitive concerns?
Top comments (0)