Over the last few years, the conversation around AI infrastructure has been dominated by a relatively simple question: who has the most compute capacity? The race for GPUs, data centers, high-speed networking, and more efficient inference turned physical infrastructure into one of the industry's most obvious battlegrounds. But while that race continues, another, much less visible transformation is happening directly above it. We are building a new software layer around models, and that layer is growing at a pace that is becoming increasingly difficult to ignore.
The model that, only a few years ago, could be understood as the center of the system is progressively becoming a component inside a much larger architecture. The moment we want it to do more than answer a question, new requirements appear. If it needs to work with information that was not present in its original context, we need retrieval mechanisms. If it needs to retain information across interactions, we need memory. If it needs to act on external systems, we need tools. If those tools need to be discovered and used in an interoperable way, we need protocols. If the work requires multiple steps, we need some form of orchestration. Once those steps begin running for extended periods, we need state, failure recovery, and observability. And once an agent gains real ability to act, identity, authorization, policy, and evaluation become unavoidable problems.
None of these layers is unreasonable. In fact, most of them solve real problems. What is interesting is the pattern that emerges when we put them together: every new capability we try to obtain from the model seems to produce a new architectural surface around the model.
That leads to a much more uncomfortable question than the usual discussion about which model is better.
Are we actually building intelligent systems, or are we building an ever-growing amount of software around an intelligence we still do not know how to make fully reliable?
The question matters because there is a fundamental difference between model capability and system reliability. A model can be extraordinarily good at reasoning about a problem while still being a difficult component to operate inside a system that needs to maintain state, use tools, respect permissions, handle failures, and produce reproducible results. The intelligence we observe in a response does not automatically tell us anything about the stability of the system required to execute that response in the real world.
This distinction is relatively easy to miss when looking at a demo. A demo can show an agent reading documents, using a tool, and producing an impressive answer. The real system, however, has to answer much less spectacular questions. What happens if the tool returns incomplete information? What happens if the agent misinterprets the result? How do we know what context it received when it made a particular decision? How do we recover state after a failure? What permissions did it have at that moment? Can we reconstruct the complete sequence of events?
In other words, the problem is no longer simply making the model intelligent. The problem becomes making that intelligence operate inside a system.
And this is where one of the most unusual characteristics of this new architecture appears: we are using deterministic software to try to control a fundamentally probabilistic component. Traditional software can expose relatively precise contracts. A function receives certain parameters and returns a result. A database maintains state that we can query. A protocol defines how two systems exchange information. With a generative model, however, we can impose instructions, schemas, and validations, but we cannot completely describe its behavior through a traditional contract.
Our response has been to build layers.
If the model needs external information, we add retrieval. If it needs memory, we add memory. If it needs to act, we add tools. If we need interoperability between those tools, we add protocols. If we need to control execution, we add orchestration. If we need to know what happened, we add observability. If we need to verify that the result was correct, we add evaluation. If we need to restrict possible actions, we add identity and authorization.
Each layer makes sense on its own. The problem appears when we stop looking at the layers individually and start looking at the architecture as a whole.
We are building a system whose central component is probabilistic and progressively surrounding it with mechanisms intended to introduce structure, memory, boundaries, traceability, and control.
That may be exactly what we need. But it may also be a sign that we have not yet found the right abstraction for building software with models.
Abstraction Can Become Infrastructure
The recent evolution of MCP is particularly interesting for precisely this reason. It is easy to describe the Model Context Protocol as a standardized mechanism for connecting models with external tools and resources. That description is still valid, but it is becoming insufficient as we look at where the protocol is heading.
The specification published in July 2026 introduced changes around statelessness, routing, caching, long-running tasks, and authorization, while the subsequent roadmap includes agentic messaging, HTTP transport, agent identity, and enterprise security as explicit areas of focus. The protocol is beginning to deal with problems traditionally associated with infrastructure and distributed systems, rather than simply application integration.
That does not mean MCP is necessarily the definitive architecture for agents. It means something more important: an abstraction that could initially have looked like an integration interface is beginning to develop its own infrastructure problems.
The same pattern is likely to appear elsewhere.
Once tools stop being simple functions and become capabilities that agents can discover and execute, we need to manage their identity, permissions, availability, and state. Once memory stops being merely additional context and becomes part of a persistent architecture, we need to solve problems of consistency, recovery, and relevance. Once agents execute tasks over extended periods, we need mechanisms to observe and reconstruct their decisions.
Abstraction simplifies a problem, but it does not necessarily eliminate its complexity. Very often, it simply moves that complexity to another layer.
That movement is normal in software engineering. What is new is the speed at which it is happening around AI systems.
The Model No Longer Defines the Architecture
This also changes how we should think about competition between model providers.
If an application uses a model as an interchangeable component, the model is no longer necessarily the center of gravity of the entire architecture. The inference provider can change while memory, tools, protocols, authorization mechanisms, observability, and much of the execution logic remain intact.
That creates a different dynamic.
The model can be extremely important to the quality of a decision, but the architecture that determines what information it receives, which tools it can use, what actions it can execute, and how its results are evaluated can become equally important to the final behavior of the system.
This is where infrastructure begins to become architectural power.
Not because the model stops mattering, but because the complete system increasingly depends on the layers surrounding it.
That is also why interoperability matters so much. If every provider builds its own way of exposing tools, memory, context, and execution, the ecosystem becomes fragmented around the models. Protocols that manage to become common boundaries can acquire an importance far beyond their original technical purpose.
The battle, therefore, is not happening only inside the model.
It is happening at the interfaces between the model and everything it can do.
The Risk of Building Too Much Architecture Too Early
This is where the story becomes less comfortable.
There is an obvious temptation in any emerging technology: when a capability is still unstable, building infrastructure around it seems like a reasonable way to make it usable. The problem is that this infrastructure can eventually crystallize decisions we do not yet know are correct.
Agentic systems are still searching for their boundaries. It is not yet completely clear how much state an agent should retain, what should be handled by the runtime and what should remain inside the application, what portion of memory should be persistent, how complex executions should be evaluated, or what level of autonomy is actually useful.
Yet we are already building products and platforms around those decisions.
That creates a familiar risk in software architecture: turning a provisional solution into a structural dependency.
It would not be the first time. The history of computing is full of abstractions that were created to solve an immediate problem and eventually ended up shaping architecture for decades. The difference is that we are now making those decisions around systems whose fundamental behavior is still evolving.
That is why the important question should not simply be how many layers the AI stack has.
The question should be what fundamental problem each layer is solving and what new dependency it introduces.
If a layer removes one limitation of the model but forces us to introduce three more layers to keep it operational, we may not be reducing complexity at all.
We may simply be redistributing it.
Maybe the Real Product Is Not the Model
This is the part I find most interesting about the whole process.
For years, we have talked about models as if they were complete products. But a model by itself is not an operational system.
It is a capability.
The product appears when that capability can reliably interact with data, tools, users, and other systems.
That is why the infrastructure we are building around models may ultimately have a longer lifespan than some of the models themselves.
Models will change. Providers will change. Benchmarks will change. Context windows will change. Even the internal architectures of models will probably change radically.
But the architectural questions will remain.
Who controls the context? Who controls the tools? Where does state live? How is an action authorized? How is an execution observed? How is a decision evaluated? How does a system recover when the probabilistic component at its center does something unexpected?
Those questions do not disappear because a better model arrives.
If anything, they become more important.
Because the more capable models become, the greater the operational radius we will be willing to give them. And the greater that radius becomes, the more important the architecture responsible for controlling what happens around them will be.
That is why I believe the next stage of AI infrastructure will not be decided solely by who builds the most capable model. It will also be decided by who can build the operational layer that turns that capability into reliable behavior without creating an architecture that becomes impossible to maintain.
That is the real challenge.
We are not simply adding tools around a model. We are trying to build a new form of software around a component that does not behave like traditional software.
And we still do not know what the right abstraction is.
Maybe the AI stack has to become thicker before it can become simpler. Maybe we need all these layers to discover which ones are unnecessary. Or maybe we will eventually discover that some of the layers we are building today are not permanent solutions at all, but scaffolding around an architecture that has not fully emerged yet.
What does seem clear is that the model is no longer the system.
It is one component of the system.
And the more capable those models become, the more important everything we build around them becomes.
The question that remains is uncomfortable, but I think it is worth asking before adding another layer:
Are we building the architecture of the next generation of software, or are we simply building infrastructure to compensate for an abstraction we still do not understand?
T474::AUTH
AI::ASSISTED
HUMAN::DIRECTED
ANTI_HYPE::013
Top comments (0)