Right now, somewhere in the world, an engineer is tweaking a prompt, adjusting an LLM's temperature, or writing a system command that says: "Please format your response strictly as valid JSON and do not hallucinate."
And the most terrifying part of this reality? As an industry, we genuinely believe this is software engineering.
Let’s take a step back and look at our modern infrastructure with brutal honesty: We are no longer building systems. We are operating a massive statistical casino where we roll dice between code blocks. And the worst part? We don't even know what the house is going to do next.
The Ontological Error: Words Are Not Inputs
The fundamental trap the industry has fallen into is trying to solve AI failure modes at the wrong layer. When an agent chain breaks or spirals into an infinite loop, developers immediately look for a better sequence of words to fix it.
But there is a glaring, structural wall here that nobody is talking about: Words are not inputs; they are outputs.
Human language is chaotic, fluid, probabilistic, and entirely open to interpretation. It is the final, lossy expression of a structured thought process. When you build an entire system architecture on top of language, you are trying to anchor your logic into liquid. You cannot force a cloud of probabilities (an LLM) to act like a rigid silicon processor. Building a skyscraper on a swamp and wondering why the concrete is cracking is not engineering.
The Death of Determinism
In traditional software architecture, there is an unshakeable, foundational rule: If you cannot trust a system to audit its own output, the system was poorly designed from the start. A logic gate is either open or closed. A validation layer either passes or fails. This process takes less than a millisecond, and the computational cost is practically zero.
Yet, in modern "agentic" architectures, what are we doing instead?
To arrive at a simple, binary decision—like whether a workflow should proceed—we spin up massive GPU clusters, calculate probabilities inside a neural network with hundreds of billions of parameters, wait seconds for the response, and hope the generated text is correct. If we are afraid it might fail, we throw another prompt at a different LLM just to audit the first one.
In any other branch of engineering, unpredictability is explicitly defined as a bug. Yes, distributed networks and financial markets deal with probabilistic environments—but the systems we build to govern them are designed to enforce deterministic constraints. In modern AI development, however, we have packaged the unpredictability itself and marketed it as "autonomy."
We have traded microsecond mathematical certainty for an incredibly expensive, multi-second game of "What if?".
The Solution Isn’t "Smarter" Models
The fix for this reliability crisis is not bigger context windows, nor is it training models on more data. The solution is deterministic governance enforced at the architectural level.
An LLM should not be the sole source of control in your system's logic. It should merely act as a translation layer between human intent and machine execution. We need to stop obsessing over what the AI generates, and start focusing on the logical, non-semantic cage we trap that generation inside of.
If your production pipeline allows a probabilistic model to spend 5 seconds guessing tokens just to authorize a critical workflow or deploy code, you are no longer the architect of that system. The model is.
The future of software engineering does not belong to those who increase the noise generated by AI. It belongs to the architects who build the deterministic filters underneath it.
So ask yourself: Are you actually engineering resilient systems, or are you just sitting at a statistical casino, waiting for the dice to roll in your favor? Who is casting the dice in your architecture?
Top comments (1)
If your AI agent executes 10 million actions per day:
Who should have the final authority?
The model?
A rule engine?
A governance layer?
A human?
Curious how others think about this.