More than a year ago, which is practically ancient history in the AI years, I wrote a blog about using AI to build new self-service capabilities.
It felt like the future.
We built a self-service action that could create new self-service actions, helping us move faster, reduce bottlenecks, and scale a small Platform Engineering team supporting hundreds of developers.
One of the most interesting parts was using Amazon Bedrock to generate Terraform code dynamically at runtime, allowing the system to determine how a new cloud resource should be provisioned using our existing Terraform modules.
It worked.
It was impressive.
And… we removed it.
Looking back, abandoning that approach turned out to be one of the best engineering decisions we made.
At the time, it felt like an isolated technical decision.
It wasn’t.
Recently, we faced a much smaller problem. We wanted to automate the creation of DNS records in Cloudflare through our self-service platform.
The first proposal was exactly what you’d expect today: “Let’s build a Claude Skill.”
Immediately, I had a strong sense of deja vu.
But my hesitation wasn’t about whether AI could do it — it was about whether it should.
We were simply asking the wrong question.
The Industry Shift
A lot of engineers today feel like everything they learned over the last decade suddenly became less relevant.
We are DevOps engineers.
We are Platform Engineers.
We used to spend time designing systems, defining standards, reviewing architectures, and planning before writing a single line of code.
Every automation started with the same question:
“How should we automate this?”
Today, that question has quietly changed. Now we ask:
“How can AI do this?”
At first glance, that sounds like progress. And sometimes it is.
Large Language Models have fundamentally changed the way we build software. Tasks that used to take hours now take minutes, and entire prototypes appear from a single prompt.
The temptation is obvious. If AI can do it… why not let AI do it?
Somewhere along the way, though, many of us unconsciously changed more than just our tools. We changed our engineering process.
Instead of designing the architecture first, we choose the technology first.
The discussion becomes:
- “Should we use Claude or GPT?”
- “Should this be an Agent?”
- “Can Hermes do it?” Before answering a much simpler question:
Does this problem actually need “reasoning”?
Runtime Reasoning vs. Deterministic Execution
People often talk about AI Agents, Skills, MCP servers, and LLM workflows as if they’re simply the next generation of automation.
They’re not.
They solve a different type of problem.
LLMs are great at reasoning through messy, ambiguous problems. They can adapt and change plans. But reasoning adds probability — it’s no longer 100% certain.
That capability is incredibly powerful, but it comes with fundamental tradeoffs. Reasoning, by definition, introduces probability into a process.
Traditional automation is deterministic. Every step is predictable, testable, and produces the same result every single time.
Now let me go back to our DNS example:
Slack Form ──► Lambda ──► DynamoDB ──► Terraform ──► Cloudflare
Nothing in that pipeline required interpretation.
We knew the state we wanted, the code already existed, and the steps were clear. So why introduce a “maybe” into a “definitely” process?
Reason at the Edge. Execute in the Core.
An LLM can be incredibly valuable at the system boundaries. Imagine a developer writing in Slack: “Create a CNAME for api.staging pointing to our new load balancer.”
Turning that natural language sentence into a structured JSON payload is exactly the kind of ambiguity LLMs excel at. Natural language is messy, humans omit context, and intent needs interpretation. That is a genuine reasoning problem.
Structured Outputs solve formatting. They don’t solve decision-making.
The risk is letting a probabilistic model decide which tools to run in production.
Once user intent becomes structured data (or if you collected it via a structured Slack form in the first place), the remaining pipeline no longer benefits from reasoning. Every step after that should behave like standard production software: predictable, testable, and deterministic.
Crucially, the structured data generated by the LLM passes through strict schema validation and deterministic authorization policies (RBAC) in the core. Even if an LLM is manipulated at the edge, it can only request actions — the core independently validates whether the user is authorized to perform them before execution ever touches infrastructure.
The LLM proposes actions.
The platform decides whether they’re allowed.
That realization led us to a clear architectural principle:
Reason at the edge. Execute deterministically in the core.
The Hidden Cost of Runtime Reasoning
Using an AI Skill instead of a workflow seems fast at first, but it creates hidden costs for stability and security.
However, when you introduce runtime reasoning into an execution path, you accept fundamental tradeoffs that directly impact production stability, security, and cost.
Here is how those tradeoffs manifest in production:
Human in the Loop Bottlenecks
If an LLM cannot be trusted to execute production changes unattended, someone must manually review every single run (When it’s open a PR for example).
This brings back the very bottleneck you were trying to fix.
Probabilistic Infrastructure
A script follows orders; an LLM makes choices.
Production isn’t the place to find out the AI made a mistake.
Paying for Unused Intelligence
Scripts cost almost nothing. LLMs cost tokens.
If the workflow was already predictable, you’re paying for “intelligence” you don’t actually need.
Expanding the Attack Surface
A deterministic workflow only executes code you explicitly wrote.
An LLM executes actions based on how it interprets a prompt.
The moment an LLM sits directly in the execution path, prompt injection, context poisoning, and unexpected tool execution become part of your threat model.
For infrastructure, we need predictability, not improvisation.
A script that does the same thing a million times is better than a system that “guesses” correctly most of the time.
Use AI Where It Actually Adds Value
We use AI more than ever — just not to run our production systems.
We use AI to design Terraform modules, generate Lambda functions, build CI/CD pipelines, review Infrastructure as Code, write tests, and understand unfamiliar systems.
Let AI build automation. Don’t let AI be the automation.
Once we’ve reviewed the generated code, the reasoning phase is over.
From that point onward, production executes deterministic software — not prompts. The model participates during development, not in the production control plane.
The AI reasons once. The software runs forever.
The 30-Second Architecture Review
Before introducing an AI Agent, Skill, or runtime LLM into your next system, ask yourself:
[ ] Does this problem actually require runtime reasoning?
[ ] Can every execution step already be described in advance?
[ ] Will the same input always produce the same desired output?
[ ] Could AI generate the workflow code instead of executing it?
If the steps are predictable, you don’t need an LLM. You need reliable software.
Final Thoughts
LLMs are amazing at reasoning, but not every problem needs it.
Use AI to understand what users want and to help you write code. But once the plan is set, let the software take over.
But once your production system knows exactly what needs to happen, stop reasoning and start executing.
User
│
▼
Natural Language
│
▼
LLM (Intent Extraction)
│
▼
Validated JSON
│
▼
Authorization (RBAC / Policies)
│
▼
Deterministic Execution
Lambda ──► DynamoDB ──► Terraform ──► Cloudflare
Reason where things are uncertain. Execute where things are clear.
Because great architecture isn’t about eliminating AI.
It’s about confining uncertainty to the parts of the system that actually benefit from it.
Top comments (4)
This is a sensible distinction, and one that is easy to miss when AI becomes the default answer to every technical problem. Using an LLM to interpret messy human intent makes perfect sense, but once that intent has been converted into validated, authorised instructions, the rest should be handled by predictable software. “Reason at the edge, execute in the core” is a strong principle because it keeps the flexibility where it adds value without introducing unnecessary cost, risk and uncertainty into production systems. AI is often most useful for helping us design better workflows, not for replacing reliable workflows that already know exactly what to do.
I think the key distinction here is between decision-making and execution. I'd take it one step further: deterministic workflows and AI agents don't have to compete they should complement each other.
We've had good results at IT Path Solutions treating the LLM as a planner that proposes what should happen, while the workflow engine owns how it happens. Once an action crosses into infrastructure creating DNS records, deploying services, rotating secrets, or changing cloud resources it goes through deterministic validation, policy checks, and idempotent execution.
That separation keeps the flexibility where it's valuable (understanding messy human intent) without making production behavior probabilistic. The orchestration layer becomes the contract, and the LLM becomes an advisor rather than the control plane.
absolutely agree
I like the edge/core split here. One extra check I’d add is a boring replay test. If the structured request can’t pass through auth, schema validation, and a dry-run plan without fresh reasoning, it probably still belongs at the edge. Keeps the LLM useful without making it part of the control plane.