Stop trusting AI coding agents.
Last week, I caught myself doing something I never thought I'd do as an engineer:
I accepted an AI-generated summary as proof.
The agent had a clean explanation. It sounded confident. It claimed tests passed. The reasoning looked solid.
And for a moment, that felt like verification.
It wasn't.
As AI coding becomes normal, I think we're developing a new blind spot: We're starting to trust claims instead of evidence.
The New Software Delivery Problem
For years, software delivery looked like this:
Developer → Code → CI → Deploy
Today, it's increasingly becoming:
Agent → Code → Claim → Verification → Deploy
That extra step matters.
An AI agent can tell you:
- ✅ Tests passed
- ✅ Linting passed
- ✅ Ready for merge
But what certifies those statements?
- Not what the model intended.
- Not what the prompt asked.
- Not what sounds convincing.
- What actually happened.
Confidence Isn't Evidence
One realization keeps bothering me: A skipped check and a successful check can look surprisingly similar when you're only reading an agent's summary.
Humans are wired to trust polished explanations. Large language models are exceptionally good at producing polished explanations.
That's a dangerous combination. The more capable our agents become, the more important independent verification becomes.
A Better Question
Most discussions around AI coding focus on:
- Which model is best?
- Which agent is fastest?
- Which IDE has the best experience?
Those are useful questions. But I think the more important question is:
How do we prove an AI-generated change is safe to ship?
Not trust. Not confidence. Proof.
Building for Proof Instead of Trust
That's the idea behind Thea Software, an open-source project I'm building.
HLIntel
/
TheaSoftware
The verdict layer for AI-written software. Deterministic checks, the right context and governed tools on every change, ending in one verdict and a receipt anyone can audit. CLI, MCP, CI, desktop.
Thea Software
Protocol: a change maps to proof; Thea selects evidence and the build enforces the verdict (evidence). Agents: llms.txt; chats: CHAT.md.
Start: install/use · architecture · enterprise · agents · chat · adapters
Why this exists
An agent can report a change as done without running a single check. A check that never ran looks the same as a check that passed, and the change ships. Thea closes that gap:
-
Every change maps to proof.
atlas.yamllists the checks each kind of file owes and the gate runs them before the change is called finished. - Refusal happens at commit, for any agent. A git hook runs the same check-only command whether Claude, Codex, Cursor or a person made the commit.
-
A skipped check is never a pass. A missing tool is reported as
SKIP, counted, and kept out of the pass count. Gates report…
The premise is simple:
- Every change owes proof.
- Every proof produces evidence.
- Every evidence chain ends in a verdict.
Instead of treating an agent's summary as truth, verification should come from deterministic checks, recorded evidence, and an auditable outcome. The project positions itself as a "verdict layer for AI-written software" built around evidence-backed verification and enforceable checks.
What Changed My Thinking
The biggest lesson wasn't technical. It was psychological.
For decades, engineers learned: Don't trust user input.
I think AI is teaching us a new version of the same lesson: Don't trust agent output. Verify it. Every time.
Because eventually most software will involve AI-generated changes. When that happens, the question won't be "Which model wrote this?"
The question will be: What evidence exists that it was verified?
The organizations that answer that question well will move faster and safer than everyone else. And I suspect that's where the next generation of software tooling is headed.
What edge cases did I miss here? If an AI coding assistant tells you, "Tests passed," what proof do you require before merging?
Top comments (0)