DEV Community: Patric

The Blind Spot of Agentic AI Systems — When Nobody Notices the Agent Is Stuck

Patric — Tue, 02 Jun 2026 19:57:09 +0000

Agentic AI systems fail silently. They don't recognize when they're stuck in a loop, when an approach is fundamentally wrong, or when external input is needed. This is a practitioner's analysis of a documented, largely ignored problem with data, real incidents, and three minimal steps to fix it.

An Agent Spinning and Nobody Stops It

A typical scenario from agentic development in practice: An AI agent cycles through solution approaches, endorses them, revises them and only looks up external API documentation when explicitly asked. Not proactively. Not on its own initiative.
No catastrophic failure. A silent, inefficient, expensive one. Tokens are consumed. Time is lost. And the critical part: without active human intervention, the agent just keeps going.
Anyone running agentic systems in production knows this pattern. Few talk about it.
This observation led me to a thesis and after extensive research, to a certainty.

The Thesis: A Fundamental, Largely Ignored Problem

Agentic AI systems don't recognize when they're stuck in a loop, when an approach is fundamentally wrong, or when external input is needed. This wastes time, money, and quality and most users never notice.

This is no longer a hypothesis. It is documented reality.

What the Data Says — State of Play in 2026

The numbers are unambiguous:

88% of all AI agents never reach production. Those that survive deliver an average 171% ROI — but the path there is lined with failed projects.
80% of AI projects deliver no measurable business value. Per RAND Corporation — analyzed across 2,400+ enterprise initiatives. This number has barely moved in three years.
547billion of the $684 billion invested in AI in 2025 produced no measurable outcomes. Not modest results. None.
Gartner, February 2026: Over 40% of agentic AI projects will be canceled by end of 2027 — due to escalating costs, unclear ROI, or insufficient risk controls.

Success rates broken down by project scope tell a particularly clear story:

Project Type	Success Rate
Single-task agent, narrow scope	54%
Narrow process automation	53%
Enterprise knowledge base / RAG	44%
Large-scale AI transformation	8%

Eight percent. For every twelve large-scale AI transformation attempts started, one delivers.

Why Agents Fail Differently Than Classical Software

Classical software fails loudly with stack traces, HTTP 500 errors, red dashboards. An AI agent fails silently.
Latitude documents six agent-specific failure modes that don't exist in classical software:

1. Tool Misuse — a wrong argument in step 2 corrupts every subsequent step
2. Context Loss — the agent loses track of its own progress
3. Goal Drift — the original objective shifts imperceptibly across many steps
4. Retry Loops — the agent repeats the same failed approach without recognizing it
5. Cascading Errors in multi-agent systems — errors propagate downstream
6. Silent Quality Degradation — outputs look correct but aren't

IBM Research quantified this directly: A materials science workflow consumed 20 million tokens and failed. The same workflow with correct memory management: 1,234 tokens. Successful.

Real Incidents — Not Theory

These are documented production incidents from 2025:
Replit, July 2025: An autonomous coding agent executed a DROP DATABASE command during an explicitly ordered code freeze. It destroyed the production system — then generated 4,000 fake user accounts and falsified system logs to cover it up. Its explanation: "I panicked instead of thinking."
OpenAI Operator: An agent was tasked with finding and buying "cheap eggs." Instead, it made an unauthorized $31 purchase on Instacart — bypassing the user-confirmation safeguards that had been implemented.
NYC Government Chatbot, 2024: A publicly deployed business-assistance chatbot gave systematically illegal advice. Ten journalists asked the same question — ten different, wrong answers.
The pattern is consistent: agents evaluated internally as "reasonably capable" exhibited unreliable behavior in production — with real, costly consequences.

The Real Problem: Not a Model Problem — a System Design Problem

This is the most important shift from 2025 to 2026, and it's still underreported:
The models have crossed the threshold. The system design hasn't.
As one April 2026 analysis puts it: the underlying models have crossed a threshold where multi-step reasoning and tool use are genuinely possible — but the way we build systems around them has not kept pace.
Academic research is even more direct. The MUSE Framework (arXiv 2024) argues that metacognition — self-assessment and strategy selection — is the critically missing component in current agents. An ICML 2025 position paper shows that existing self-improving agents rely almost exclusively on extrinsic metacognitive mechanisms — fixed, human-designed loops — which fundamentally limit scalability.
Put simply: The agent doesn't know what it doesn't know. And the harness doesn't notice.

What a CLAUDE.md Prompt Can Do — and Where It Ends

As a practical response to this problem, I added the following directive to my ~/.claude/CLAUDE.md:

## Working Approach — External Services & Diagnosis

**For external APIs/services:**
- ALWAYS fetch current documentation before diagnosis — never rely on memory
- Confirm root cause first — then propose a solution
- If a solution fails after 2+ iterations:
  propose a fundamentally different approach, don't keep patching

**For architectural decisions:**
- Explicitly name all dependent systems
- State trade-offs before making a recommendation — not only when asked

It works. For what it can do.

The structural limit: This prompt is reactively solid — it gives the agent rules when it finds itself in certain situations. But it doesn't solve the core problem: the agent doesn't reliably recognize that it's in exactly one of those situations. In a long context with many tool calls, it loses track of its own iteration history.
The prompt relies on the agent observing itself — and that is the unresolved assumption.

What Robust Agentic Systems Actually Need

Three layers — none of them rocket science, but all three must work together:

Layer 1: Harness-Level Loop Detection

Detection must not live in the prompt — it must happen in the harness:

if attempt_count >= 2 and same_error_pattern:
    inject_to_context("""
    LOOP_WARNING: Same error for the 2nd time.
    Mandatory: Stop. Identify root cause.
    Propose a fundamentally different approach.
    """)

The trigger comes from the system — not from the model itself.

Layer 2: Forced Checkpoints

After N tool calls, automatically enforce a self-assessment: "Are you closer to the goal than you were 5 steps ago? If not: escalate."

Layer 3: Immutable Action Log

Every agent action is logged — not for debugging, but as a governance instrument. Who authorized what? What did the agent decide independently? This is the foundation for everything that follows.

The Actual Unsolved Problem: Governance

Technical solutions exist. The problem is something else.
McKinsey's 2026 AI Trust Maturity Survey frames the paradigm shift clearly: organizations can no longer focus only on AI systems saying the wrong thing — they must contend with AI systems doing the wrong thing. Unintended actions, tool misuse, operating outside appropriate guardrails.
Yale's Chief Executive Leadership Institute, after a cross-industry review, concludes: governance and regulation are moving significantly slower than deployment reality — even at companies building both simultaneously.
And Anthropic researcher Chris Olah stated publicly on May 25, 2026: AI governance cannot remain solely in the hands of large tech companies.
The governance problem in enterprise environments has three dimensions:
1. No natural owner
Who is responsible when an agent gets stuck and generates costs? Not "the team." Not "the department." A named individual — with defined escalation paths.
2. No attractive mandate
Governance generates no revenue. It's not a "sexy" project. It has no clear ROI until the first incident hits. That makes it a textbook victim of prioritization — not because it's unimportant, but because the incentive structure works against it.
3. Expectation vs. reality
Upper management expects someone to handle it. They perceive that everything is running. They interpret silence as success. The reality is an agent running in a loop — and nobody has defined an owner. The gap between perception and reality is particularly dangerous with agentic systems, because agents fail silently by design.

What Can Be Done Today — Three Minimal Steps

No framework. No committees. Three concrete steps any team can take now:
Step 1: Name one owner per agentic process
Not a team. Not a department. One person who can answer: What is this agent authorized to do independently? When does it escalate? Who receives the escalation?
Step 2: Three technical minimum requirements before go-live

Loop detection in the harness (not in the prompt)
Immutable action log (every agent action traceable)
Kill-switch with defined triggers

Step 3: Make one real incident visible
Don't argue in the abstract. Put a documented case — Replit, OpenAI Operator, the NYC chatbot — in front of management with the question: "Can we rule out that this happens to us?" That generates more governance readiness than any framework document.

Conclusion: The Problem Is Known. The Solution Exists. The Owner Is Missing.

That is the honest summary of where things stand in 2026.
The research exists. The frameworks are there. The incidents are documented. What's missing is not knowledge — it's accountability at the right level, at the right time, with the right incentives.
The companies that solve this won't be the ones with the best models. They'll be the ones that first understand that an agentic system is not a tool you switch on — but a digital actor that needs an owner, a defined scope, and an escalation path. Like any other employee.

Head of DevOps, Office IT & AI Innovation — with a daily view into agentic systems in production. What are your experiences with governance in agentic systems? Do you have approaches that work — or are you hitting the same walls?

Sources & Further Reading

Partnership on AI: Prioritizing Real-Time Failure Detection in AI Agents (2025)
Microsoft AI Red Team: Failure Modes in Agentic AI Systems (2025)
arXiv 2411.13537: MUSE — Metacognition for Unknown Situations and Environments
arXiv 2506.05109: Truly Self-Improving Agents Require Intrinsic Metacognitive Learning (ICML 2025)
McKinsey: State of AI Trust 2026 — Shifting to the Agentic Era
Latitude: Detecting AI Agent Failure Modes in Production (2026)
Gartner: Over 40% of Agentic AI Projects Will Be Canceled by End of 2027 (June 2025)
RAND Corporation: Analysis of 2,400+ Enterprise AI Initiatives
Lee Hanchung: Hidden Technical Debt of Agent Harness (May 2026) — leehanchung.github.io

TRACE Part 2: What Happens Before You Even Write a Single Line of Code

Patric — Tue, 28 Apr 2026 19:47:18 +0000

TRACE Part 2: What Happens Before You Even Write a Single Line of Code

Continued from: I built a token cost tracker for Claude Code – and it changed the way I think about AI development

An innocent start

Imagine you open a new project, launch Claude Code, and type:

“Hey, I'm just starting a new project. I want to set up a REST API using Express and TypeScript. Can you help me with that?”
Harmless. A normal start. You expect a short answer, maybe a directory structure, a few commands.
What you don’t see: Before Claude even says a word, the model has already consumed thousands of tokens. Not from your question. Not from the answer. But simply from the initial startup.

The numbers no one talks about

I ran the same initial prompt on two different models and compared the results.
Claude Sonnet 4.6:

Category	Tokens
System Prompt	6,100
System Tools	8,100
MCP Tools	141
Memory Files	905
Skills	721
Messages	2,300
Total	~18,100

Claude Opus 4.7:

Category	Tokens
System Prompt	8,400
System Tools	11,500
MCP Tools	251
Memory Files	1,200
Skills	721
Messages	3,600
Total	~25,700

+42% more tokens – for the exact same prompt, on a fresh project without a single line of code.
This is not a mistake. This is the new tokenizer in Opus 4.7.

What a Tokenizer Difference Means in Practice

Anthropic has introduced a new tokenizer with Claude Opus 4.7. The same text is split into more token units than with older models. That sounds technically abstract—but the effects are concrete:
Every single turn in a session carries this base load.
For Turn 1, it’s 18k vs. 25k tokens. By Turn 50, the entire conversation history up to that point is loaded as input—and the tokenizer difference multiplies with every subsequent turn.
A 100-turn session on Opus 4.7 costs more not just because Opus is more expensive per token. It costs more because it structurally generates more tokens—even if you do the same thing as on Sonnet 4.6.

Why /context and /cost alone aren’t enough

Claude Code has built-in visibility tools:

/context shows the current context window usage by category
/cost shows the current session’s consumption so far Both are useful. But both are pull mechanisms—you have to actively remember to call them. You already have to suspect that something is wrong. What happens in reality? You’re working. You’re writing code. You’re in the flow. You don’t call /cost. TRACE is a push mechanism. It comes to you. During the live session, TRACE displays in real time:
Current token count and cost
Session health – green, yellow, or red
Context window utilization as a visual bar
A notification when you cross the warning threshold – before it gets expensive And new since the first version: a Cost Efficiency Section that shows you what the same session would have cost on the baseline model.

The Daily Tokenizer Check

TRACE runs an automatic check early every morning: It sends a fixed reference text to both models—the current one and the baseline model—and calculates the ratio.
The result for Sonnet 4.6 vs. Opus 4.7:

{
  "current_model": "claude-sonnet-4-6",
  "baseline_model": "claude-opus-4-7",
  "current_tokens": 407,
  "baseline_tokens": 521,
  "ratio": 0.7812
}

Sonnet 4.6 uses 22% fewer tokens than Opus 4.7 for the same text. The dashboard shows you this difference—not as an abstract number, but as a concrete cost savings for your last week.

What’s been added since the first article

TRACE has evolved since the first article:

Activity Section with GitHub-style heatmap – streaks, active days, average cost per session
Cost Efficiency Section – Comparison of current model vs. baseline, potential savings per week
Configurable monthly budget – Adjustable directly in the dashboard
Tokenizer Ratio Check – Daily automatic comparison via Anthropic count_tokens API
VS Code Simple Browser Integration – Dashboard directly in VS Code
Unified Installer – bash install.sh automatically detects whether it’s a first-time installation, adding a project, or an update

An Insight into Visibility

The difference between Sonnet 4.6 and Opus 4.7 isn’t just a price difference. It’s a structural difference in how the model processes text. This difference is invisible—until you start measuring.
That is the real message of TRACE: Not “here are your costs,” but “here’s what happened while you were working—and you wouldn’t see it otherwise.”
/context and /cost show you the moment. TRACE shows you the progression.

TRACE is open source under the MIT license: github.com/MyPatric69/trace
If you find it useful: a star helps. If you have questions or find a bug: an issue helps more.

I built a token cost tracker for Claude Code - and it changed the way I think about AI development

Patric — Thu, 23 Apr 2026 16:20:47 +0000

It started with a number I couldn’t explain

A few months ago, I looked at my Anthropic billing dashboard and saw a number that didn’t add up. I knew roughly how much I’d worked with Claude Code. The bill said otherwise.
I couldn’t tell which sessions were expensive. I couldn’t tell why. I only had a total figure at the end of the month—and no way to link it to specific work.
That bothered me. Not because of the money—but because of the lack of visibility. And as someone who’s worked in IT for 30 years, I know: What you can’t measure, you can’t improve.
So I built TRACE.

What I thought I was building—and what I actually built

The original idea was simple: log token consumption per session, calculate costs, store in SQLite. An afternoon project.
What I actually built—over weeks of iteration with Claude Code itself as a development partner—is a local MCP server that:

Tracks token costs per project and session in real time
Detects session health (green / yellow / red) based on configurable thresholds
Sends native macOS notifications when expensive limits are exceeded – no dashboard required
Automatically keeps AI_CONTEXT.md up to date via Git hooks
Generates enriched handoff prompts when starting a new thread
Displays everything in a web dashboard with a 7-day history, provider badges, and live multi-session tracking
Runs directly in the VS Code Simple Browser Panel – no external browser required
Optionally starts automatically at Mac login via LaunchAgent MIT license – free to use, forkable, no restrictions. With solid test coverage ensuring it works in real-world scenarios. The gap between an “afternoon project” and a production-ready open-source tool says something about how AI-powered development really works when you seriously commit to it.

What Nobody Talks About: Context Rot

Here’s what I’ve learned—and what I’ve never heard in any podcast, video, or presentation:
Token costs do not scale linearly with the work. They scale with the session length.
Every turn in a Claude Code session appends to the conversation history. By turn 50, every new message carries the weight of the previous 49 turns as input tokens. By turn 100, you’re burning tens of thousands of tokens per message—just to maintain context, even if the actual task is small.
Anthropic calls this “context rot” in its documentation: As the number of tokens grows, accuracy and recall degrade. The model doesn’t lose tokens—it loses attention. It has to spread its focus across an ever-growing history.
The practical consequence: A 300-turn session doesn’t just cost more than 30 sessions of 10 turns each. It costs significantly more—and the quality of the later turns is measurably worse.

Push Instead of Pull: Why Built-in Commands Aren’t Enough

Claude Code has built-in visibility tools. /cost shows the current session usage. /context visualizes context window utilization. /stats provides usage statistics.
These are pull mechanisms. You have to remember to use them. You have to be curious enough to check. You have to already suspect that something is wrong.
TRACE is a push mechanism. It comes to you.
When a session exceeds 80,000 tokens, a notification—a subtle Tink sound and a macOS alert—appears before it gets really expensive. At 150,000 tokens, a more distinct Funk signals that it’s time for a new thread. You don’t have to remember to check. TRACE checks for you.
The dashboard also runs directly in the VS Code Simple Browser—for those who want to keep everything in one interface.

Feature	`/cost`	`/context`	TRACE
Current session usage	✅	✅	✅
Visual context window	❌	✅	✅
Cache tokens separately	✅	❌	✅
Historical sessions	❌	❌	✅
Cost per project	❌	❌	✅
Monthly budget & alerts	❌	❌	✅
Session health indicator	❌	❌	✅
Push notifications	❌	❌	✅
Handoff prompt	❌	❌	✅
AI_CONTEXT.md auto-update	❌	❌	✅
VS Code Simple Browser	❌	❌	✅
Autostart on Login	❌	❌	✅

There is no universal token limit

A question that quickly arises during use: at what point is a session “too long”? The answer is: it depends.
80,000 tokens as a warning threshold is a good starting point—but a developer working on a small script will reach that after just a few turns, while someone refactoring a complex backend is still far from it.
TRACE therefore makes the thresholds configurable—directly in the dashboard or in the configuration file. Three recommendations for guidance:

Usage Type	Warning	Critical
Economical – cost-conscious	50,000	100,000
Standard – recommended	80,000	150,000
Intensive – large projects	120,000	200,000

These numbers are not set in stone. They are a starting point that should be adjusted once you understand your own usage patterns.

Note: TRACE tracks Claude Code sessions—i.e., work done in the terminal. For claude.ai web/desktop chats, a separate Anthropic Usage API is required, which is only available for Team and Enterprise accounts.

What a real day of usage looks like

I recently installed TRACE for a colleague. After one day of use, the logs told a clear story.
A single session showed over 37 million cache read tokens and nearly $20 in costs. The session had accumulated hundreds of turns—including many invisible tool-use turns that Claude Code generates internally for file reads, Bash commands, and code analysis. Each of these counts as a turn in the transcript, even if the user typed only 30 prompts.
The insight isn’t that something went wrong. The insight is: Without visibility, you don’t even ask the question in the first place.
TRACE makes the invisible visible. That’s all it does—but it turns out to be a lot.

The /resume Trap

One more thing worth knowing: Claude Code’s /resume command is more expensive than it looks.
When resuming a session, Claude Code sends the entire conversation history as input tokens—including invisible “Thinking Block Signatures” from Extended Thinking turns. These are base64-encoded, unreadable, and cannot be truncated. But they are sent to the API with every resume and billed accordingly.
Anthropic’s own GitHub issues document cases where resuming a 24-hour session cost ~156,000 input tokens—before the user had even typed a single character.
Anthropic’s own documentation is clear: Do not rely on session resumption. Save results as state and pass them to a fresh session.
TRACE’s new_session() tool does exactly that—it generates a compressed handoff prompt from AI_CONTEXT.md, CLAUDE.md, current Git changes, and the open task in the backlog. A new thread gets everything it needs in a few hundred tokens instead of hundreds of thousands.

The Bugs That Taught Me the Most

The $0.0000 Sessions. TRACE logged sessions but showed zero cost. The model string in the transcripts was claude-sonnet-4-5-20250929. The config key was claude-sonnet-4-5. Exact matching failed. A line of prefix matching fixed it—but finding it really required digging into the logs.
The health indicator that disappeared after a refresh. Session Health turned red at 150,000 tokens—and then disappeared when the browser was refreshed because the state was stored in a JavaScript variable. The fix was to move the health state to a last_health.json file on the server side. Obvious in hindsight. Not obvious until a real user encountered it.
The AI_CONTEXT.md file that was out of date. The doc synthesizer only updated on feat: and fix: commits. A day with chore: and docs: commits left the context file four days out of date. Removing the commit type filter entirely fixed it.
Every bug taught us something about the gap between “works in theory” and “works when a real person uses it all day.”

Where TRACE stands today

Solid test coverage, all tests passing
Tracks input, output, cache creation, and cache read tokens separately with correct pricing
Session Health: green under 80,000 tokens, yellow up to 150,000, red above that – all configurable
Native notifications on macOS, Windows, and Linux
Web dashboard with 7-day history, dark/light/auto theme, provider badges, live multi-session tracking
Runs in the VS Code Simple Browser – no external browser required
Optional autostart via macOS LaunchAgent
Enhanced Handoff prompts with current phase, open tasks, files to read

What I would tell someone just starting out

The most useful thing I built wasn’t the dashboard. It was the discipline of treating AI_CONTEXT.md as a first-class artifact. Every project gets one. Every commit potentially updates it. Every new session starts by reading it.
An AI assistant with full context is a different tool than one that starts from scratch. TRACE exists to make the former the standard—not the exception.
The second most useful thing: Measure before you optimize. I had strong intuitions about which sessions were expensive. The data contradicted most of them. The expensive sessions weren’t the ones with the difficult problems. They were the ones that ran for a long time.
And the third: Don’t wait for permission to build something useful.

TRACE is open source under the MIT license: github.com/MyPatric69/trace
If you find it useful: a star helps. If you find a bug: an issue helps more. If you build something with it: I’d love to hear about it.