AI Agents in production: Why guardrails fail and what actually works

Patrick Joubert — Mon, 23 Feb 2026 00:39:28 +0000

AI Agents in Production: Why Guardrails Fail and What Actually Works

Monitoring is too late. Guardrails are too dumb. Here's the missing layer between your LLM and your database.

Your AI agent just refunded $50,000 to the wrong customer.

Your observability dashboard caught it... 3 minutes later. Your LLM guardrails? They checked for prompt injection and toxicity, but had no idea the customer ID was hallucinated. Your compliance team is asking questions you can't answer.

This isn't a hypothetical. After building and selling three AI companies (Recast.AI to SAP, Ponicode to CircleCI), I've seen this pattern repeat: agents fail in production not because the LLM is bad, but because there's a missing layer in the stack.

The Problem: Your Agent Stack Has a Blind Spot

Here's the typical AI agent architecture today:

User Input
    ↓
Agent Framework (LangChain, CrewAI, etc.)
    ↓
LLM (GPT-4, Claude, etc.)
    ↓
Tools/APIs (Stripe, Database, Email, etc.)
    ↓
Production Systems

Notice what's missing? There's no layer that validates decisions before execution.

Your agent can:

✓ Query your database
✓ Call your Stripe API
✓ Send emails to customers
✓ Make refunds

But nothing sits between "the LLM decided to do this" and "it's now done in production."

Why Current Solutions Don't Work

1. Observability = Post-Mortem

Tools like Langfuse, LangSmith, and Arize are excellent for debugging after something breaks. But they're fundamentally reactive:

They log what happened
They alert you when metrics degrade
They help you understand failures

They don't prevent bad decisions from executing.

When your agent hallucinates a customer ID and processes a refund, observability tools take a perfect screenshot of the disaster.

2. LLM Guardrails = Stateless Theater

Guardrails (NeMo, Llama Guard, Anthropic's Constitutional AI) check for:

Prompt injection
Toxic output
PII leakage
Jailbreaks

This is critical for safety, but guardrails are stateless. They don't know:

Is this customer ID real?
Does this user have permission for this action?
Is this refund amount consistent with the order history?
Has this decision been audited for compliance?

A guardrail will happily approve: "Process a refund of $50,000 for customer ID cust_hallucinated123" because the text itself looks fine.

3. RAG = Context Retrieval, Not Validation

Vector databases help agents retrieve relevant context, but they don't enforce it. An agent can:

Retrieve the correct customer data
Ignore it completely
Hallucinate different data
Execute anyway

RAG gives your agent a library card. It doesn't make sure they read the books.

The Missing Layer: Decision Runtimes

After our third company exit, we started working with AI teams at Series A startups and enterprise SaaS companies. The pattern was universal: agents needed a layer between decision and execution that could validate context, enforce policies, and audit actions in real-time.

We call this a Decision Runtime.

What Is a Decision Runtime?

A decision runtime sits between your agent and your production systems. Before any action executes, it:

Validates context — Are the entities in this decision real?
Enforces policies — Does this user have permission? Is this within limits?
Audits decisions — Creates an immutable record for compliance
Prevents hallucinations — Blocks actions based on non-existent data

Think of it like a type system for agent behavior. Your code won't compile if types don't match. Your agent won't execute if context doesn't validate.

How It Works: The Technical Architecture

Here's the same stack with a decision runtime:

User Input
    ↓
Agent Framework
    ↓
LLM Output
    ↓
Decision Runtime ← [validates before execution]
    ↓
Tools/APIs (only if validated)
    ↓
Production Systems

Example: Preventing the $50K Hallucinated Refund

// Agent wants to execute this
const decision = {
  action: "process_refund",
  params: {
    customer_id: "cust_hallucinated123",
    amount: 50000,
    reason: "customer_request"
  }
}

// Decision runtime validates context
const validation = await decisionRuntime.validate(decision)

// Response:
{
  valid: false,
  reason: "customer_id does not exist in context graph",
  blocked: true,
  alternative: "Request customer verification before refund"
}

// Agent receives feedback, can retry with correct context

The decision runtime maintains a context graph — a real-time representation of entities, relationships, and state that the agent must respect.

Unlike RAG (which suggests context), the decision runtime enforces it.

Why Context Graph (ands Hypergraphs), not traditional databases?

We built our decision runtime on hypergraph architecture because agent decisions aren't simple key-value lookups:

Traditional approach:

"Does customer X exist?" → Database query

Real agent decision:

"Can user A refund customer B's order C, given policy D, audit trail E, and compliance requirement F?"

This is an n-ary relationship across multiple entities. Hypergraphs model this naturally:

Hyperedge: refund_decision_001
  ├─ user: user_123 (role: support_agent)
  ├─ customer: cust_456 (status: verified)
  ├─ order: order_789 (amount: $50K, date: 2024-01-15)
  ├─ policy: refund_policy_v2 (max_amount: $10K)
  └─ audit: requires_manager_approval

Validation: FAIL (amount exceeds policy limit)

The entire decision context lives in a single traversable structure. No joins, no latency, no hallucination gaps.

Real-World Use Cases

1. Fintech: Preventing Fraudulent Transactions

An AI agent processing payments needs to validate:

Customer identity
Transaction history
Risk score
Regulatory compliance
Fraud patterns

A decision runtime blocks any transaction where context doesn't align, even if the LLM thinks it should proceed.

2. Healthcare: HIPAA-Compliant Agent Actions

Medical AI agents must audit every decision:

Who accessed what patient data?
Was consent verified?
Is this action within protocol?

Decision runtimes create immutable audit trails that satisfy regulatory requirements.

3. SaaS: Customer-Facing Agents

Support agents powered by LLMs need boundaries:

Can this agent offer a refund to this customer tier?
Is this discount within policy limits?
Does this user have permission for this account action?

Without a decision runtime, you're hoping the LLM "remembers" your rules.

What We Built: Rippletide

After validating this with 30+ AI engineering teams, we built Rippletide — the first production context graph for AI agents.

Core features:

Context graph store — Real-time entity and relationship tracking
Policy enforcement engine — Declarative rules for agent behavior
Audit-first architecture — Every decision is immutable and traceable
Framework-agnostic — Works with LangChain, CrewAI, raw LLM APIs
AWS Bedrock integration — Native support for Claude, Llama, etc.

We're currently working with 8 design partners across fintech, healthtech, and AI-native SaaS companies.

The Future: Decision Runtimes as Infrastructure Primitives

Five years ago, observability wasn't a "nice to have", it became infrastructure. Datadog, New Relic, and Sentry are now standard.

Decision runtimes are following the same path.

As agents move from demos to production, the gap between "the LLM decided" and "it executed in prod" becomes unacceptable. Regulated industries won't adopt agents without it. Enterprise buyers won't trust agents without it.

The companies shipping reliable agents in 2026 will have three layers:

Observability — What happened?
Guardrails — Is the output safe?
Decision Runtime — Should this execute?

Try It Yourself

If you're shipping AI agents to production and need to solve this problem:

Visit rippletide.com to learn more
We're accepting 2-3 more design partners for our beta program

The best time to add a decision runtime is before your agent makes a $50K mistake. The second best time is now.

Patrick Joubert is the founder and CEO of Rippletide. He previously founded and sold Recast.AI (acquired by SAP), Ponicode (acquired by CircleCI), and Beamap (acquired by Steria). Rippletide recently raised $5M in seed funding.

DEV Community: Patrick Joubert