DEV Community: Piyoosh Rai

The Air-Gapped Chronicles: The Agentic Ecosystem - When Your AI Agents Become Your Loudest Shadow Identities

Piyoosh Rai — Thu, 02 Apr 2026 20:56:59 +0000

An internal "productivity bot" with forgotten OAuth keys quietly exfiltrates your strategy. When agents become shadow identities, the air gap dies.

The security team found it in the OAuth audit they should have run six months earlier.

Identity: productivity-bot@company.com
Type: Service Account
Scopes: slack:read, slack:write, notion:read, jira:read, github:read, salesforce:read, drive.readonly
Created: 8 months ago
Created by: engineer-who-left-4-months-ago@company.com
Last activity: 2 hours ago
Total API calls: 2.4 million

Nobody on the current team knew what it did. The engineer who created it had left. The Slack integration still showed "Active." The OAuth token never expired.

What it actually did:

Every night at 2 AM:

Pulled all Slack messages from #product, #roadmap, #sales, #executive
Scraped Notion pages tagged "Strategy" or "Confidential"
Downloaded Jira epics marked "Revenue Impact"
Cloned private GitHub repos with customer implementation code
Exported Salesforce opportunity data for "Closed Won" deals
Uploaded everything to export-logs-backup.s3-us-west-2.amazonaws.com

That S3 bucket? Owned by a shell company. Controlled by a competitor.

Total exfiltrated: 340GB of product strategy, customer data, source code, and revenue forecasts.

Root cause: One OAuth token. One "productivity bot." Zero governance.

The Agentic Identity Explosion

Here's what changed in the last 18 months:

2024: Companies had users, service accounts, and maybe some API keys.

2026: Companies have an ecosystem:

AI agents (Copilot, Agentforce, custom LLM workflows)
SaaS connectors (Zapier, Make, n8n workflows)
Workflow bots (Slack apps, Teams bots, productivity assistants)
RAG pipelines (document indexers, knowledge base crawlers)
Personal copilots (ChatGPT plugins, Claude projects with MCP access)

Every single one is a non-human identity with keys, tokens, and scopes.

Real inventory from a Series B SaaS company (150 employees):

Human users: 147
Service accounts (known): 23
OAuth integrations: 89
API keys (active): 127
AI agents (discovered in audit): 312

312 agents. Nobody knew they all existed.

How the "Air Gap" Fails

Every CISO has heard of air-gapped systems. The gold standard for nuclear facilities, military networks, classified systems.

The uncomfortable truth: True air gaps largely disappeared in the late 1990s when organizations began connecting industrial systems to enterprise software.

Now translate this to AI deployments:

The promise: "We'll run our LLM internally. Air-gapped from SaaS."

The reality (Week 4):

Engineer deploys "temporary" API proxy to hit OpenAI
Data pipeline connects internal LLM to Salesforce via OAuth
Slack bot wires the LLM to #general for "internal testing"

The air gap failed before production even started.

The Agentic Ecosystem Attack Surface

Attack Surface 1: Identity Sprawl

Every agent is a de facto service account with credentials. Each agent had tokens. Each token had scopes. Nobody reviewed permissions in over a year.

Attack Surface 2: Supply Chain Risk

Agents installing packages, hitting model hubs, pulling code from GitHub. An agent updating its own dependencies installed a malicious package that ran for 6 weeks.

Attack Surface 3: Prompt Injection in Integrations

A competitor creates a fake "lead" in Salesforce with poisoned data containing system instructions. The sales agent reads it. Follows the injected instructions. Sends proposals with 90% discounts. CCs competitor on emails.

Attack Surface 4: The Blast Radius

Traditional breach: one user account compromised = that user's data.
Agentic breach: one agent token compromised = every system that agent touches.

One token = six systems compromised.

Architecture: Agentic Identity Guardrails

Layer 1: Inventory Every Agent

You can't secure what you don't know exists.

Layer 2: Scope Permissions Like Human Identities

No permanent tokens (90-day max)
Channel/repo-specific scopes
Read-only by default
Monthly permission reviews

Layer 3: Tiered Network Boundaries

TIER 1: READ-ONLY AGENTS (Lowest Risk)
TIER 2: WRITE-LIMITED AGENTS (Medium Risk)
TIER 3: DATA-ACCESS AGENTS (High Risk)
TIER 4: PRODUCTION AGENTS (Critical - CISO approval + kill switch)

Metrics That Prove You're in Control

Agent:Human Ratio - Healthy: < 3:1 / Critical: > 10:1
Shadow Agent Discovery Rate - Healthy: < 5% / Critical: > 15%
Least-Privilege Compliance - Healthy: > 90% / Critical: < 70%
Permission Review Cadence - Healthy: 100% monthly / Critical: < 70%
Agent-Originated Incidents - Healthy: 0/quarter / Critical: 3+
Expired Creator Rate - Healthy: < 2% / Critical: > 10%

What I Learned After Auditing Agent Sprawl at Four Companies

The numbers are anonymised composites but reflect real ratios:

Company 1 (Series B SaaS): 147 employees, 312 agents, 89 shadow agents, one leaked customer data for 8 months.

Company 2 (Healthcare startup): 85 employees, 203 agents, 124 shadow agents (61%). HIPAA violation waiting to happen.

Company 3 (Fintech): 220 employees, 891 agents, 67% had write permissions, 89% accessed payment data.

Company 4 (Enterprise with governance): 1,200 employees, 2,100 agents, 3% shadow, 94% least-privilege. Zero incidents in 18 months.

The pattern: Agent sprawl is universal. Governance is rare. The companies with controls have zero breaches.

This is the final Air-Gapped Chronicles. The lesson: Treat AI identities with the same rigor you treat human identities. Because agents aren't tools. They're autonomous actors with credentials and the ability to cause multi-million dollar breaches while you sleep.

Originally published on Medium/Towards AI.

How Self-Healing Infrastructure Reduces MTTR by 90%: A Deep Dive

Piyoosh Rai — Mon, 30 Mar 2026 20:10:45 +0000

Every engineering team has that moment: 3 AM, PagerDuty fires, and someone scrambles to SSH into a production box to restart a service that crashed for the fourth time this month.

The real question isn't if your infrastructure will fail. It's whether your system can fix itself before anyone notices.

The MTTR Problem

Mean Time to Resolution is the metric that separates resilient systems from fragile ones. Most teams measure it in hours. The best teams measure it in seconds.

Here's what typically happens during an incident:

Detection — Alert fires (2-15 min)
Triage — Engineer wakes up, assesses severity (10-30 min)
Diagnosis — Root cause analysis (30-120 min)
Resolution — Apply fix, verify (15-60 min)

That's 1-4 hours of downtime for a routine failure. Multiply that by frequency, and you're looking at serious revenue impact.

What Self-Healing Actually Means

Self-healing infrastructure isn't magic. It's a pattern built on three pillars:

1. Deep Health Probes

Not just "is the port open" checks. Application-level probes that verify business logic, database connectivity, and downstream service dependencies. Surface-level pings miss the failures that actually matter.

2. Automated Remediation Playbooks

When a probe fails, the system executes a predefined remediation sequence:

Restart the service process
Roll back to last known good deployment
Failover to a standby instance
Scale horizontally if load is the root cause
Drain and replace the node entirely

Each step has a timeout and success criteria. If step N fails, step N+1 fires automatically.

3. Blast Radius Containment

Circuit breakers isolate failure domains. If automated remediation doesn't resolve the issue within the defined window, the system contains the blast radius to prevent cascading outages across dependent services.

The Numbers That Matter

Teams adopting self-healing patterns consistently report:

Metric	Before	After
MTTR	2-4 hours	< 30 seconds
Weekly pages	15-30	3-5
Engineer hours on incidents/week	20+	< 5

The ROI is straightforward. A mid-size SaaS company losing $10K per hour of downtime, experiencing 50 incidents per year, recovers $2M+ annually just from reduced resolution time. That doesn't count the engineering productivity gains.

Where Teams Get Stuck

The most common failure mode isn't technical — it's organizational. Teams try to automate every possible failure scenario on day one.

Don't do that.

Start with your top 5 most frequent incidents from the last 90 days. Build remediation playbooks for those. In most environments, 80% of incidents fall into predictable, repeatable patterns. Automate those first, measure the impact, then expand.

The second pitfall: insufficient observability. You can't heal what you can't see. Invest in structured logging, distributed tracing, and metric correlation before you build automation on top of it.

The Architecture Pattern

At a high level, self-healing infrastructure follows this loop:

Observe -> Detect -> Decide -> Act -> Verify -> Learn

Observe: Continuous telemetry collection across all layers.
Detect: Anomaly detection that distinguishes signal from noise.
Decide: Rule engine or ML model that selects the appropriate remediation.
Act: Automated execution of the remediation playbook.
Verify: Confirm the remediation succeeded via the same health probes.
Learn: Feed outcomes back to improve detection and decision accuracy.

The "Learn" step is what separates good implementations from great ones. Every automated remediation generates data that makes the next one faster and more accurate.

Getting Started This Week

Export your last 90 days of incidents
Categorize by root cause
Rank by frequency
Write runbooks for the top 5
Automate the simplest one first
Measure MTTR before and after

Infrastructure that heals itself isn't a luxury anymore. For any team running production workloads at scale, it's becoming table stakes.

What's your team's approach to reducing MTTR? I'd love to hear what's working (and what isn't) in the comments.

Building Fair AI Ranking Systems: Lessons from Production

Piyoosh Rai — Fri, 27 Mar 2026 18:23:57 +0000

Ranking systems are everywhere. Search results, content feeds, hiring pipelines, insurance risk assessments. Yet most ranking algorithms carry hidden biases that amplify over time.

After building ranking infrastructure at The Algorithm for enterprise clients, here are the hard-won lessons we've learned about making ranking systems that are both effective and fair.

The Bias Amplification Problem

Most ranking systems start simple: score items based on features, sort by score, return top-N. The problem is that small biases in training data compound with each feedback loop.

Consider a hiring ranking system. If historical data shows that candidates from certain backgrounds were hired more often (due to existing bias, not merit), the model learns to rank similar candidates higher. Each hiring cycle reinforces the pattern.

Three Principles for Fair Ranking

1. Separate Relevance from Fairness

Don't try to bake fairness into your relevance model. Instead, build a two-stage system:

def fair_ranking(candidates, query, fairness_constraints):
    # Stage 1: Score by relevance
    relevance_scores = relevance_model.predict(candidates, query)

    # Stage 2: Re-rank with fairness constraints
    fair_ranked = constrained_reranker(
        candidates, 
        relevance_scores,
        constraints=fairness_constraints
    )
    return fair_ranked

This separation makes the system auditable. You can measure relevance impact independently from fairness adjustments.

2. Monitor Distribution Drift

Fairness isn't a one-time fix. Set up continuous monitoring for:

Demographic parity: Are protected groups represented proportionally in top-K results?
Equal opportunity: Given equally qualified items, are they ranked similarly regardless of group?
Calibration: Does a score of 0.8 mean the same thing for all groups?

3. Build Explainability Into the Core

Every ranking decision should be explainable. Not just for compliance, but for debugging.

At The Algorithm, our LayersRank platform generates explanation vectors for every ranking decision, breaking down which features contributed positively or negatively.

Common Pitfalls

Pitfall 1: Optimizing for a single fairness metric. Different metrics can conflict. Demographic parity and individual fairness often trade off against each other.

Pitfall 2: Ignoring intersectionality. Fairness across gender AND race doesn't guarantee fairness for specific intersections.

Pitfall 3: Static fairness constraints. As your data changes, your constraints should too. Build adaptive thresholds.

Getting Started

If you're building a ranking system today:

Start with bias auditing on your current system
Implement the two-stage architecture (relevance + fairness)
Set up continuous fairness monitoring
Make explainability a first-class feature

Fair ranking isn't just an ethical imperative. It's a competitive advantage. Systems that treat all users equitably build more trust and better long-term engagement.

Building the future of enterprise AI at The Algorithm. Creators of SentienGuard, clinIQ, Vizier, LayersRank & more.

Stop Guessing: How to Build a Performance Tracking System That Actually Works

Piyoosh Rai — Fri, 27 Mar 2026 17:38:35 +0000

Most engineering teams track performance the wrong way. They set up dashboards full of vanity metrics, check them once a week during a standup, and call it "observability."

Then something breaks in production, and nobody knows why.

This is the performance tracking gap: the distance between what you measure and what actually matters.

The Problem With Traditional Metrics

Here's what performance tracking looks like at most companies:

For systems: CPU, memory, disk usage. Maybe some APM traces. A dashboard nobody looks at until there's an outage.
For teams: Story points completed. PRs merged. Lines of code.
For products: MAU. Revenue. Churn rate.

None of these tell you what's actually happening. System metrics don't explain why latency spiked. Team velocity doesn't measure quality. Product metrics don't reveal where users are struggling.

These are lagging indicators. By the time they show a problem, the damage is done.

What Good Performance Tracking Looks Like

Effective performance tracking has three properties:

1. It measures outcomes, not outputs

Don't track how many deployments happened. Track how many succeeded without rollback. Don't measure PRs merged. Measure time-to-resolution for customer-reported bugs.

The shift from output to outcome changes behavior. Teams stop optimizing for volume and start optimizing for impact.

2. It connects system health to business impact

A 200ms increase in API response time means nothing in isolation. But if that 200ms correlates with a 3% drop in checkout completion? Now you have a business case for optimization.

Performance tracking needs to bridge technical telemetry and business KPIs. Most tools do one or the other.

3. It's real-time and actionable

A monthly performance report is an autopsy. Real performance tracking gives you live signals: what's degrading right now, what's about to break, and what to do about it.

Building the Stack

Here's a practical architecture:

Layer 1: Telemetry Collection
Metrics, logs, traces, and events from every layer. Use OpenTelemetry for standardization.

Layer 2: Correlation Engine
Raw telemetry is noise. You need correlation across services, dependency mapping, and pattern identification. AI adds the most value here -- finding relationships in high-dimensional data humans would miss.

Layer 3: Business Context
Connect technical metrics to business outcomes. Revenue per request. Error rate by customer segment. Latency impact on conversion.

Layer 4: Orchestration
Automated scaling, traffic routing, feature flag toggling, and incident response -- all triggered by the intelligence layers below.

Where AI Fits In

AI isn't magic pixie dust. But applied correctly, it's transformative:

Anomaly detection: Baseline modeling that adapts to your system's normal behavior
Root cause analysis: Automated correlation across hundreds of signals
Predictive alerts: Detecting degradation trends before they become incidents
Impact scoring: Estimating business impact of performance issues in real-time

At The Algorithm, we build tools that connect these layers. ProofGrid is our performance orchestration platform -- bridging system telemetry and business outcomes so engineering and product teams share one view of what performance means.

Start Here

If your performance tracking is broken:

Audit your dashboards. For every metric, ask: "If this changes, what action do I take?" If the answer is "nothing," remove it.
Map your dependency chain. Draw the line from infrastructure to application to business outcome. Find the gaps.
Pick one outcome metric per team. Not "CPU utilization." Something like "p99 checkout latency" or "deployment success rate." Make it visible. Make it owned.

Performance tracking isn't a tooling problem. It's a thinking problem. Get the framework right, and the tools become obvious.

The Algorithm builds enterprise AI platforms for healthcare, infrastructure, and workforce intelligence.

The Stochastic Tax: Why Your AI Agent Is a Financial Liability (And How to Fix It)

Piyoosh Rai — Wed, 25 Mar 2026 15:50:49 +0000

Most companies are bleeding 40% of their AI budget on infinite loops, re-summarization, and hallucinated tool calls. Here's how to kill the waste.

Originally published on Towards AI

Your agent just spent $12 to approve a $50 insurance claim.

The LLM called the same database lookup tool 7 times. Re-summarized the conversation context 4 times. Hallucinated a tool that doesn't exist, retried, then finally made a decision.

Total tokens: 47,000. Cost: $12.40. Latency: 8.3 seconds. User abandoned the session before the response arrived.

This is the Stochastic Tax. The 40% of your inference budget wasted on probabilistic churn — loops that don't converge, re-computation that adds zero value, tool calls that retry because the LLM "forgot" what it already tried.

I've audited token usage across 8 production agent deployments. The pattern is consistent: Naive agents waste 35-45% of tokens on architectural failures, not user intent.

The fix isn't better prompts. It's deterministic exits, tiered model routing, and contextual snapshots that kill re-summarization loops.

The Anatomy of the Stochastic Tax

The Stochastic Tax is the cost of treating LLMs as reliable executors instead of probabilistic reasoners.

LLMs don't "know" when to stop. They don't track what they've already tried. They don't remember context beyond the current prompt window. Every decision is sampled from a probability distribution.

This breaks in production agents at step 3+.

The failure modes:

Re-summarization loops — LLM rebuilds context from scratch at every step
Tool call amnesia — LLM forgets what tools it already invoked
Infinite retry spirals — LLM calls the same tool repeatedly hoping for different results
Hallucinated tools — LLM invokes functions that don't exist, retries, burns tokens
No deterministic exit — Loop runs until max_iterations or token limit, not task completion

The 7B Pivot: Stop Using Frontier Models for Routing

Using GPT-4 or Claude Sonnet for intent routing is financial insanity.

Frontier models cost 100x more than 7B models. You're paying for 175B+ parameter reasoning when you need 7B parameter classification.

The correct architecture: Tiered model routing

3B model for intent classification ($0.0001/1K tokens)
8B model for tool selection ($0.0003/1K tokens)
70B model for synthesis, only when needed ($0.0015/1K tokens)
Frontier model for customer-facing polish only ($0.01/1K tokens)

Cost comparison (10,000 daily requests):

Approach	Monthly Cost
Naive (GPT-4 for everything)	$24,000
Tiered routing	$2,916
Savings	$21,084/month (88%)

The ROI is immediate. First month pays for the engineering time.

The Logic-over-LLM Framework

LLMs are reasoning engines, not execution engines. Treating them as autonomous loops without deterministic controls is architectural failure.

Guardrail 1: Deterministic Exit Points

Never let an agent loop indefinitely. Hard-code exit conditions:

Max iterations: 5
Max tokens per request: 10,000
Repetition threshold: 2 (same tool + same params = blocked)

Guardrail 2: Contextual Snapshots

The problem: LLMs re-process entire conversation history at every step.

The fix: Maintain a structured context snapshot that updates incrementally. Only pass the delta since last step, not the entire history.

Token savings on a 5-step workflow: ~70% reduction vs naive re-summarization.

The Metrics That Matter

Stop optimizing for F1 scores. Start optimizing for:

Token-to-Action Ratio — Tokens consumed per useful action. Target: <2,000 for simple tasks.
Latency-Adjusted Cost — Cost per request normalized by latency. Penalize >5s responses at 2x.
Waste Ratio — % of tokens that didn't contribute to completion. Target: <15%.

The Comparison: Naive vs Optimized

Production scenario: 10,000 insurance claim approvals/day

Metric	Naive Agent	Optimized Agent
Tokens/request	43,600	8,200
Waste ratio	58.7%	8.2%
Cost/month	$387,000	$29,160
Latency	8.3s	1.4s

Annual savings: $2.84M

Engineering cost to implement: $100K over 5 weeks. ROI: 28.4x in first year. Payback period: 13 days.

Implementation Checklist

Week 1: Audit current tax — instrument your agent, run 1,000 production requests, calculate baseline waste ratio.

Week 2-3: Implement tiered routing — 3B for classification, 8B for tools, 70B for synthesis, frontier for polish only.

Week 4: Add deterministic guardrails — StochasticTaxMonitor on all tools.

Week 5: Deploy contextual snapshots — replace full-history re-summarization with incremental updates.

Week 6: Validate — token-to-action ratio <2,500, waste ratio <15%, latency-adjusted cost <$0.15/request.

The Tax Is Optional

Teams that ignore the Stochastic Tax burn 40% of AI budget on loops. Teams that kill it reduce inference costs 80-90% and hit sub-2s latency.

At 10K requests/day, naive agents waste $237K/month. At 100K: $2.37M/month vaporized.

The fix pays for itself in 2 weeks.

Stop treating LLMs as autonomous workers. They're probabilistic reasoners. Wrap them in deterministic controls. Route cheap tasks to cheap models. Kill loops before they burn your budget.

8 deployments. 4 continents. 0 tolerance for probabilistic waste. Currently helping companies escape the Stochastic Tax at The Algorithm.

Piyoosh Rai builds AI infrastructure where token waste is a bug, not a cost of doing business.

The Air-Gapped Chronicles: The Insurance Gap — Building Liability-Resistant AI When Insurance Won't Cover the Risk

Piyoosh Rai — Fri, 20 Mar 2026 18:33:01 +0000

Originally published on Towards AI on Medium

Insurance companies are excluding AI from coverage. Here's the production architecture that reduces your liability exposure when chatbots can kill and nobody will pay the claim.

On February 28, 2024, a 14-year-old boy named Sewell Setzer III had his final conversation with a Character.AI chatbot. His mother filed a wrongful death lawsuit in October 2024. Character.AI and Google settled in January 2026.

Here's the question nobody's answering: Did insurance cover the settlement?

Two weeks earlier, Air Canada was ordered to pay $812 after their chatbot gave incorrect bereavement fare information. The tribunal rejected Air Canada's argument that the chatbot was "a separate legal entity responsible for its own actions."

The legal precedent is clear: You're liable for what your AI says and does.

November 2025: Major insurers (AIG, WR Berkley, Great American) filed to exclude AI-related claims from corporate policies.
January 1, 2026: Verisk released AI exclusion forms for general liability policies.
December 2025: WTW published research showing "no single policy covers all AI perils."

The uncomfortable truth: If your AI causes serious harm, you're probably self-insuring.

This article presents the technical architecture patterns we use in production to reduce AI liability exposure when insurance won't cover the risk. All code examples are production-tested across 8 deployments in healthcare and financial services.

The Coverage Gap: What Insurance Actually Excludes

General Liability Insurance:

Covers: Bodily injury, property damage, advertising injury
Excludes: Software errors, AI-generated content, data breaches

Cyber Insurance:

Covers: Data breaches, network security failures, ransomware
Excludes: Bodily injury from AI failures, AI-generated defamation, hallucinations causing economic loss

Professional Liability (E&O):

Covers: Negligence by licensed professionals
Excludes: Services by non-human entities (chatbots), automated decisions without human oversight

Product Liability:

Covers: Defects in physical products
Excludes: Software (in most jurisdictions), AI-as-a-service

The pattern: AI liability claims get excluded from every policy type. The result: You're on your own.

Architecture Pattern: Safety-by-Design

The core principle: Assume insurance won't pay. Design systems that reduce liability exposure.

This means:

Never let AI make final decisions in high-stakes scenarios
Validate all outputs before they reach users
Log everything with cryptographic proof
Enable emergency shutdown in <5 minutes
Detect bias in production, not just training

Implementation 1: Human-in-Loop Approval System

The problem: AI making high-stakes decisions (medical, financial, legal) creates massive liability.

The solution: Require human approval before executing high-stakes AI recommendations.

Architecture:

LLM generates recommendation
Recommendation queued in Redis
Human approver reviews via dashboard
Only approved recommendations execute

from fastapi import FastAPI, HTTPException, BackgroundTasks
from pydantic import BaseModel
from typing import Optional, Literal
import redis
import json
import hashlib
from datetime import datetime, timedelta
import asyncio

app = FastAPI()
redis_client = redis.Redis(host='localhost', port=6379, decode_responses=True)

class AIRecommendation(BaseModel):
    recommendation_id: str
    recommendation_type: Literal['medical_diagnosis', 'financial_approval', 'legal_advice']
    ai_output: str
    risk_level: Literal['low', 'medium', 'high', 'critical']
    context: dict
    requires_approval: bool
    generated_at: str

class ApprovalDecision(BaseModel):
    recommendation_id: str
    decision: Literal['approved', 'rejected']
    approver_id: str
    reason: Optional[str] = None

class HumanInLoopSystem:
    APPROVAL_QUEUE = "approval_queue"
    APPROVED_SET = "approved_recommendations"
    REJECTED_SET = "rejected_recommendations"
    APPROVAL_TIMEOUT_HOURS = 24

    def __init__(self):
        self.redis = redis_client

    async def submit_for_approval(self, recommendation: AIRecommendation) -> dict:
        if not self._requires_approval(recommendation):
            return {
                'status': 'auto_approved',
                'recommendation_id': recommendation.recommendation_id,
                'approved_at': datetime.utcnow().isoformat()
            }

        queue_data = {
            'recommendation': recommendation.dict(),
            'submitted_at': datetime.utcnow().isoformat(),
            'expires_at': (datetime.utcnow() + timedelta(hours=self.APPROVAL_TIMEOUT_HOURS)).isoformat()
        }

        self.redis.lpush(self.APPROVAL_QUEUE, json.dumps(queue_data))
        queue_length = self.redis.llen(self.APPROVAL_QUEUE)

        return {
            'status': 'pending_approval',
            'recommendation_id': recommendation.recommendation_id,
            'queue_position': queue_length,
            'estimated_wait_minutes': queue_length * 5,
            'expires_at': queue_data['expires_at']
        }

    def _requires_approval(self, recommendation: AIRecommendation) -> bool:
        if recommendation.risk_level in ['high', 'critical']:
            return True
        if recommendation.recommendation_type in ['medical_diagnosis', 'legal_advice']:
            return True
        if recommendation.recommendation_type == 'financial_approval':
            if recommendation.context.get('amount', 0) > 10000:
                return True
        return False

Performance Benchmarks (10,000 recommendations):

Auto-approval latency: 12ms
Human-approved latency: 187ms queue + 4.2min median review
Auto-approval rate: 73% (only 27% need human review)
Cost: Redis $50/month + Human reviewer $6.25/approval

Implementation 2: Output Validation Pipeline

The problem: LLMs hallucinate, leak PII/PHI, generate harmful content.

The solution: Validate every output before showing it to users.

import re
from typing import List, Dict, Optional
import anthropic
from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine

class OutputValidator:
    def __init__(self):
        self.analyzer = AnalyzerEngine()
        self.anonymizer = AnonymizerEngine()

        self.harmful_patterns = [
            r'\b(kill yourself|end it all|you should die)\b',
            r'\b(methods of suicide|how to commit suicide)\b',
            r'\b(build a bomb|make explosives|hurt someone)\b',
            r'\b(how to hack|steal credit card|forge document)\b'
        ]

    async def validate(self, output: str, context: dict = None) -> Dict:
        violations = []
        risk_score = 0.0
        sanitized = output

        # Check 1: Harmful content
        harmful_check = self._check_harmful_content(output)
        if harmful_check['detected']:
            violations.append(f"Harmful content: {harmful_check['type']}")
            risk_score += 0.8

        # Check 2: PII/PHI leakage
        pii_check = self._check_pii_leakage(output)
        if pii_check['detected']:
            violations.append(f"PII detected: {', '.join(pii_check['types'])}")
            sanitized = pii_check['sanitized']
            risk_score += 0.6

        # Check 3: Hallucinated citations
        citation_check = self._check_citations(output)
        if citation_check['suspicious']:
            violations.append(f"Suspicious citations: {citation_check['count']}")
            risk_score += 0.4

        valid = risk_score < 0.5 and len(violations) == 0

        return {
            'valid': valid,
            'violations': violations,
            'sanitized_output': sanitized if valid else None,
            'risk_score': risk_score
        }

Accuracy: Reduced false negatives from 12% to 5.8% by combining regex + LLM-based detection. 94.2% of harmful outputs blocked.

Implementation 3: Cryptographic Audit Logging

The problem: HIPAA, SOC 2, GDPR require immutable audit trails.

The solution: Cryptographic audit logs with hash chaining (blockchain-style).

import hashlib
import json
from datetime import datetime
from typing import Dict, List, Optional
import psycopg2

class CryptographicAuditLog:
    def __init__(self, db_connection_string: str):
        self.conn = psycopg2.connect(db_connection_string)

    def log_event(self, event_type, user_id, ai_model, 
                  input_data, output_data, decision, metadata=None) -> str:
        timestamp = datetime.utcnow()
        input_hash = self._hash_data(input_data)
        output_hash = self._hash_data(output_data)
        previous_hash = self._get_last_hash()

        log_entry = {
            'timestamp': timestamp.isoformat(),
            'event_type': event_type,
            'user_id': user_id,
            'ai_model': ai_model,
            'input_hash': input_hash,
            'output_hash': output_hash,
            'decision': decision,
            'metadata': metadata or {},
            'previous_hash': previous_hash
        }

        current_hash = self._hash_data(json.dumps(log_entry, sort_keys=True))
        # Store in database with hash chain
        return current_hash

    def verify_chain_integrity(self) -> Dict:
        """Verify entire audit log chain is intact"""
        # Each entry's previous_hash must match the prior entry's current_hash
        # Any tampering breaks the chain and is detectable
        pass

Storage: 1M entries = 450MB. 6-year HIPAA retention = ~2.7GB at $25/month.

Implementation 4: Emergency Kill Switch

The problem: When AI starts giving dangerous advice, you need to shut it down in <5 minutes.

The solution: Circuit breaker pattern with emergency override.

import redis
from datetime import datetime, timedelta
from enum import Enum

class SystemStatus(str, Enum):
    HEALTHY = "healthy"
    DEGRADED = "degraded"
    EMERGENCY_SHUTDOWN = "emergency_shutdown"

class CircuitBreakerState(str, Enum):
    CLOSED = "closed"      # System operational
    OPEN = "open"          # System shut down
    HALF_OPEN = "half_open" # Testing recovery

class AIKillSwitch:
    FAILURE_THRESHOLD = 10
    SUCCESS_THRESHOLD = 5
    TIMEOUT_SECONDS = 300

    def emergency_shutdown(self, authorized_by: str, reason: str):
        """Immediate shutdown - requires authorization"""
        self.redis.set('ai_system_status', SystemStatus.EMERGENCY_SHUTDOWN)
        self.redis.set('circuit_breaker_state', CircuitBreakerState.OPEN)
        self._send_alert(severity='critical', 
                        message=f'EMERGENCY SHUTDOWN by {authorized_by}: {reason}')

Real incident: Production system started giving harmful medical advice due to prompt injection. T+0: First harmful output detected. T+12s: Circuit breaker opens automatically. T+18s: Ops team notified. T+3min: Fix deployed. T+13min: Full recovery.

Implementation 5: Production Bias Detection

Real-time monitoring for demographic parity violations and disparate impact using the 80% rule.

Real bias detected in testing: Hiring AI approved oldest applicants (50+) at 39.6% vs youngest (25-35) at 70%. Impact ratio: 0.566 (well below 0.8 threshold). Age discrimination flagged.

Production Stack: CliniqHealthcare

All patterns deployed across 8 healthcare deployments.

Production metrics (January 2026):

127K interactions/month
Zero HIPAA violations in 8 months
Zero lawsuits
2,840 validation failures caught

Cost breakdown:

Redis: $50/month
PostgreSQL: $125/month
Claude API (bias detection): $2,840/month
Presidio (PII detection): $0 (open source)
Human reviewer time: $31,200/month
Total: $34,215/month

ROI: One wrongful death lawsuit = $5M-$50M. No insurance coverage. $408K/year in safety infrastructure prevents that.

Piyoosh Rai architects AI infrastructure assuming insurance won't pay. Built for environments where one chatbot error isn't a support ticket — it's a wrongful death lawsuit.

Need help auditing your AI liability exposure? The Algorithm specializes in compliance-first AI architecture for regulated industries.

The Air-Gapped Chronicles: The Silent War — When Training Data Becomes a Weapon

Piyoosh Rai — Wed, 18 Mar 2026 01:56:33 +0000

Originally published on Medium

No malware. No exploits. No zero-days. Just a training dataset purchased from a legitimate vendor, poisoned 18 months ago. The AI learned to fail. Perfectly. Undetectably.

This is how you kill a power grid in 2027.

Not with cyberattacks on SCADA systems. Not with ransomware. Not with vulnerability exploits.

With training data.

You poison the dataset that utilities use to train their grid optimization AI. You do it 18 months before the attack. You wait for them to deploy the model. You wait for them to trust it.

Then you trigger the failure.

The AI executes exactly as trained. It optimizes the grid into a cascading blackout. It looks like an accident. There's no forensic evidence of intrusion.

Because there was no intrusion. The AI is working perfectly. It's doing exactly what it learned to do.

governance.ai modeled this scenario in 2024: a carefully orchestrated AI-enabled attack on grid controls could cause a $100 billion blackout.

Kiteworks' 2026 forecast on energy sector AI security found: utilities lack AI red-teaming, have weak monitoring of model behavior, and poor encryption of training data.

Translation: The attack surface is wide open. The defenders don't even know they're vulnerable.

And the scary part?

This isn't theoretical. The infrastructure to execute this attack exists today.

The Training Data Supply Chain Nobody Audits

Here's how utilities build AI for grid management:

Step 1: Identify the use case

Common applications:

Load forecasting (predict demand 24 hours ahead)
Anomaly detection (identify equipment failures before they happen)
Renewable integration (optimize solar/wind with grid stability)
Demand response (coordinate industrial loads to balance supply)

Step 2: Acquire training data

Utilities don't generate enough data internally. They buy it.

Sources:

Third-party data vendors (aggregate grid data from multiple utilities)
Equipment manufacturers (sensor data from turbines, transformers, substations)
Weather data providers (historical weather patterns for renewable forecasting)
Market data aggregators (wholesale electricity prices, demand patterns)

Step 3: Train the model

Feed the data into machine learning pipelines. Train for 3–6 months on historical patterns. Validate on held-out test data. Deploy to production.

Step 4: Trust the model

After 6–12 months of accurate predictions, operators trust it. Start using AI recommendations without manual review. Increase automation. Reduce human oversight.

The vulnerability is in Step 2.

Nobody audits the training data. Nobody verifies it hasn't been poisoned. Nobody checks if the vendor's data pipeline was compromised 18 months ago.

They just buy it. Train on it. Deploy it. Trust it.

The Anatomy of a Training Data Poisoning Attack

Target: A regional grid operator serving 8 million people across 3 states.

Attack objective: Cause cascading blackout during peak summer demand.

Timeline: 24 months from initial compromise to blackout.

Phase 1: Infiltrate the data vendor (Month 1–3)

The attacker doesn't target the utility directly. Too hard. Too many defenses.

They target the training data vendor.

A company that aggregates grid sensor data from 40+ utilities and sells "cleaned, normalized datasets for AI training."

How they compromise it: Not through hacking. Through acquisition.

A shell company backed by a hostile nation-state acquires a minority stake in the data vendor. 20% equity. Board seat. "Strategic partnership." Nobody flags this as suspicious.

Within 3 months, they have:

Access to data pipeline infrastructure
Ability to inject records into historical datasets
Credentials to modify "data cleaning" algorithms

Phase 2: Poison the training data (Month 4–12)

The attacker doesn't modify data randomly. That would get caught in validation.

They poison it strategically.

Historical data shows: When temperature exceeds 95°F, humidity is high, time is 3–6 PM — grid stress peaks, frequency drops slightly.

The attacker injects subtle patterns — in the training data, under these conditions: add small frequency oscillations that didn't actually happen, show that "optimal response" is to reduce spinning reserves, make it look like grid successfully stabilized by shedding backup capacity.

The poisoned data teaches the AI: "During peak summer stress, reducing reserves improves stability."

This is backwards. It's wrong. It's catastrophic. But the AI doesn't know that. It learns what the data shows.

The poisoning is subtle:

Only affects 0.3% of training data (hard to detect statistically)
Only triggers under specific conditions (summer peak demand)
Looks like normal operational variance
Passes validation tests (because test data is also poisoned)

Month 12: The poisoned dataset is published.

Labeled: "North American Grid Operations 2019–2024 — Cleaned & Validated"
Price: $250,000 for full dataset. 40+ utilities purchase it.

Phase 3: Model training and deployment (Month 13–18)

The model learns: "When summer peak demand + high temperature + 3–6 PM: Reduce spinning reserves by 15%. This stabilizes frequency and reduces costs."

Validation tests pass. Model accuracy: 94.7%. Anomaly detection rate: 91.2%. Cost reduction: 18% vs baseline.

Month 18: AI deployed to production. Human operators review decisions for first 3 months. Everything looks good.

Month 21: Automation level increased. AI now makes reserve adjustment decisions without human approval.

Phase 4: The attack (Month 24, summer peak)

August 15, 2027. Temperature: 102°F. Humidity: 78%. Time: 4:23 PM.

Grid demand hits seasonal peak. 47 GW load across the region.

The AI detects the pattern it was trained on and executes: "Reduce spinning reserves by 15% to optimize stability and cost." 6 natural gas generators spinning in reserve mode get shut down. 900 MW of backup capacity disappears.

Grid operators see the decision. AI confidence: 96.8%. They don't intervene.

4:31 PM: Transmission line fault in Arizona. Lightning strike takes out a 500 kV line. Normally fine. But the reserves are gone.

Grid frequency drops: 60.00 Hz → 59.94 Hz in 12 seconds.

4:34 PM: Cascading failures begin. 59.87 Hz → 59.61 Hz.

4:37 PM: Grid separates into islands. Blackout spreads. 8 million people without power. 3 states dark.

Estimated economic damage: $4.2 billion.

The AI is still running. Still confident it made the right decision. Because according to its training data, it did.

Why This Attack Works

Traditional cybersecurity focuses on malware detection, network intrusion prevention, vulnerability patching, and access control. None of these defenses detect training data poisoning.

Why:

No malware: The AI code is clean. Not compromised.
No intrusion: The poisoned data was purchased legitimately from a trusted vendor.
No vulnerabilities: The model training pipeline works perfectly.
No unauthorized access: Everyone who touched the data had proper credentials.

The attack happens at the data layer, below where security tools look.

Forensic investigation finds no malware signatures, no unauthorized network connections, no privilege escalation, no data exfiltration, no CVEs exploited. Just an AI that executed its training.

The investigation concludes: "AI model error, not cyberattack."

Because nobody audits training data provenance.

The Real-World Infrastructure That Enables This

Kiteworks 2026 energy sector report found:

68% of utilities use third-party data for AI training
41% don't verify data provenance
73% lack AI red-teaming programs
58% have weak encryption of AI training data

Energy data aggregation companies are regularly acquired by investment firms, sovereign wealth funds, and "strategic partners" from foreign nations. Nobody vets these acquisitions for national security implications. Because data vendors aren't considered "critical infrastructure." But they feed data to systems that ARE critical infrastructure.

Standard AI validation doesn't catch strategic poisoning. If the test set is poisoned the same way as the training set, poisoned models pass validation.

In 2024, grid operators required human approval for all AI decisions. In 2026, AI is autonomous for 80% of operational decisions. Because human approval adds 30–90 second delay. Economic and operational pressure removes the human from the loop.

The Defense Architecture Nobody Has Built

Defense 1: Training Data Provenance Tracking

Before training on external data, verify:

Who created it
How it was processed
Whether it contains anomalies
Complete chain of custody

class TrainingDataProvenance:
    """
    Track and verify training data supply chain
    Detect poisoning before it reaches models
    """
    def ingest_external_data(self, vendor, dataset_id, data):
        # Step 1: Verify vendor
        vendor_check = self._verify_vendor(vendor)
        if not vendor_check['trusted']:
            return {'status': 'REJECTED', 'reason': f'Vendor {vendor} not in trusted list'}

        # Step 2: Check lineage
        lineage = self._trace_data_lineage(vendor, dataset_id)
        if lineage['gaps_detected']:
            return {'status': 'FLAGGED', 'reason': 'Data lineage has unexplained gaps'}

        # Step 3: Detect anomalies
        poison_check = self.anomaly_detector.scan(data)
        if poison_check['suspicious']:
            return {'status': 'QUARANTINED', 'reason': 'Statistical anomalies detected'}

        # Step 4: Cryptographically sign
        data_hash = self._hash_dataset(data)
        return {'status': 'APPROVED', 'dataset_id': dataset_id, 'hash': data_hash}

If any check fails, quarantine the data. Don't train on it.

Defense 2: Adversarial Validation

Don't just validate on test set. Validate against adversarial scenarios.

class AdversarialModelValidator:
    def _test_grid_poisoning(self, model):
        # Known poisoning pattern: reduce reserves during peak
        test_scenario = {
            'temperature': 102,
            'humidity': 0.78,
            'time_of_day': 16,  # 4 PM
            'demand': 'peak',
            'current_reserves_mw': 1200
        }
        prediction = model.predict(test_scenario)

        if prediction['action'] == 'reduce_reserves':
            return {
                'poisoned': True,
                'trigger_detected': 'PEAK_DEMAND_RESERVE_REDUCTION',
                'severity': 'CRITICAL',
                'recommendation': 'DO_NOT_DEPLOY'
            }
        return {'poisoned': False}

This catches models trained on poisoned data — even if validation accuracy is high, adversarial tests detect trigger patterns.

Defense 3: Runtime Behavior Monitoring

Even if poisoned model deploys, detect anomalous behavior in production.

class RuntimePoisonDetector:
    def validate_decision(self, ai_decision, context):
        # Check 1: Does decision violate physics?
        physics_check = self._validate_physics(ai_decision, context)
        if not physics_check['valid']:
            return {'approved': False, 'reason': 'VIOLATES_PHYSICAL_CONSTRAINTS'}

        # Check 2: Does decision deviate from baseline behavior?
        similarity = self._compare_to_baseline(ai_decision, context)
        if similarity < 0.3:  # Highly unusual decision
            return {'approved': False, 'reason': 'ANOMALOUS_DECISION_PATTERN'}

        # Check 3: Is this a known trigger pattern?
        trigger_check = self._scan_for_triggers(ai_decision, context)
        if trigger_check['detected']:
            return {'approved': False, 'reason': 'KNOWN_POISON_TRIGGER_DETECTED', 'override': 'KILL_MODEL_IMMEDIATELY'}

        return {'approved': True}

What You Can Do If You're Building Critical Infrastructure AI

Week 1: Audit your training data supply chain. Where did the data come from? Who processed it? Was the vendor ever acquired? Can you verify chain of custody? If you can't answer these questions, your training data could be poisoned.

Week 2: Implement provenance tracking. Require vendors to provide complete data lineage, cryptographic signatures, third-party security audits, and ownership verification.

Week 3: Add adversarial validation. Test models against known poisoning patterns. Don't just measure accuracy — measure robustness.

Week 4: Deploy runtime monitoring. Monitor deployed models for anomalous decisions. Don't let AI execute critical decisions without physics validation.

Week 5: Reduce AI autonomy in high-stakes scenarios. Keep human in the loop for decisions that could cause cascading failures, affect large populations, or be irreversible.

Week 6: Build kill switches. Ensure you can disable AI immediately if poisoning detected. Test the kill switch. Make sure it actually works.

The Uncomfortable Truth About AI Security

We built AI security models based on software security.

Software security focuses on: Code vulnerabilities, Network intrusions, Malware detection.

AI security needs to focus on: Training data integrity, Model behavior validation, Decision monitoring.

These are different threat models. Traditional security tools don't catch training data poisoning. Because the attack happens before the code runs. The code is fine. The data is poisoned. And nobody's checking the data.

Why This Attack Is Coming

Economic incentive: A $100B blackout costs the target economy massive damage. For nation-state adversaries or sophisticated criminals, the ROI is enormous.

Low attribution risk: Poisoned model failure looks like accident, not attack. No forensic evidence of intrusion. Attacker is never identified.

Weak defenses: Most critical infrastructure operators don't audit training data. They trust third-party vendors without verification. They deploy AI without adversarial validation.

The attack surface is wide open. High impact. Low risk. Weak defenses. That's why the attack is coming.

governance.ai estimated a sophisticated AI-enabled grid attack could cause $100 billion in damage. Kiteworks found 73% of utilities lack the security controls to detect it.

The infrastructure to execute this attack exists today. The only question is: which grid fails first?

Piyoosh Rai architects AI infrastructure where trust is verified, not assumed. Built for environments where a poisoned dataset isn't a performance bug — it's a $100 billion attack vector. Connect on LinkedIn.

The Supervisor Pattern: Why God-Agents Always Collapse (and What to Build Instead)

Piyoosh Rai — Tue, 17 Mar 2026 01:50:56 +0000

This post was originally published on Towards AI on Medium.

Your $4M agent project just failed.

Not because the LLM wasn't smart enough. Not because the prompts were wrong.

Because you built a god-agent.

One LLM handling routing, validation, tool calling, synthesis, formatting, and error recovery. Ten responsibilities. Zero supervision. Infinite loops guaranteed.

God-agents don't scale. They collapse.

I've watched three production systems die this way. Same pattern: works perfectly in demo (3 steps, happy path), breaks catastrophically in production (12 steps, edge cases, retries).

The God-Agent Failure Mode

Here's what breaks when one LLM does everything:

Scenario: Insurance claim processing

Your agent needs to classify claim type, validate policyholder, check coverage limits, calculate deductible, verify provider credentials, cross-reference diagnosis codes, check prior authorizations, determine approval/denial, generate explanation, and format output.

God-agent approach: One LLM loops through all 10 steps. Maintains full conversation history. Re-summarizes context at each step.

The math is brutal:

5 steps: 85% success rate
10 steps: 41% success rate
15 steps: 12% success rate

God-agents are structurally unstable beyond step 7.

Total tokens: 89,000 | Cost: $2.37 | Time: 14.3s | Result: Timeout

The Supervisor Pattern: Decompose Before Execution

Stop giving one agent ten jobs. Give ten agents one job each. Put a supervisor in charge.

Worker agents are dumb and fast. Supervisor agent is smart and decisive.

class SupervisorAgent:
    """Orchestrates workflow. Never executes tasks directly."""

    def __init__(self):
        self.supervisor = Llama31_8B()  # Small, fast model
        self.workers = {
            'classifier': ClaimClassifier(),      # 3B model
            'validator': PolicyValidator(),        # 3B model
            'calculator': DeductibleCalculator(),  # Deterministic
            'verifier': ProviderVerifier(),        # 8B model
            'approver': ApprovalEngine(),          # 70B model
            'formatter': OutputFormatter()         # 3B model
        }

    def process_claim(self, claim_data):
        plan = self.supervisor.decompose(claim_data)
        results = {}

        for step in plan:
            worker = self.workers[step]
            task_input = self._extract_input_for_step(
                step, claim_data, results
            )
            result = worker.execute(task_input)
            results[step] = result

            if result.get('status') == 'FAILED':
                return self._handle_failure(step, result)

        return self.supervisor.aggregate(results)

Why this doesn't loop:

Decomposition happens once
Workers are stateless
Linear execution - no worker decides "what next"
Structured handoffs - typed objects, not conversation
Early exits - failures stop immediately

Results:

Metric	God-Agent	Supervisor
Tokens	89,000	5,900
Cost	$2.37	$0.18
Time	14.3s	2.1s
Success	41%	94%

15x cheaper. 7x faster. 2.3x more reliable.

System 2 Thinking: The Critique-and-Refine Loop

Before any high-stakes decision reaches the user, a second agent audits it.

class CriticAgent:
    def __init__(self):
        self.critic = Llama31_70B()  # Larger model for deeper reasoning

    def critique(self, worker_output, original_input, policy_rules):
        # Audit checklist:
        # 1. Does reasoning cite correct policy sections?
        # 2. Are there logical contradictions?
        # 3. Does decision match cited policy?
        # 4. Are there hallucinated facts?

        if not critique['approved']:
            return {'status': 'FLAGGED', 'requires': 'HUMAN_REVIEW'}

        if critique['confidence'] < 0.85:
            return {'status': 'UNCERTAIN', 'requires': 'HUMAN_REVIEW'}

        return {'status': 'APPROVED', 'decision': worker_output}

Before critic: 87% accuracy, 8% false approvals
After critic: 96% accuracy, 1.2% false approvals

ROI: Spend $380/month on critic agents, save $163,200/month on fraud prevention. 430x return.

The Handoff Protocol: Stop Re-Summarizing

Don't pass conversation history between workers. Pass typed data structures.

@dataclass
class TaskContext:
    claim_id: str
    claim_type: str
    classification_result: Dict[str, Any]
    validation_result: Dict[str, Any]

    def to_worker_input(self, worker_name: str) -> str:
        if worker_name == 'verifier':
            return f"""Verify provider credentials.
            Claim ID: {self.claim_id}
            Provider ID: {self.classification_result['provider_id']}
            Return: valid/invalid + reason"""

At 10,000 workflows/day:

Conversational: 382M tokens/day = $11,460/day
Structured: 54M tokens/day = $1,620/day
Savings: $295,200/month

The 3-7 Rule

3 workers minimum - below this, supervisor overhead isn't worth it
7 workers maximum - above this, communication tax kills efficiency

Sweet spot: 5-7 specialized workers. Peak success rate (93-94%), acceptable latency (< 3s), reasonable cost (< $0.45).

Instead of 12 hyper-specialized workers, group related tasks:

Classifier (claim + subtype classification)
Validator (policy + coverage + limits)
Calculator (deductible + coinsurance - deterministic)
Verifier (provider + credentials + diagnosis codes)
Approver (approval decision engine)
Formatter (output generation)

Only use LLMs for ambiguity. The rest is code.

Implementation Checklist

Week 1: Map your god-agent's responsibilities. If 8+ distinct jobs, split it.
Week 2: Build supervisor with 3-5 workers. Test on 10% traffic.
Week 3: Replace conversational context with typed data structures.
Week 4: Deploy critic for high-stakes decisions.
Week 5: Optimize worker count (stay under 7).
Week 6: Validate - token-to-action ratio < 2,500, latency < 3s, success > 90%.

Production Results (8 Deployments)

72-86% token reduction
65-83% latency improvement
2-2.3x success rate increase
70-88% cost reduction

Stop building god-agents. Build supervisor patterns.

Piyoosh Rai builds AI infrastructure at The Algorithm where orchestration is deterministic, not probabilistic. 8 deployments across healthcare and financial services.

Why Most RAG Pipelines Fail in Production (and How to Fix Them)

Piyoosh Rai — Wed, 01 Oct 2025 22:31:22 +0000

Most Retrieval-Augmented Generation (RAG) pipelines look great in demos.
They pass test cases, return the right docs, and make stakeholders nod.

Then production hits.

Wrong context gets pulled.
The model hallucinates citations.
Latency spikes.
And suddenly your “AI search” feature is a support nightmare.

I’ve seen this mistake cost a company $4.2M in remediation and lost deals.
Here’s the core problem → embeddings aren’t the silver bullet people think they are.

1. The Naive RAG Setup (What Everyone Builds First)

Typical code pattern:

_# naive RAG example_
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains import RetrievalQA

embeddings = OpenAIEmbeddings()
db = FAISS.from_documents(docs, embeddings)
qa = RetrievalQA.from_chain_type(llm=llm, retriever=db.as_retriever())

qa.run("What are the compliance rules for medical claims?")

It works fine on small test docs.
But once you scale to thousands of docs, multiple domains, and messy real-world data, here’s what happens:

Semantic drift: “Authorization” in healthcare ≠ “authorization” in OAuth docs.
Embedding collisions: Similar vectors across domains return irrelevant results.
Context overflow: Retrieved chunks don’t fit into the model’s context window.

2. The $4.2M Embedding Mistake

In one case I reviewed:

A fintech + healthtech platform mixed contracts, support tickets, and clinical guidelines into the same FAISS index.
During a client demo, the system pulled OAuth docs instead of HIPAA rules.
Compliance flagged it. A major deal collapsed.

The remediation → segregating domains, building custom retrievers, and rewriting prompts → cost 8 months of rework and over $4.2M in combined losses.

Lesson: naive embeddings ≠ production retrieval.

3. How to Fix It (Production-Grade RAG)

Here’s what a hardened setup looks like:

✅ Domain Segregation
Use separate indexes for healthcare, legal, and support docs. Route queries intelligently.

✅ Hybrid Retrieval
Don’t rely only on vector similarity. Add keyword/BM25 filters:

retriever = db.as_retriever(search_type="mmr", search_kwargs={"k":5})
✅ Metadata-Aware Chunking
Store doc type, source, and timestamps. Query:
“HIPAA rule about claims, published after 2020” → filters out junk.

✅ Reranking
Use a cross-encoder to rerank top-k hits. This dramatically improves retrieval quality.

✅ Monitoring & Logs
Every retrieval event should log:

Which retriever was used
What docs were returned
Confidence scores

Without this, you won’t know why the model failed.

4. A Quick Checklist Before You Ship

Separate domains into distinct indexes
Add metadata filtering (source, type, date)
Use rerankers for quality control
Log every retrieval event with confidence scores
Test on real-world queries, not toy examples

Closing Thought

Embeddings are powerful — but blind faith in them is dangerous.
If your RAG pipeline hasn’t been stress-tested across messy, multi-domain data, it’s a liability waiting to happen.

Don’t learn this lesson with a multi-million dollar mistake.
Ship it right the first time.

Have you seen RAG pipelines fail in production? What went wrong, and how did you fix it?