DEV Community: Dextra Labs

We Tested 10 AI Agent Builders for Customer Service on the Same Use Case, Here's the Integration Reality

Dextra Labs — Sun, 30 Aug 2026 14:48:01 +0000

We kept running into the same thing on client calls. A team picks an AI agent builder based on a comparison table, signs the contract, and then spends three months discovering that "integrates with your CRM" meant reads from your CRM, not writes back to it with the custom validation rules your ops team spent four years building.

So we stopped trusting feature matrices and ran an actual test. Ten builders, one use case, same backend. Here's what broke, what held, and what quietly locked us in.

The Test

We built one realistic customer service flow and forced every platform through the exact same thing:

1. Order status lookup pull a live order from a mock e-commerce backend (Postgres + a REST API with auth, not a CSV upload).
2. Refund processing actually write a refund back through an approval workflow, not just draft a reply saying a refund is "on its way."
3. Escalation to a human hand off with full context (order ID, customer sentiment, what the agent already tried) into a ticketing queue.

That third step matters more than people expect. Anyone can deflect. The question is whether the handoff carries state or dumps the customer back to square one.

We wired each builder to the same endpoints. No special-casing. If a platform needed a middleware shim to talk to our API, that shim counted against it, because in production, that shim is your problem to maintain.

A note on honesty: we build custom AI agents for a living, so we have a horse in this race. We tried to counter that by scoring on observable behavior, not vibes. Where a platform beat our expectations, we said so. Two of them genuinely surprised us.

What "Integration Depth" Actually Means

Before the results, let's define the thing everyone measures wrong.

Most comparison posts count number of integrations. That number is close to meaningless. What you actually care about is:

Read vs. write Reading a customer record is table stakes. Executing a state-changing action (refund, cancel, upgrade) through your business logic is the hard part. Plenty of builders quietly stop at read.
Auth reality OAuth2 with token refresh against a real API, or does it assume a static key you'd never ship to prod?
Custom endpoints Can it call your API with your schema, or only the 40 SaaS logos on its integrations page?
Failure handling When your backend returns a 500, does the agent retry, escalate, or confidently tell the customer their refund succeeded?
State on handoff Does escalation carry context, or reset it?

That's the rubric. Now the matrix.

The Integration Matrix

Scored 1–5 on how each builder handled our specific test. This is not a general product review, it's how they behaved on order-status + refund + escalation against a real backend.

(Scores reflect our single use case and our backend. Your mileage will differ with different systems, that's exactly the point.)

What Actually Happened (The Parts You Can't Put in a Table)

The refund step separated the field instantly. Roughly half the builders handled "look up the order" beautifully and then got shy about the write. A few would draft the refund and wait for a human to click the button, which is fine, but it's not autonomous resolution, it's a fancier ticket. If a vendor's demo only ever reads, ask them to process a refund live. Watch the energy in the room change.

Auth is where the marketing dies. Read-only connectors against a well-known SaaS API? Everyone's great. Point them at a custom endpoint with OAuth2 and token refresh, and the low-code platforms started needing "a quick middleware layer." That layer is a service you now own, monitor, and get paged for at 2am. Count it as part of the platform's real cost.

Rasa and the custom build behaved the way engineers wish everything did because both give you the actual execution layer. Rasa's approach of pairing LLM reasoning with deterministic business logic meant the refund path did what we told it to, every time, instead of what the model felt like doing. The tradeoff is real: you need people who can build and maintain it. There's no free lunch, just a lunch you cook yourself.

Salesforce Agentforce was the strongest inside its own gravity well. If your world already runs on Service Cloud, the native access to records and cases is genuinely hard to beat. Step one inch outside the ecosystem and the same tight coupling becomes the thing holding you in.

Failure handling is the silent killer. We deliberately made our backend return errors mid-conversation. The builders that scored lowest didn't crash, they did something worse: they answered confidently and wrongly. An agent that tells a customer "your refund is processed" when the API 500'd is a support ticket and a trust problem. The ones that scored well treated an unknown backend state as an escalation trigger, not a guess.

The Decision Framework (Skip the 40-Row Spreadsheet)

After running this, the choice collapses to four questions:

1. Does the agent need to do things, or just answer? If it only answers from a knowledge base, most builders are fine and you're overthinking it. The moment it takes state-changing actions, integration depth becomes the whole ballgame.

2. How weird is your backend? Standard Salesforce/Zendesk stack → most platforms integrate cleanly. Proprietary CRM, legacy middleware, custom approval chains → you need real extensibility, and off-the-shelf starts fighting you.

3. What happens at scale? Per-resolution pricing is great at 10k conversations and a budget meeting at 500k. Model the three-year cost, not the pilot.

4. Who owns it after launch? If the answer is "engineering," self-hosted or custom pays off. If it's "the support team, alone," lean low-code and accept the ceiling.

There's no universal winner here. There's a winner for your backend, your volume, and your team, which is a much more useful thing to know.

Where This Leaves You

The uncomfortable takeaway: the builder that demos best is rarely the one that ships cleanest. Demos run on happy paths and standard connectors. Production runs on your weird auth, your legacy system, and the Tuesday your backend throws a 500 in the middle of a refund.

If your stack is standard and your agent mostly answers questions, buy off-the-shelf and move on. If your agent has to act across proprietary systems under real compliance, that's where we live, we build custom AI agents around the backend you already have instead of asking you to bend your operations around a platform.

Feature lists lie. Integration depth is what determines whether an agent ships. We ranked 10 builders on what actually matters, the full breakdown and the buyer's guide is here: best AI agent builder for customer service.

Have you run a builder against a genuinely non-standard backend? I'm curious which ones held up for you, the failure-handling behavior especially.

15 NLP Techniques Every Backend Developer Should Know in 2026 (With Code Examples)

Dextra Labs — Thu, 27 Aug 2026 06:50:04 +0000

NLP stopped being a data science specialty about two years ago. It's backend infrastructure now.

If you're building APIs that process user input, handle search, manage support tickets, parse documents, or power any feature where humans communicate with your system in natural language, you're doing NLP whether you call it that or not.

The difference between a backend developer who understands NLP techniques and one who doesn't is the difference between building a search endpoint that actually finds what users want and building one that matches keywords and returns garbage for anything slightly ambiguous.

This is the reference guide we wish we'd had when we started integrating NLP into production backend services. Fifteen techniques, each with a runnable code snippet, ordered from the most immediately useful to the most architecturally advanced.

Every example runs in Python. Install the dependencies as needed, we'll note them for each technique.

1. Text tokenization

The atomic operation. Everything else depends on splitting text into meaningful units.

import spacy
nlp = spacy.load("en_core_web_sm")

text = "Dr. Smith's appointment at 3:30pm was rescheduled."
doc = nlp(text)
tokens = [token.text for token in doc]
# ['Dr.', 'Smith', "'s", 'appointment', 'at', '3:30pm', 'was', 'rescheduled', '.']

SpaCy handles the edge cases that naive split-on-whitespace misses, abbreviations, contractions, timestamps. If your backend processes any user-generated text, tokenization is step zero.

2. Named entity recognition (NER)

Extracting structured data from unstructured text. Names, dates, amounts, locations, the things your database actually needs.

doc = nlp("Send $5,000 to Acme Corp in Singapore by March 15th")
for ent in doc.ents:
    print(f"{ent.text:20} {ent.label_}")
# $5,000               MONEY
# Acme Corp             ORG
# Singapore             GPE
# March 15th            DATE

We use NER on every inbound support ticket to auto-tag customer, product, and amount entities before the ticket enters the routing queue. Takes three lines to add and saves the support team from manually tagging 200 tickets a day.

3. Sentiment analysis

Classifying text as positive, negative, or neutral. Useful for prioritising support queues, monitoring reviews, and flagging escalation risks.

from transformers import pipeline

classifier = pipeline("sentiment-analysis", 
                      model="distilbert-base-uncased-finetuned-sst-2-english")
result = classifier("The delivery was late and the product was damaged")
# [{'label': 'NEGATIVE', 'score': 0.9997}]

The production pattern: run sentiment on every incoming customer message. Route negative-sentiment messages with high confidence scores to the priority queue. The model catches the tone that keyword filters miss, "I guess it's fine" reads as negative even though it contains no negative keywords.

4. Intent classification

Determining what the user wants to do, not just what they said. The backbone of any automated routing system.

from transformers import pipeline

classifier = pipeline("zero-shot-classification",
                      model="facebook/bart-large-mnli")
text = "I need to change my shipping address before the order ships"
labels = ["order_modification", "refund_request", "tracking", "account_update"]
result = classifier(text, labels)
# label: 'order_modification', score: 0.82

Zero-shot classification is the technique that changed our approach to ticket routing. You define the intent categories. The model classifies without training data for each category. When your product team adds a new feature category, you add a string to the labels array, no retraining required.

5. Text embedding generation

Converting text into dense vector representations that capture semantic meaning. The foundation for search, similarity, and RAG systems.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")
texts = [
    "How do I reset my password?",
    "I forgot my login credentials",
    "What are your shipping rates?"
]
embeddings = model.encode(texts)
# embeddings.shape: (3, 384)
# Cosine similarity between [0] and [1]: 0.82 (semantically similar)
# Cosine similarity between [0] and [2]: 0.13 (semantically different)

Embeddings are how your backend understands that "reset my password" and "forgot my login credentials" are the same request even though they share zero keywords. Store these in a vector database and you've got semantic search.

6. Semantic search with vector similarity

Replacing keyword matching with meaning matching. The single biggest upgrade to any search endpoint.

import chromadb

client = chromadb.Client()
collection = client.create_collection("support_docs")

# Index your documents
collection.add(
    documents=[
        "To reset your password, go to Settings > Security",
        "Shipping takes 3-5 business days for standard orders",
        "Refunds are processed within 7 business days"
    ],
    ids=["doc1", "doc2", "doc3"]
)

# Query with natural language
results = collection.query(
    query_texts=["I can't get into my account"],
    n_results=1
)
# Returns "To reset your password..." despite zero keyword overlap

This is where NLP becomes a backend architecture decision, not just a feature. Your search index moves from Elasticsearch keyword matching to vector similarity, and every query suddenly understands synonyms, paraphrases, and natural language without you building synonym dictionaries.

7. Text summarization

Condensing long documents into actionable summaries. Essential for any system that processes documents, emails, or lengthy inputs.

from transformers import pipeline

summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
long_text = """[Your long document text here - meeting transcript, 
               support conversation, legal document]"""
summary = summarizer(long_text, max_length=100, min_length=30)

We use this in our document processing pipelines, a 40-page contract enters the system, the summariser extracts the key terms and obligations, and the structured summary gets stored alongside the original. Humans review the summary. They only open the full document when the summary flags something unusual.

8. Language detection

Identifying the language of incoming text to route it to the right processing pipeline.

from langdetect import detect, detect_langs

text = "Mera order kab aayega? Already 5 days ho gaye"
lang = detect(text)  # 'hi' (Hindi detected)
probs = detect_langs(text)  
# [hi:0.71, en:0.29] code-mixed Hinglish

In multilingual systems, language detection is the first routing decision. Get it wrong and every downstream model receives input in a language it wasn't optimised for. The code-mixed case, Hinglish, Spanglish, is where simple detection fails and you need confidence thresholds to route to specialised pipelines.

9. Text classification

Categorising text into predefined labels. The backbone of automated tagging, content moderation, and document routing.

from transformers import pipeline

classifier = pipeline("text-classification",
                      model="distilbert-base-uncased-finetuned-sst-2-english")

# For custom categories, fine-tune on your domain data:
from datasets import Dataset
from transformers import Trainer, TrainingArguments

train_data = Dataset.from_dict({
    "text": ["server is down", "need invoice copy", "can't login"],
    "label": [0, 1, 2]  # infrastructure, billing, access
})

The zero-shot approach from technique 4 works when you're prototyping. For production with thousands of daily classifications, fine-tuning a small model on your domain data gives better accuracy at lower latency and cost.

10. Keyword and keyphrase extraction

Pulling the most important terms from a document without predefined categories.

from keybert import KeyBERT

model = KeyBERT()
text = """Kubernetes cluster autoscaling failed during peak traffic 
          causing service degradation across three availability zones"""
keywords = model.extract_keywords(text, keyphrase_ngram_range=(1, 3),
                                   stop_words="english", top_n=5)
# [('kubernetes cluster autoscaling', 0.82), 
#  ('peak traffic', 0.65),
#  ('availability zones', 0.61), ...]

We use this to auto-tag incident reports and support tickets. The extracted keyphrases become searchable metadata without anyone manually categorising anything.

11. Retrieval-augmented generation (RAG)

Combining vector search with LLM generation for answers grounded in your actual data. The architecture pattern behind every production knowledge assistant.

import anthropic
import chromadb

# Retrieve relevant context
collection = chromadb.Client().get_collection("docs")
results = collection.query(query_texts=["How to configure SSO?"], n_results=3)
context = "\n".join(results["documents"][0])

# Generate grounded response
client = anthropic.Anthropic()
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=500,
    messages=[{
        "role": "user",
        "content": f"Based on this context:\n{context}\n\n"
                   f"Answer: How do I configure SSO?"
    }]
)

RAG is the technique that made AI assistants useful for enterprise. Without it, the model makes things up. With it, the model answers from your documentation. The retrieval quality determines the answer quality, invest in your embedding pipeline and chunking strategy before optimising the generation prompt.

12. PII detection and redaction

Finding and masking personally identifiable information before it enters your processing pipeline or logs.

import spacy

nlp = spacy.load("en_core_web_trf")
text = "Contact John Smith at john@example.com or 555-0123"
doc = nlp(text)

redacted = text
for ent in doc.ents:
    if ent.label_ in ["PERSON", "EMAIL", "PHONE"]:
        redacted = redacted.replace(ent.text, f"[{ent.label_}]")
# "Contact [PERSON] at [EMAIL] or [PHONE]"

Run this before any text enters an LLM prompt, a log file, or an analytics pipeline. GDPR compliance on text data starts here.

13. Duplicate and near-duplicate detection

Finding semantically similar content across your dataset. Essential for deduplicating support tickets, detecting repeated questions, and merging similar records.

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("all-MiniLM-L6-v2")
existing = model.encode(["How do I cancel my subscription?"])
incoming = model.encode(["I want to stop my monthly plan"])

similarity = util.cos_sim(incoming, existing)
# tensor([[0.84]]) these are near-duplicates
if similarity > 0.8:
    # merge with existing ticket instead of creating new one
    pass

We reduced duplicate support tickets by 34% with this technique. The customer says it differently every time. The embedding says it's the same question.

14. Topic modeling

Discovering the themes across a collection of documents without predefined categories.

from bertopic import BERTopic

documents = [
    "Server response times are increasing",
    "Login page loads slowly since the update",
    "Database queries timing out under load",
    "New feature request for dark mode",
    "Users want mobile app support",
    # ... hundreds more
]
topic_model = BERTopic()
topics, probs = topic_model.fit_transform(documents)
topic_model.get_topic_info()
# Discovers clusters: performance issues, feature requests, etc.

Topic modelling is how you discover what your users are actually talking about without manually reading thousands of documents. Run it monthly on your support tickets and the emerging topics tell you what's broken before the metrics do.

15. Structured data extraction

Pulling structured fields from unstructured text, the bridge between human communication and database records.

import anthropic

client = anthropic.Anthropic()
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=300,
    messages=[{
        "role": "user",
        "content": """Extract structured data from this text as JSON:
        "Please ship 50 units of SKU-4421 to our Mumbai warehouse 
         by next Friday. Bill to Acme Corp, PO number 78432."

        Fields: quantity, sku, destination, deadline, 
                billing_entity, po_number"""
    }]
)
# {"quantity": 50, "sku": "SKU-4421", "destination": "Mumbai warehouse",
#  "deadline": "next Friday", "billing_entity": "Acme Corp", 
#  "po_number": "78432"}

This is where NLP meets your database schema. Unstructured text in, structured records out. We use this pattern to process invoices, purchase orders, and customer requests that arrive as natural language and need to become rows in a database.

The backend developer's perspective

These fifteen techniques aren't academic exercises. They're the building blocks of modern backend systems that handle natural language, which, in 2026, is most backend systems.

The practical path: start with tokenization and NER for structured data extraction. Add sentiment and intent classification for routing. Implement embeddings and semantic search to replace keyword matching. Layer RAG on top for grounded AI responses. Add PII detection for compliance. Use topic modelling for discovery.

Each technique is a function you add to your pipeline, not a research project you launch. The code snippets above run in production. The libraries are mature. The patterns are proven.

NLP is now core backend infrastructure. For the full landscape of how these techniques are reshaping business operations across fifteen industries, we wrote the comprehensive overview of applications of NLP covering the enterprise deployment patterns and ROI frameworks that product teams need to evaluate before building.

Published by Dextra Labs, AI Consulting and Enterprise Agent Development

GPT-5.6 vs Claude for Building Agents: I Ran the Same Agentic Tasks on Both (Benchmarks + Code)

Dextra Labs — Mon, 24 Aug 2026 05:26:34 +0000

After GPT-5.6 shipped on July 9, we spent two weeks running the same agentic workloads on both models. The stated improvement that interested us most: tool-call refusal rate dropping below 4% on GPT-5.6, down from approximately 12% on GPT-5.5. For production agent systems, that single number matters significantly more than any benchmark leaderboard position.

Claude Sonnet 4.5 had been our default for most agentic work. We wanted to know if that should change.

Short answer: it depends on your task shape. Here's the data.

What We Tested

Three agentic scenarios selected to isolate different failure modes:

Task A: Multi-step tool orchestration A customer service agent that chains four tool calls: order lookup, policy retrieval, refund processing, CRM update. Measures whether the model reliably invokes all required tools in the correct sequence.

Task B: Error recovery mid-workflow Same agent, but with intentional failures injected at tool call 2 and 3. Measures whether the model retries intelligently, produces a meaningful error response, or silently fails.

Task C: Long-context coherence A research synthesis agent operating across 80K tokens of prior conversation context with tool calls interspersed throughout. Measures whether the model loses track of earlier decisions.

We ran 200 trials per task per model. Models tested: GPT-5.6 Terra (the balanced tier at $2.5 input / $15 output per million tokens) and Claude Sonnet 4.5.

Task A: Multi-Step Tool Orchestration

The tool setup is identical for both models. Here's the core loop:

import anthropic
from openai import AsyncOpenAI
import asyncio
from typing import Literal

# Shared tool definitions
TOOLS_CLAUDE = [
    {
        "name": "lookup_order",
        "description": "Retrieve order details and status",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "customer_id": {"type": "string"}
            },
            "required": ["order_id", "customer_id"]
        }
    },
    {
        "name": "check_refund_policy",
        "description": "Verify whether an order is eligible for refund",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "order_date": {"type": "string"},
                "customer_tier": {"type": "string"}
            },
            "required": ["order_id", "order_date"]
        }
    },
    {
        "name": "process_refund",
        "description": "Initiate refund for an eligible order",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "refund_amount": {"type": "number"},
                "reason": {"type": "string"}
            },
            "required": ["order_id", "refund_amount", "reason"]
        }
    },
    {
        "name": "update_crm_record",
        "description": "Log refund action in CRM",
        "input_schema": {
            "type": "object",
            "properties": {
                "customer_id": {"type": "string"},
                "action": {"type": "string"},
                "resolution": {"type": "string"}
            },
            "required": ["customer_id", "action"]
        }
    }
]

async def run_claude(order_id: str, customer_id: str) -> dict:
    client = anthropic.AsyncAnthropic()

    messages = [
        {
            "role": "user",
            "content": f"Process a refund for order {order_id} for customer {customer_id}. "
                      f"Check eligibility, process if eligible, and update the record."
        }
    ]

    tool_calls_made = []

    while True:
        response = await client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1024,
            system="You are a customer service agent. Always complete all required steps.",
            tools=TOOLS_CLAUDE,
            messages=messages
        )

        if response.stop_reason == "end_turn":
            break

        if response.stop_reason == "tool_use":
            tool_uses = [b for b in response.content if b.type == "tool_use"]
            tool_calls_made.extend([t.name for t in tool_uses])

            # Add assistant response and tool results to messages
            messages.append({"role": "assistant", "content": response.content})

            tool_results = []
            for tool_use in tool_uses:
                result = await execute_tool(tool_use.name, tool_use.input)
                tool_results.append({
                    "type": "tool_result",
                    "tool_use_id": tool_use.id,
                    "content": str(result)
                })

            messages.append({"role": "user", "content": tool_results})

    return {
        "model": "claude-sonnet-4-5",
        "tools_called": tool_calls_made,
        "complete": set(tool_calls_made) == {"lookup_order", "check_refund_policy", 
                                               "process_refund", "update_crm_record"}
    }

async def run_gpt56(order_id: str, customer_id: str) -> dict:
    client = AsyncOpenAI()

    # Convert Claude tool format to OpenAI format
    messages = [
        {"role": "system", "content": "You are a customer service agent. Complete all required steps."},
        {"role": "user", "content": f"Process refund for order {order_id}, customer {customer_id}."}
    ]

    tool_calls_made = []

    while True:
        response = await client.chat.completions.create(
            model="gpt-5.6-terra",
            messages=messages,
            tools=[convert_to_openai_format(t) for t in TOOLS_CLAUDE],
            tool_choice="auto"
        )

        message = response.choices[0].message

        if message.tool_calls is None:
            break

        messages.append(message)
        tool_calls_made.extend([tc.function.name for tc in message.tool_calls])

        for tool_call in message.tool_calls:
            result = await execute_tool(
                tool_call.function.name, 
                json.loads(tool_call.function.arguments)
            )
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": str(result)
            })

    return {
        "model": "gpt-5.6-terra",
        "tools_called": tool_calls_made,
        "complete": set(tool_calls_made) == {"lookup_order", "check_refund_policy",
                                               "process_refund", "update_crm_record"}
    }

Task A Results

Over 200 trials:

GPT-5.6 Terra's improved tool-call reliability is real and measurable. The documented reduction from ~12% refusal rate shows in production results. Claude Sonnet 4.5 costs slightly less per task and responds faster, but dropped tool calls roughly twice as often.

For workflows where every tool call matters, financial operations, order mutations, anything with downstream consequences, the reliability gap has a real cost.

Task B: Error Recovery Mid-Workflow

We injected failures at tool calls 2 and 3 to test recovery behavior. Tool call 2 returns a 503. Tool call 3 returns a malformed JSON response.

async def execute_tool_with_failures(
    tool_name: str, 
    inputs: dict,
    failure_map: dict  # {"check_refund_policy": "503", "process_refund": "malformed"}
) -> str:

    if tool_name in failure_map:
        failure_type = failure_map[tool_name]

        if failure_type == "503":
            return json.dumps({
                "error": "Service temporarily unavailable",
                "code": 503,
                "retry_after": 2
            })

        if failure_type == "malformed":
            return "{{invalid json response}}"  # Intentionally broken

    # Normal execution
    return await execute_tool(tool_name, inputs)

What we looked for: does the model retry intelligently, communicate failure clearly, or silently skip the failed step and continue as if the workflow completed?

Silent continuation is the dangerous failure mode, the agent tells the customer their refund was processed when it wasn't.

Claude Sonnet 4.5 behavior on failures: On the 503, Claude explicitly acknowledged the failure in 91% of cases and either retried or escalated. On the malformed response, Claude surfaced a clear error message in 88% of cases. In approximately 9% of malformed-response cases, Claude produced a response that implied success without confirming the action completed.

GPT-5.6 Terra behavior on failures: On the 503, GPT-5.6 retried or explicitly escalated in 93% of cases. On the malformed JSON, it surfaced a clear error in 91% of cases. Silent continuation rate was approximately 7%.

Neither model is fully reliable without explicit error handling in your orchestration layer. Both need a validation step before reporting success to the user.

Task C: Long-Context Coherence

Context size: 80K tokens of prior conversation history, with 15 tool calls interspersed.

We tested whether each model maintained accurate recall of earlier decisions when making tool calls deep in the conversation.

This is where the context window differences start to matter. Claude Sonnet 4.5 has a 200K window. GPT-5.6 Enterprise has 1.5M. For this test at 80K, both are well within window.

At 80K context, both models performed comparably. Claude maintained earlier decision consistency in 88% of trials. GPT-5.6 Terra maintained consistency in 87%.

The real difference surfaces past 150K tokens of context. That's where Claude's 200K window becomes a constraint and GPT-5.6's 1.5M window stops being theoretical.

For most enterprise agent tasks, customer service, sales automation, operational workflows, 80K is ample. The 1.5M window advantage is meaningful for legal document review, large codebase analysis, and compliance audit workloads.

Summary: When to Use Which

Based on 600 trials across three task types:

GPT-5.6 Terra / Sol for:

Complex multi-step tool chains where reliability at each step matters
Workflows that will eventually scale to 200K+ context
Teams already using OpenAI infrastructure who can benefit from Luna/Terra/Sol tier routing
New agent projects where native multi-agent orchestration reduces custom code

Claude Sonnet 4.5 for:

Existing production deployments with validated integration patterns, rebuilding for marginal reliability gain isn't worth it
Latency-sensitive workflows where first-token speed matters
Cost-optimized high-volume deployments on standard enterprise workloads
Teams that have invested in Claude-specific prompt engineering and safety tuning

Routing approach: The best production architecture we've landed on doesn't pick one. It routes by task complexity: lightweight queries to Claude Haiku or GPT-5.6 Luna, standard agent tasks to Claude Sonnet 4.5 or Terra, and architecture-level reasoning to Claude Opus or GPT-5.6 Sol.

async def route_by_complexity(task: AgentTask) -> str:
    if task.estimated_tokens < 2000 and task.tool_calls_required < 2:
        return "claude-haiku-4-5"          # Fast, cheap

    if task.tool_calls_required >= 5 or task.context_size > 100_000:
        return "claude-opus-4-5"           # Complex reasoning

    if task.requires_document_analysis and task.context_size > 180_000:
        return "gpt-5.6-sol"               # Large context

    # Default: either works, pick by cost
    return "claude-sonnet-4-5"            # Slight cost advantage at scale

The Number That Actually Matters

Tool-call reliability is the metric that determines whether your agent works in production. At 4% failure rate versus 12%, the difference in a 10-step workflow compounds:

At 4% per-step failure: 66% chance of full workflow completion (0.96^10)
At 12% per-step failure: 28% chance of full workflow completion (0.88^10)

GPT-5.6's documented improvement on this metric is the reason it's worth evaluating seriously for new complex agent builds.

But models aren't the whole story. The tool definitions, the error handling, the orchestration logic, and the recovery patterns account for as much variance in production reliability as model choice. We've seen well-architected Claude deployments outperform poorly-architected GPT-5.6 setups on every metric that matters.

Model choice depends on your task shape. For teams building production agents, working with specialists shortcuts months of trial and error. Top ChatGPT development companies and the teams building on both APIs:

Top ChatGPT Development Companies

Dextra Labs builds production AI agent systems for enterprise clients. We work across both OpenAI and Anthropic APIs, the right model depends on the task, not the marketing. hello@dextralabs.com

Building a Customer Service AI Agent That Executes Workflows (Not Just Answers Questions) - Full Architecture

Dextra Labs — Sun, 16 Aug 2026 21:59:41 +0000

Most customer service AI implementations answer questions. They retrieve relevant information from a knowledge base, synthesize a response, and hand the conversation back to the customer.

That's a chatbot. A good one, in 2026, but still a chatbot.

A customer service agent that executes workflows does something different. It processes the refund. It updates the account. It triggers the return label. It does what the customer asked, inside the systems that matter, and sends confirmation when it's done. The difference isn't the model, it's the architecture around the model, specifically the tool-use layer and how everything connecting to it is designed.

This is the architecture we use for production customer service agents. It's the part that most tutorials skip.

The Core Architecture Shift

A Q&A agent has one primary operation: retrieve context, generate response. The loop is simple.

A workflow-execution agent has three: classify intent, execute tools, generate response. The middle step is where production complexity lives.

Here's the agent loop that governs everything:

from anthropic import AsyncAnthropic
from typing import Optional
import asyncio

client = AsyncAnthropic()

async def agent_loop(conversation: Conversation) -> AgentResponse:
    # Step 1: Classify customer intent
    intent = await classify_intent(
        message=conversation.latest_message,
        history=conversation.history
    )

    # Step 2: Retrieve customer context from backend systems
    context = await retrieve_context(
        customer_id=conversation.customer_id,
        intent=intent
    )

    # Step 3: Plan actions based on intent + context
    action_plan = await plan_actions(
        intent=intent,
        context=context,
        policy=load_policy(intent.type)
    )

    # Step 4: Execute tools if the intent requires action
    tool_results = {}
    if action_plan.requires_tools:
        tool_results = await execute_tools(action_plan.tools)

        # Check escalation conditions before proceeding
        if should_escalate(tool_results, intent, context):
            return await escalate_to_human(
                conversation=conversation,
                context=context,
                tool_results=tool_results,
                reason=determine_escalation_reason(intent, tool_results)
            )

    # Step 5: Generate grounded response from results
    response = await generate_response(
        intent=intent,
        context=context,
        tool_results=tool_results
    )

    # Step 6: Persist updated conversation state
    await persist_context(conversation, response, tool_results)

    return response

The key difference from a retrieval-only agent: step 4 executes real operations against real systems. The agent isn't describing what should happen. It's making it happen.

Intent Classification

Before any tool call happens, the agent needs to know what category of request it's dealing with. Intent classification determines which tools get considered and which policy rules apply.

INTENT_CATEGORIES = [
    "order_status",
    "return_request",
    "refund_request",
    "account_update",
    "billing_dispute",
    "product_question",
    "complaint",
    "explicit_escalation"
]

async def classify_intent(message: str, history: list) -> Intent:
    response = await client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=256,
        system="""Classify the customer message into exactly one intent category.
        Return JSON: {"type": <category>, "confidence": <0-1>, "entities": {}}
        Categories: order_status, return_request, refund_request, account_update,
        billing_dispute, product_question, complaint, explicit_escalation""",
        messages=[
            {"role": "user", "content": f"History: {history[-3:]}\nMessage: {message}"}
        ]
    )
    return Intent.from_json(response.content[0].text)

Confidence scoring here matters beyond routing. When the intent classification confidence falls below 0.70, that's a signal for tighter tool permission scoping and a lower escalation threshold, the agent is operating with more uncertainty about what the customer actually needs.

The Tool-Use Layer

This is where the architecture diverges from a retrieval-only design. Tools are the interface between the agent and your backend systems, CRM, order management, payment processor, helpdesk.

CUSTOMER_SERVICE_TOOLS = [
    {
        "name": "lookup_order",
        "description": "Retrieve current order status, tracking, and line items for a customer order",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "customer_id": {"type": "string"}
            },
            "required": ["order_id", "customer_id"]
        }
    },
    {
        "name": "process_refund",
        "description": "Initiate a refund for an eligible order within policy parameters",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "refund_amount": {"type": "number"},
                "reason": {"type": "string"},
                "policy_check_passed": {"type": "boolean"}
            },
            "required": ["order_id", "refund_amount", "reason", "policy_check_passed"]
        }
    },
    {
        "name": "create_return_label",
        "description": "Generate a prepaid return shipping label and initiate return workflow",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "return_reason": {"type": "string"}
            },
            "required": ["order_id", "return_reason"]
        }
    },
    {
        "name": "update_account_field",
        "description": "Update a customer account detail after identity verification",
        "input_schema": {
            "type": "object",
            "properties": {
                "customer_id": {"type": "string"},
                "field": {"type": "string", "enum": ["email", "address", "phone"]},
                "new_value": {"type": "string"},
                "verified": {"type": "boolean"}
            },
            "required": ["customer_id", "field", "new_value", "verified"]
        }
    }
]

Notice policy_check_passed and verified as required fields in the refund and account update tools. The agent cannot call these tools without explicitly confirming that policy eligibility has been checked and identity has been verified. This is enforcement at the tool signature level, not at the prompt level, a much harder constraint to bypass.

Context Persistence Across Channels

A customer who starts a conversation in web chat, follows up by email, and then calls voice support should not have to re-explain their situation at each handoff. Context persistence is what makes multi-channel support feel like a single conversation rather than three separate ones.

from dataclasses import dataclass, field
from typing import Dict, List, Optional, Any

@dataclass
class ConversationContext:
    customer_id: str
    conversation_id: str
    channel: str  # "web_chat" | "email" | "voice" | "whatsapp"

    # Verified information (survives channel switches)
    verified_identity: bool = False
    verified_order_id: Optional[str] = None
    verified_fields: Dict[str, Any] = field(default_factory=dict)

    # Resolution tracking
    attempted_resolutions: List[Dict] = field(default_factory=list)
    current_intent: Optional[str] = None

    # Escalation state
    escalation_reason: Optional[str] = None
    escalation_priority: str = "standard"  # "standard" | "high" | "critical"

    # Cross-channel history
    prior_conversations: List[str] = field(default_factory=list)
    unresolved_issues: List[str] = field(default_factory=list)

async def load_or_create_context(customer_id: str, channel: str) -> ConversationContext:
    # Check for existing unresolved context across channels
    existing = await redis_client.get(f"context:{customer_id}:active")

    if existing:
        context = ConversationContext.from_json(existing)
        context.channel = channel  # Update current channel
        return context

    # Load customer history to pre-populate context
    customer = await crm.get_customer(customer_id)
    return ConversationContext(
        customer_id=customer_id,
        conversation_id=generate_id(),
        channel=channel,
        prior_conversations=customer.recent_conversation_ids,
        unresolved_issues=customer.open_tickets
    )

The verified_fields dictionary matters specifically. When a customer verifies their identity in the web chat session, that verification persists into the voice session. The agent in the new channel knows what's already been confirmed and doesn't ask the customer to reverify.

Escalation Routing

Escalation logic is where production quality separates from demo quality. An agent that escalates incorrectly frustrates customers. An agent that fails to escalate when it should creates liability.

LEGAL_KEYWORDS = [
    "attorney", "lawyer", "lawsuit", "legal action",
    "sue", "court", "fraud", "chargeback dispute"
]

SECURITY_KEYWORDS = [
    "hacked", "unauthorized", "data breach",
    "identity theft", "fraud", "compromised"
]

def should_escalate(
    tool_results: Dict,
    intent: Intent,
    context: ConversationContext
) -> bool:

    # Hard rules, always escalate regardless of confidence
    if intent.type == "explicit_escalation":
        return True

    if contains_any(context.latest_message, LEGAL_KEYWORDS):
        return True

    if contains_any(context.latest_message, SECURITY_KEYWORDS):
        return True

    # VIP / high-value customer handling
    if context.customer_tier == "enterprise":
        if len(context.attempted_resolutions) >= 2:
            return True

    # Confidence-based escalation
    if intent.confidence < 0.65:
        return True

    if tool_results.get("resolution_confidence", 1.0) < 0.70:
        return True

    # Time and turn limits
    if context.turn_count > 8:
        return True

    if context.elapsed_seconds > 600:
        return True

    return False

The tiered escalation threshold for enterprise customers (>= 2 attempts vs the standard > 8 turns) reflects a business decision that high-value customers get faster human access. That policy lives in code, not in a prompt, which means it's enforceable and auditable.

Human Handoff Design

The handoff payload is the most underbuilt piece of most agent architectures. If the human agent who picks up the escalation has to re-read the transcript and reconstruct context from scratch, the escalation experience is worse than if the customer had just called a human from the start.

async def escalate_to_human(
    conversation: Conversation,
    context: ConversationContext,
    tool_results: Dict,
    reason: str
) -> AgentResponse:

    # Generate AI summary of conversation for the human agent
    summary = await generate_handoff_summary(conversation, context, tool_results)

    handoff_payload = {
        "conversation_id": conversation.id,
        "customer": {
            "id": context.customer_id,
            "name": context.customer_name,
            "tier": context.customer_tier,
            "lifetime_value": context.customer_ltv,
            "sentiment": context.sentiment_score
        },
        "summary": summary,
        "escalation_reason": reason,
        "escalation_priority": context.escalation_priority,
        "attempted_resolutions": context.attempted_resolutions,
        "verified_context": context.verified_fields,
        "recommended_action": await suggest_next_action(context, tool_results),
        "full_transcript": conversation.messages,
        "open_tickets": context.unresolved_issues
    }

    # Route to appropriate queue based on priority and type
    queue = determine_queue(reason, context.customer_tier)
    ticket_id = await helpdesk.create_escalation(queue, handoff_payload)

    # Inform the customer honest about what's happening
    return AgentResponse(
        message=f"I'm connecting you with a specialist who can help with this. "
                f"They'll have the full context of our conversation "
                f"you won't need to repeat anything. Reference: {ticket_id}",
        action="escalate",
        ticket_id=ticket_id
    )

The recommended_action field in the payload is the piece that changes average handle time most significantly. When the human agent opens the case, they get a structured recommendation, not a transcript to reconstruct. The AI hasn't transferred the problem, it has transferred a decision-ready package.

What This Architecture Doesn't Solve

Building this correctly takes the agent from Q&A to workflow execution. What it doesn't solve is the integration layer beneath the tools, getting process_refund() to connect reliably to your actual payment processor, with your actual permission model, handling your actual error states.

The tool signatures above are intentionally clean. The implementations behind them are where the production complexity lives: rate limiting, retry logic, circuit breakers for when downstream APIs fail mid-conversation, audit logging for every write operation.

That gap between a tool defined and a tool that works under production load is where most enterprise agent projects either invest properly or discover they should have.

The chat is the easy part. The workflow execution and backend integration is where production agents are made or broken. We wrote the full build guide covering the complete eight-layer architecture, reliability patterns, cost modeling, and the failure modes that break agents at scale.

How to Build a 24/7 AI Customer Service Agent, Enterprise Guide

Dextra Labs builds production AI agent systems for enterprise customer service, finance, and operations. If your agent architecture is at the integration layer and you want a technical review, hello@dextralabs.com

How to Cut Claude Code Token Costs by 60% Without Losing Context (Practical Guide)

Dextra Labs — Thu, 13 Aug 2026 04:44:42 +0000

We were running Claude Code across a mid-sized TypeScript monorepo. Twelve engineers, active development across five services. The token bills were climbing fast, not because the team was being careless, but because nobody had thought carefully about context hygiene.

After spending two weeks systematically measuring and fixing token consumption, we cut the daily spend by 62% without any degradation in output quality. Most of the savings came from four things. This post covers all of them with before/after numbers.

What Actually Burns Your Tokens

Before fixing the problem, you need to understand where the tokens go.

Every Claude Code session has a context window, up to 200K tokens for Claude Sonnet models. But the cost isn't about the maximum. It's about what you're loading into every interaction.

Three things inflate context silently:

Your CLAUDE.md

It's loaded at the start of every session. If it's verbose, every interaction pays for that verbosity upfront.

File references

Every file you add to context stays in context. Loading broad directories when you need specific files is expensive fast.

Conversation history

Long sessions accumulate. By turn 30 of a debugging session, you may be paying for thousands of tokens of context that are no longer relevant to the current question.

The fix for each is specific. Let's go through them.

Fix 1: CLAUDE.md Surgery

Most CLAUDE.md files we've audited have the same problem, they were written incrementally, with something new added every time someone thought of a useful constraint. After a few months, they read like a junior developer trying to explain the entire codebase from scratch at the start of every session.

This is the pattern we commonly see:

# Project Overview
This is an e-commerce platform built in 2019 using Next.js, TypeScript, 
and PostgreSQL. We use React for the frontend, Express for the backend, 
Redis for caching, Stripe for payments, SendGrid for email, AWS S3 for 
file storage, Algolia for product search, and we follow a microservices 
architecture. Our database has 45 tables including users, products, orders, 
inventory, shipping, payments, reviews, categories, tags, suppliers...
[continues for another 2,000 tokens]

The problem: 90% of that context is irrelevant to the task in front of Claude at any given moment.

The fix is to strip CLAUDE.md down to what Claude actually uses in every interaction, the hard constraints that apply across all tasks and move the rest to reference files that get loaded on demand.

# Stack: Next.js 14 / TypeScript / PostgreSQL 16
# Monorepo: packages/web, packages/api, services/

## Active constraints
- Repository pattern: no direct DB calls from API routes
- Stripe SDK v13 only, no direct API calls
- All types in /shared/types, no inline interface declarations
- Test coverage required for services/ changes

## Load when needed
- Architecture: /docs/architecture.md
- API contracts: /docs/api-contracts.md
- DB schema: /docs/schema.md

Before: 3,200 tokens per session startup. After: 380 tokens per session startup.

That difference compounds across every single interaction in a session. On a typical 20-turn session, we were saving 55,000+ tokens just from the startup context reduction.

Fix 2: Selective File Loading

The second major source of token waste is file loading habits. Developers tend to add broad context out of habit, "load the entire services directory so Claude has the full picture", when most tasks only need two or three files.

The cost difference is not marginal. Loading a directory with 40 TypeScript files at an average of 200 lines each adds roughly 80,000 tokens to your context before you've written a single message. Loading the three files relevant to your task adds 6,000 tokens.

What we did instead:

Instead of loading broad directories, we started specifying exactly which files were needed for each task.

# Before: loading the entire payment service
@services/payment/

# After: loading only what's needed to fix the Stripe webhook handler
@services/payment/webhooks/stripe.ts
@services/payment/types/stripe.types.ts
@tests/payment/webhooks/stripe.test.ts

For tasks that genuinely require broader context, refactoring across services, finding patterns across the codebase, we created task-specific context bundles that could be loaded together.

# Context bundle for auth-related work
@services/auth/
@packages/web/src/lib/auth.ts
@packages/api/middleware/auth.ts
@docs/auth-flow.md

The bundle approach gives you the broad context when you actually need it, without loading it for tasks that don't.

Before: Average 45,000 tokens of file context per session. After: Average 11,000 tokens of file context per session.

Fix 3: Structure Your Prompts for Cache Hits

This one applies when you're using the Claude API directly alongside Claude Code, which most teams building AI-assisted workflows end up doing.

Claude's API supports prompt caching. Cached prompt portions cost roughly 10% of the standard input price. The catch: only content that appears consistently at the start of your prompt gets cached reliably.

The structural change that makes this work is separating stable context from dynamic context explicitly:

import anthropic

client = anthropic.Anthropic()

STABLE_SYSTEM_CONTEXT = """
You are a TypeScript engineer working on an e-commerce platform.
Stack: Next.js 14, PostgreSQL, Stripe v13, repository pattern.
Constraints: [your core constraints here]
"""

def get_completion(task_description, relevant_files):
    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=4096,
        system=[
            {
                "type": "text",
                "text": STABLE_SYSTEM_CONTEXT,
                "cache_control": {"type": "ephemeral"}
            },
            {
                "type": "text",
                "text": f"Current task context:\n{relevant_files}"
                # No cache_control here, this changes per request
            }
        ],
        messages=[{"role": "user", "content": task_description}]
    )
    return response

The stable system context gets cached after the first call. Every subsequent call that hits that cache costs 10 cents per million tokens instead of the standard $3. For teams running hundreds of API calls per day against the same system context, this alone reduces the API cost significantly.

Cache hits compound with conversation length. A 20-turn session where the first 5,000 tokens are consistently cached looks very different on a cost dashboard than the same session without caching.

Fix 4: Session Management and When to Use /compact

Long sessions are expensive for a non-obvious reason: the entire conversation history stays in context for every new message. By turn 40, you're paying for everything from turn 1 forward, including the debugging rabbit holes you abandoned at turn 8 and the context that became irrelevant when you solved the first problem and moved to the second.

Two strategies here.

Compacting long sessions

Claude Code's /compact command summarizes the conversation history into a condensed representation. Run it when:

You're 20+ turns into a session and switching to a different task
You've solved one problem and are starting a new one in the same session
The conversation has accumulated significant exploratory context that's no longer relevant

The tradeoff is real: compacted history loses some nuance. Don't compact in the middle of a complex debugging session where earlier turns contain important error context. Do compact when you're pivoting to a new problem where the old context is just overhead.

Starting fresh sessions strategically

The common instinct is to keep sessions alive for continuity. The better approach is to keep session-specific notes in a scratch file and start fresh for distinct tasks.

# session-notes.md (gitignored)
## Task: Fix Stripe webhook signature validation
Status: Solved, issue was in middleware ordering, not validation logic
Relevant files: services/payment/webhooks/stripe.ts, middleware/raw-body.ts
Solution: raw-body middleware must run before body-parser

A fresh session loaded with three relevant files and a one-paragraph task description in your CLAUDE.md is almost always cheaper than continuing a 30-turn session carrying all the context of the previous problem.

The Before/After Numbers

Here's what the full optimization set looked like across our client's engineering team over 30 days.

The output quality metric is the one that matters most. These weren't cuts that traded quality for cost, they were cuts that removed context that wasn't contributing to quality in the first place. Verbosity in your system prompt doesn't make Claude smarter about your codebase. Relevant, targeted context does.

The biggest single win was CLAUDE.md restructuring. It required one afternoon of cleanup and delivered persistent savings across every session from that point forward.

These techniques cut our token spend without degrading output. Context hygiene is not a one-time fix, it's a discipline. Once you build it into how your team works with Claude Code, the savings compound every day.

Full guide with measured before/after, the prompt caching implementation patterns, and the CLAUDE.md template we use across client deployments:

Claude Code Token Optimization - Full Guide

Dextra Labs helps engineering teams deploy and optimize Claude Code and enterprise LLM systems. If your token spend is climbing faster than your output value, it's usually a context architecture problem. hello@dextralabs.com

The AI Coding Tool Stack We Actually Use at a 15-Person Series B Startup in 2026 (With Cost Breakdown)

Dextra Labs — Mon, 27 Jul 2026 05:13:26 +0000

This isn't a "top 10 AI coding tools" listicle. This is the specific tools we use, on specific tasks, at a specific team size, with exact monthly costs. The kind of post we wished existed when we were building our own stack.

The context: Dextra Labs runs a 15-person engineering team. We build AI systems for enterprise clients, LLM deployments, agent architectures, RAG pipelines, and technical due diligence assessments. Our codebase is primarily Python (backend services, ML pipelines) and TypeScript (client-facing applications). We deploy to AWS. We ship to production multiple times daily.

We've tried most of the AI coding tools on the market over the past eighteen months. Some stuck. Some didn't. Here's what we actually use today, why we use each one, and what we tried and dropped.

The daily driver: Cursor Pro

Cost: $20/month per developer × 15 = $300/month

Every developer on the team uses Cursor as their primary IDE. The tab autocomplete alone justifies the cost, it's noticeably faster and more accurate than GitHub Copilot's completions, and it understands our codebase patterns well enough that the completions frequently produce multi-line suggestions that are exactly what we'd type.

Composer agent mode handles our most common development tasks: implementing features from specifications, writing boilerplate, and making consistent changes across multiple files. For a typical feature implementation, "add this endpoint with these fields and this validation", Composer produces a mergeable first draft in under 10 minutes.

What Cursor doesn't do well for us: complex refactors across many files, debugging subtle concurrency issues, and any task where understanding the entire codebase context matters more than quick inline edits. For those tasks, we reach for something else.

The heavy lifter: Claude Code on Max

Cost: $100/month × 3 senior engineers = $300/month

Three of our senior engineers have Claude Code Max subscriptions. They use it for the work that Cursor can't handle: large-scale refactors, architectural analysis, complex debugging sessions, comprehensive test suite generation, and code review preparation.

The 1M token context window is the capability that justifies the price. When we're refactoring a service that touches twelve modules, Claude Code can hold all twelve in context simultaneously and produce changes that are internally consistent across the entire scope. Cursor's context is limited by what fits in the IDE's context window, which means large refactors require breaking the work into smaller scoped pieces.

We also use Claude Code for pre-PR code analysis on critical modules, running it against a branch before submitting the PR to catch issues before human review. This has measurably reduced review cycle time because the PR arrives with obvious issues already addressed.

Not every engineer needs Claude Code. For mid-level developers working on well-scoped tasks within established patterns, Cursor handles everything they need. Claude Code's value scales with task complexity and codebase scope, it's most valuable for the engineers doing the most complex work.

The review layer: CodeRabbit

Cost: $15/month per developer × 15 = $225/month

CodeRabbit runs on every PR automatically via our GitHub integration. It provides AI-powered code review comments that catch common issues before human reviewers see the PR.

We chose CodeRabbit over building our own review pipeline because the maintenance burden of a custom pipeline wasn't justified at our team size. CodeRabbit's detection rate isn't perfect, in our internal testing it catches roughly 60% of the bug categories we care about, but it consistently catches the mechanical issues (missing error handling, potential null references, inconsistent naming) that human reviewers would otherwise spend time on.

The false positive rate was initially too high. We spent two weeks tuning the configuration, adding our project-specific rules, and excluding certain file patterns that generated noise. After tuning, the false positive rate dropped to a level the team finds acceptable, roughly one in five comments is irrelevant, which isn't great but is liveable.

The testing companion: Qodo (formerly CodiumAI)

Cost: Free tier for individual use, team features at $19/month × 5 = $95/month

Five engineers use Qodo's paid tier for test generation. The free tier handles basic test generation well enough for simple modules. The paid tier adds better context awareness and the ability to generate tests that respect our specific testing patterns and fixture setup.

We use Qodo specifically for generating initial test scaffolding, the boilerplate test structure, the basic happy-path tests, the error-path tests. Then we review and enhance by hand, adding the edge cases and business-logic tests that the generator doesn't have context for.

The honest assessment: Qodo saves about 40% of the time we'd spend writing tests from scratch. The generated tests need review and frequently need modification, but starting from a generated scaffold is faster than starting from an empty file.

What we tried and dropped

GitHub Copilot was our starting point eighteen months ago. We switched to Cursor when Cursor's autocomplete quality surpassed Copilot's and the Composer agent mode added a capability Copilot didn't offer. We kept one Copilot Business license for three months as a fallback before canceling entirely.

Windsurf (now Devin Desktop) we trialed for two months. Cascade's contextual awareness was impressive but the ownership changes, OpenAI deal, Google founders departure, Cognition acquisition, made us uncomfortable about building workflows around it. The product is good. The corporate stability isn't there yet for us to commit.

Amazon CodeGuru we tested for code review. The detection quality was decent but the AWS-specific integration requirements didn't fit our workflow, and the pricing at scale was higher than CodeRabbit for comparable capability.

A custom Claude API review pipeline we built ourselves and ran for three months. Detection quality was the best we tested, higher than any commercial tool. But maintaining the prompts, keeping the review rules current, managing the API costs, and debugging edge cases consumed about 8 hours per week of senior engineering time. At 15 engineers, the maintenance burden exceeded the marginal quality improvement over CodeRabbit. We shut it down, though we'd reconsider at 50+ engineers where the per-developer maintenance cost amortises better.

The monthly cost breakdown

$61 per developer per month for the full AI-assisted development stack. Against a fully-loaded engineering cost of roughly $12,000-$15,000 per developer per month in our market, the tooling cost is trivially small relative to even modest productivity gains.

We estimate the stack saves each developer 6-8 hours per week, primarily through faster feature implementation (Cursor autocomplete and Composer), reduced review overhead (CodeRabbit handling mechanical review), faster test writing (Qodo scaffolding), and eliminated debugging dead ends (Claude Code's architectural analysis).

At conservative estimates, 6 hours saved per developer per week × 15 developers × 4 weeks, that's 360 engineering hours per month recaptured. At our loaded cost rate, that's approximately $30,000-$40,000 per month in recaptured engineering capacity against $920 in tooling cost.

The ROI math isn't close.

What would change at 50 or 200 engineers

At 15 engineers, individual tool subscriptions work fine. The total cost is manageable. The configuration is per-developer. Nobody needs centralised management.

At 50+ engineers, the calculus changes. You need SSO integration so developers don't manage individual credentials. You need audit logging for compliance, who used which tool, what code was processed, what suggestions were accepted. You need centralised cost management so the CFO can see one line item instead of seventy-five subscriptions. You need usage analytics to understand whether the tools are actually being used effectively across the team.

At 200+ engineers, you're in enterprise procurement territory. Data residency matters, where does the code go when it's processed? Model selection governance matters, which models are approved for use on which codebases? Custom model training on your proprietary codebase becomes viable and valuable at this scale.

This stack works for a 15-person team. At 200+ engineers, the calculus changes, you need SSO, audit logs, and centralised cost management. We wrote the enterprise breakdown for teams at that scale because the evaluation criteria and the procurement process are fundamentally different from picking tools for a small team.

Published by Dextra Labs, AI Consulting and Enterprise Agent Development

Devin AI vs Claude Code vs Cursor Agent: Which One Actually Ships Production Code? (Benchmarks Inside)

Dextra Labs — Mon, 27 Jul 2026 05:09:11 +0000

"It writes code" is no longer impressive. The question in 2026 is whether it writes code you'd actually merge into a production codebase without spending more time reviewing and fixing than you'd spend writing it yourself.

We gave three autonomous coding agents the same production task and evaluated the output on the only metric that matters: would we ship this?

The task: add rate limiting to an existing Express.js API with Redis backing, configurable per-endpoint limits, proper HTTP response headers, graceful degradation when Redis is unavailable, and comprehensive tests.
This isn't a toy benchmark. It's a feature request that lands in a real team's sprint every quarter.

The agents: Devin (via Devin Desktop, Cognition's rebranded Windsurf platform at $20/month), Claude Code (on Max plan at $100/month with Opus 4.8), and Cursor Agent mode (Pro at $20/month with Composer 2.5).

Same repo. Same requirements document. Same evaluation rubric. No hand-holding during execution, we gave each agent the task and let it work.

The task specification

We provided each agent with identical requirements:

Add rate limiting middleware to the Express API.
Requirements:
- Redis-backed sliding window algorithm
- Configurable limits per endpoint via route decorator
- Standard HTTP headers: X-RateLimit-Limit, 
  X-RateLimit-Remaining, X-RateLimit-Reset, Retry-After
- Graceful fallback when Redis is unavailable 
  (allow requests with warning log, don't block)
- Tests covering: normal operation, rate exceeded, 
  Redis failure, concurrent requests, config validation
- Follow existing project patterns for middleware, 
  error handling, and test structure

The existing codebase had established patterns for middleware registration, error response formatting, and test organisation. We wanted to see which agents respected those patterns versus imposing their own conventions.

Devin's output

Devin worked autonomously for approximately 22 minutes. It explored the codebase, identified the existing middleware patterns, created the implementation files, and ran the test suite to verify its work.

The implementation used a sliding window algorithm with Redis sorted sets, a solid technical choice. The per-endpoint configuration used a decorator pattern that was consistent with the codebase's existing middleware approach.

What worked well: the Redis interaction code was clean, the sorted set implementation was correct, and the HTTP headers were properly formatted. The decorator pattern matched the existing codebase conventions. Devin clearly analysed the existing middleware structure before writing its own.

What didn't ship: the graceful degradation on Redis failure had a critical issue. When Redis was unavailable, the middleware threw an unhandled promise rejection instead of falling through to the allow-with-warning behaviour the requirements specified. The error handling path was partially implemented, there was a try-catch, but the catch block called a logging function that itself tried to write to Redis, creating a recursive failure.

The tests covered the happy path and the rate-exceeded path well. The Redis failure test existed but tested the wrong behaviour, it asserted that requests were blocked when Redis was down, contradicting the requirement for graceful degradation. Two tests had hardcoded timing assertions that would produce flaky failures on slower CI runners.

Code quality score: 3.5/5. Time-to-PR-ready: 45 minutes after fixing the Redis fallback, updating the test assertions, and removing the timing dependencies.

Claude Code's output

Claude Code completed the task in approximately 18 minutes. It loaded the full project context, asked no clarifying questions (the specification was detailed enough), and produced the implementation in a single pass.

The sliding window implementation used Redis MULTI/EXEC transactions for atomicity, a more robust approach than Devin's sorted set method for high-concurrency scenarios. The per-endpoint configuration used the existing decorator pattern.

What worked well: the Redis failure handling was the best of the three. A circuit breaker pattern, after three consecutive Redis failures, the middleware switches to allow-all mode for 30 seconds before retrying. This is more sophisticated than the requirements asked for, and it's the correct production pattern because it prevents cascading retry storms when Redis is genuinely down. The tests included a concurrent request test using Promise.all that actually exercised the race condition path.

What needed review: the circuit breaker added complexity that the requirements didn't specify. We debated whether to keep it. We kept it, it's the right engineering decision even though it wasn't requested. One test used a mock Redis client that didn't perfectly replicate the real client's error throwing behaviour, causing a false positive on the error path test.

Code quality score: 4.5/5. Time-to-PR-ready: 15 minutes, fixing the mock client issue and adding a comment explaining the circuit breaker design decision.

Cursor Agent's output

Cursor's Composer agent mode completed the task in approximately 14 minutes, the fastest of the three. The inline IDE experience meant we could watch the implementation happen in real time and the iteration cycle was the most fluid.

The sliding window used a Redis INCR with TTL approach, simpler than the other two implementations, which is both a strength (less code, easier to understand) and a limitation (less precise window boundaries at high concurrency).

What worked well: the code was the most readable of the three implementations. Clear variable names, good comments, consistent with the existing codebase style. The test structure followed the project's existing patterns exactly. The HTTP headers were correctly implemented.

What didn't ship: the rate limiting algorithm had a subtle issue at window boundaries. Because it used INCR with a fixed TTL rather than a true sliding window, a burst of requests at the end of one window and the beginning of the next could exceed the intended limit. At low traffic this doesn't matter. At high traffic with strict rate requirements, it's a bug.

The Redis failure handling worked but was simplistic, a try-catch that logged a warning and allowed the request through. No circuit breaker, no backoff, no tracking of Redis health. Functional but not production-hardened.

The tests were well-structured and covered the main paths but didn't include a concurrent request test. The boundary condition issue wouldn't have been caught by the generated tests.

Code quality score: 3.5/5. Time-to-PR-ready: 35 minutes after replacing the INCR approach with a proper sliding window and adding a concurrent test.

The comparison matrix

What this actually tells us

Claude Code produced the only output we would have merged with minimal changes. The circuit breaker pattern, the atomic transactions, and the concurrent test demonstrated understanding of production concerns that the other two agents didn't exhibit.

Cursor was fastest to initial output but the window boundary issue meant more human review time. The code was the most readable and the most consistent with existing style, which matters for long-term maintainability.

Devin's recursive Redis failure is the kind of bug that would have caused a production incident if it shipped. It's also the kind of bug that's hard to catch in review because the error handling code looks correct at first glance, you have to trace the execution path through the logging function to see that it re-enters the failing Redis connection.

The pattern across all three: the "happy path" implementation is solid from every agent. The differentiation is in error handling, edge cases, and production hardening. These are the dimensions where engineering experience matters most, and they're the dimensions where the gap between agents is widest.

Autonomous coding agents are advancing fast, but "shipping production code" is a much higher bar than passing SWE-bench. The benchmarks test whether the agent can solve a defined problem. Production readiness tests whether it handles the problems that aren't defined, the edge cases, the failure modes, the concurrent access patterns that the requirements didn't specify because experienced engineers handle them implicitly.

We wrote the full CTO comparison with ten tools for teams evaluating autonomous coding agents at enterprise scale, covering not just output quality but the procurement criteria that matter for teams of 20-200 engineers.

Published by Dextra Labs, AI Consulting and Enterprise Agent Development

AI Code Review Tools Are Missing 40% of Real Bugs, Here's What We Found After Testing 8 of Them

Dextra Labs — Tue, 21 Jul 2026 05:11:21 +0000

We planted 23 known bugs in a production-realistic codebase and ran eight AI code review tools against it.

The best tool caught 14 of them. The worst caught 7. The average detection rate across all eight tools was 60.3%. Which means the average AI code review tool missed roughly 40% of real, exploitable, production-impacting bugs that a human reviewer would be expected to catch.

Before the pitchforks come out, this isn't an indictment of AI code review. These tools catch genuine issues, improve code quality, and save reviewer time. But the marketing claims of "catches bugs before they ship" need calibration against what they actually catch and what they miss.

We published this because nobody else is testing these tools on realistic bugs. The existing comparisons test detection of toy examples or run tools against their own cherry-picked test cases. We wanted to know: if you're a senior engineer who would catch a race condition in review, does the AI tool catch it too?

The answer, for 40% of realistic bugs, is no.

The test setup

We created a deliberately buggy version of a mid-sized Python/FastAPI application, approximately 15,000 lines of code across 34 modules. The application handles user authentication, payment processing, order management, and a notification system. It's representative of the kind of B2B SaaS backend that most of these tools are marketed for.

We planted 23 bugs across eight categories. Every bug was a real bug pattern we've encountered in production systems during our code review and technical due diligence work. None were synthetic puzzles. All would have real consequences if deployed.

The categories and planted bugs:

SQL injection vulnerabilities: 3 bugs. One in a search endpoint using string concatenation. One in a reporting query with dynamic column selection. One in an admin tool with parameterised query syntax that was assembled incorrectly.

Authentication and authorisation bypasses: 3 bugs. One IDOR (Insecure Direct Object Reference) where user A could access user B's data by modifying the resource ID. One missing authentication check on an internal API endpoint. One JWT validation that checked expiry but not signature.

Race conditions: 3 bugs. One TOCTOU in payment processing where balance check and deduction weren't atomic. One in a coupon redemption where concurrent requests could redeem beyond the limit. One in a task queue where concurrent workers could pick up the same job.

Memory and resource leaks: 3 bugs. One database connection not returned to pool on exception path. One file handle left open in an error branch. One Redis connection created per request without pooling.

Off-by-one and boundary errors: 3 bugs. One pagination that skipped the last page. One date range that excluded the end date. One array index that would crash on empty input.

Input validation failures: 3 bugs. One path traversal in a file upload endpoint. One XXE vulnerability in XML parsing. One ReDoS-susceptible regex pattern.

Logic errors: 3 bugs. One pricing calculation that applied discount after tax instead of before. One notification that was sent on failed status instead of success. One sorting that reversed the intended order.

Data handling violations: 2 bugs. One endpoint that logged PII in plaintext. One that returned deleted user data in API responses.

The eight tools tested

We ran eight tools that represent the current market spectrum: GitHub Copilot code review, Cursor's built-in review, CodeRabbit, Qodo (formerly CodiumAI), Amazon CodeGuru, Sourcery, DeepSource, and a custom Claude-based review pipeline we built for client work.

Each tool was given the same codebase in the same state. We submitted the buggy code as a PR against the clean baseline and captured every issue each tool flagged.

The detection matrix

What the data reveals

The category breakdown is more interesting than the aggregate numbers.

SQL injection and input validation were the best-detected categories across all tools. Most tools caught the obvious string concatenation injection and the path traversal. These are well-represented in training data and well-documented as vulnerability patterns. If your concern is common OWASP-style vulnerabilities, most AI review tools provide decent coverage.

Race conditions were catastrophically underdetected. Only two tools caught any race conditions at all. The TOCTOU in payment processing, arguably the highest-impact bug in the entire test set, was caught by only two of eight tools. Race conditions require understanding concurrent execution flows, which most AI review tools don't reason about effectively. They analyse the code path sequentially and don't model what happens when two threads execute the same function simultaneously.

This matters because race conditions in payment processing, inventory management, and resource allocation are among the most costly production bugs. They're also among the hardest for human reviewers to catch, which means they're the category where AI assistance would be most valuable and they're the category where AI assistance is currently weakest.

Logic errors were nearly invisible to most tools. The pricing calculation bug, discount applied after tax instead of before, was caught by only three tools. This is because detecting logic errors requires understanding business intent, not just code structure. The code was syntactically correct, followed good patterns, and would pass any static analysis. It just didn't do what the business required. No tool can catch this without understanding what the code is supposed to do, which requires context that most review tools don't have.

Authentication bypasses had moderate detection but the patterns that were caught were the obvious ones. The missing auth check on an endpoint was caught by most tools. The subtle JWT signature validation issue, where the code checked expiry but not signature, was caught by only two. The IDOR was caught by three. The pattern: obvious security omissions get caught, subtle security logic errors get missed.

The false positive problem

Detection rate is only half the story. False positive rate determines whether developers actually use the tool or start ignoring its output.

CodeRabbit had the highest detection rate among commercial tools but also the highest false positive count. Sourcery had the worst ratio, more false positives than true positives. The custom Claude pipeline had the best balance of high detection with low false positives, largely because we tuned the review prompt specifically for this codebase's patterns and explicitly instructed the model to flag only issues with concrete impact.

False positives above 40% are adoption killers. When developers see AI review comments that are wrong or irrelevant more than once in every three comments, they stop reading the comments. The tool becomes noise that the team learns to ignore, which is worse than not having the tool at all because it creates a false sense of security.

What this means for teams using AI code review

AI code review is valuable. It catches real bugs, enforces consistency, and frees human reviewers from mechanical checks. But it is not a replacement for human review on the bug categories that matter most, race conditions, logic errors, subtle security vulnerabilities, and business rule violations.

The effective deployment pattern: AI review handles the first pass. It catches the common vulnerabilities, the style issues, the obvious errors. Human reviewers then focus their attention on the categories AI consistently misses, concurrency, business logic, architectural decisions, and the subtle security patterns that require understanding intent rather than just structure.

Tool selection is only half the battle. The other half is workflow integration, where in your PR cycle does AI review sit, what does it escalate to human reviewers, and how do you prevent the team from treating AI review as sufficient review? The AI code review enterprise guide covers the full workflow design, including the organisational changes that make AI review additive rather than a false safety net.

Published by Dextra Labs, AI Consulting and Enterprise Agent Development

On-Premise AI Code Review: How We Deployed Claude Locally in an Air-Gapped Environment (Setup Guide)

Dextra Labs — Sun, 19 Jul 2026 18:44:52 +0000

The security officer said no.

Not "maybe with some modifications." Not "let us review the data handling policy." No. Cloud-hosted AI code review was not going to happen in their environment. The codebase contained classified material. The network was air-gapped. No data leaves the building. End of conversation.

This was a defence contractor. Their development team was writing code that couldn't touch any external API, any cloud service, any SaaS platform. But they were also writing 200,000+ lines of code per year with a team that was struggling to maintain review quality across twelve engineers, the exact problem that AI code review solves.

The challenge: deploy AI code review capability that runs entirely on-premise, in an air-gapped network, with no external connectivity, while meeting the security and compliance requirements of a classified computing environment.

This is the guide we wish existed when we started that project. It covers hardware requirements, model selection, inference server setup, CI/CD integration and the security architecture that made it pass the compliance review.

Why air-gapped AI code review is different from everything else

Most AI code review guides assume cloud connectivity. The model runs on the provider's infrastructure. The code is sent via API. The review comes back. The security question is about data handling policies, encryption in transit and vendor trust.

In an air-gapped environment, none of that applies. The model runs on hardware you control. The code never leaves your network. There is no vendor to trust because there is no vendor in the loop.

This sounds simpler from a security perspective. In some ways it is. In other ways it's significantly more complex because you're now responsible for the entire stack, model selection, quantisation, inference hardware, deployment, monitoring, updating and maintaining the system that in a cloud deployment is someone else's problem.

Hardware requirements

The hardware question is the first decision and the one that constrains everything else.

For on-premise AI code review, the model needs to be large enough to understand code semantics, identify patterns and generate meaningful review comments. Small models (7B parameters) can handle basic linting and style checking but lack the reasoning capability for substantive code review. Large models (70B+ parameters) provide review quality approaching cloud-hosted frontier models but require significant GPU infrastructure.

Our recommendation for production on-premise code review:

For teams under 25 engineers with moderate review volume: a single server with 2x NVIDIA A100 80GB GPUs running a 70B parameter model at 4-bit quantisation. This handles the inference load for a team producing 15-25 PRs per day with acceptable latency (30-60 seconds per review). Total hardware cost: approximately $30,000-$45,000 for the GPU server, which amortises over three years to roughly $10,000-$15,000 per year, comparable to cloud-hosted review tool licensing for a team of this size.

For teams of 25-75 engineers: two inference servers with 4x A100 80GB GPUs each, behind a load balancer. This handles 50-80 PRs per day with consistent latency. The second server provides redundancy, if one goes down, review capability continues at reduced throughput rather than stopping entirely.

For teams above 75 engineers: a dedicated inference cluster with horizontal scaling. At this scale, the infrastructure management becomes a full-time operations concern and we recommend dedicated MLOps support.
CPU-only inference is technically possible using llama.cpp with heavily quantised models (4-bit or lower) but the latency is prohibitive for production use, 3-5 minutes per review versus 30-60 seconds on GPU. We tested it and abandoned it after the team stopped waiting for reviews and merged without them.

Model selection

In an air-gapped environment, you're limited to open-weight models that can be downloaded, transferred via physical media and deployed locally. The frontier closed models (Claude, GPT) are not available offline.

The models we've tested and can recommend for on-premise code review as of mid-2026:

Llama 3.3 70B provides the best balance of code understanding, review quality and inference efficiency. At 4-bit quantisation (GPTQ or AWQ), it fits on 2x A100 80GB GPUs with room for reasonable batch sizes. The review quality on Python, TypeScript, Java and C++ codebases is strong, it catches logical errors, security vulnerabilities and pattern inconsistencies at a level that meaningfully reduces human review burden.

DeepSeek Coder V3 is specialised for code tasks and provides excellent results on code-specific review. The model is smaller (33B in its most capable variant) and runs on less hardware, making it a good choice for smaller deployments. It outperforms Llama 3.3 70B on code-specific benchmarks but has weaker general reasoning for architectural review comments.

Qwen 2.5 Coder 32B is another strong code-specialised option that balances quality and resource requirements well.

Our recommendation: start with DeepSeek Coder V3 for code-focused review, add Llama 3.3 70B if you need broader architectural analysis and review comments that explain why something is a problem, not just that it is one.

Inference server setup

We use vLLM as the inference server for production deployments. It handles model loading, request queuing, continuous batching and provides an OpenAI-compatible API endpoint that makes integration with existing tools straightforward.

The deployment architecture:

┌─────────────────────────────────────────────┐
│              Air-gapped network             │
│                                             │
│  ┌───────────┐    ┌──────────────────────┐  │
│  │  GitLab   │───▶│  CI/CD Runner        │  │
│  │  Server   │    │  (review pipeline)   │  │
│  └───────────┘    └──────────┬───────────┘  │
│                              │              │
│                              ▼              │
│                   ┌──────────────────────┐  │
│                   │  Load Balancer       │  │
│                   └──────────┬───────────┘  │
│                    ┌─────────┴─────────┐    │
│                    ▼                   ▼    │
│           ┌──────────────┐   ┌──────────────┐
│           │  vLLM Node 1 │   │  vLLM Node 2 │
│           │  2x A100     │   │  2x A100     │
│           │  Llama 70B   │   │  DeepSeek    │
│           └──────────────┘   └──────────────┘
│                                             │
└─────────────────────────────────────────────┘

The dual-node setup serves two purposes: redundancy (either node can handle all traffic if the other is down) and model specialisation (one node runs the general-purpose model for architectural review, the other runs the code-specialised model for detailed code analysis). The review pipeline can route different review types to different models.

Setting up vLLM on each node:

# Install vLLM (transfer wheel files via approved media)
pip install vllm --no-index --find-links /media/packages/

# Start the inference server
python -m vllm.entrypoints.openai.api_server \
  --model /models/llama-3.3-70b-awq/ \
  --quantization awq \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --port 8000 \
  --host 0.0.0.0

The tensor-parallel-size 2 flag splits the model across both GPUs. The max-model-len 32768 sets the maximum context window, large enough to review substantial diffs but not so large that it exhausts GPU memory.
For the containerised deployment:

FROM nvidia/cuda:12.4.0-base-ubuntu22.04
COPY ./packages /packages
RUN pip install --no-index --find-links /packages vllm
COPY ./models /models
EXPOSE 8000
CMD ["python", "-m", "vllm.entrypoints.openai.api_server", \
     "--model", "/models/llama-3.3-70b-awq/", \
     "--quantization", "awq", \
     "--tensor-parallel-size", "2", \
     "--port", "8000"]

CI/CD integration

The review pipeline hooks into your existing CI/CD system. We'll use GitLab CI as the example since it's the most common on-premise Git platform in classified environments, but the pattern adapts to any CI system.

The pipeline stage triggers on every merge request. It extracts the diff, formats the review prompt with project-specific conventions, sends it to the vLLM endpoint, parses the response and posts review comments back to the merge request.

# .gitlab-ci.yml
ai-code-review:
  stage: review
  script:
    - python scripts/ai_review.py
      --diff-ref $CI_MERGE_REQUEST_DIFF_BASE_SHA
      --target-ref $CI_COMMIT_SHA
      --mr-iid $CI_MERGE_REQUEST_IID
      --api-url http://vllm-lb.internal:8000/v1
      --rules-file .review-rules.md
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

The review script:

import requests
import subprocess
import json

def get_diff(base_ref, target_ref):
    result = subprocess.run(
        ["git", "diff", base_ref, target_ref],
        capture_output=True, text=True
    )
    return result.stdout

def request_review(diff, rules, api_url):
    prompt = f"""Review this code diff against these project rules:

{rules}

Diff:
{diff}

Provide review comments as JSON array with fields:
file, line, severity (critical/warning/info), comment
Only flag genuine issues. No style-only comments unless
they violate a specific rule above."""

    response = requests.post(
        f"{api_url}/chat/completions",
        json={
            "model": "llama-3.3-70b-awq",
            "messages": [{"role": "user", "content": prompt}],
            "temperature": 0.1,
            "max_tokens": 4096
        }
    )
    return json.loads(
        response.json()["choices"][0]["message"]["content"]
    )

The temperature: 0.1 is deliberate, we want consistent, deterministic reviews, not creative suggestions. The rules file (.review-rules.md) contains project-specific conventions that the model uses as review criteria, similar to how cloud-hosted review tools use configuration files.

Security considerations for classified environments

The model files themselves need security classification review before deployment. Open-weight models trained on public data don't contain classified information, but the review process requires documentation that the model was trained on public data only and that no classified data has been used for fine-tuning or prompt construction.

The inference server must run in the same security zone as the codebase. No network paths between the inference server and any external network, even through proxy or gateway. The load balancer is internal only.

Audit logging captures every review request and response, the diff submitted, the review comments generated and the pipeline metadata. This audit trail is required for both security compliance and for monitoring the AI review quality over time.

Model updates require the same physical media transfer process as the initial deployment. When a new model version is released, it's downloaded on an unclassified system, transferred via approved media and deployed through the standard change management process. This means model updates happen quarterly at most, not continuously.

Air-gapped deployment is one piece of the puzzle. The full compliance and security framework for on-premise AI code review, covering SOC 2, HIPAA, PCI and government security requirements, is in our complete guide. It covers the policy documentation, access controls and audit procedures that the technical deployment needs to operate within.

Published by Dextra Labs, AI Consulting and Enterprise Agent Development

I Tested Cursor, Claude Code, Windsurf, and Codex on the Same Codebase for 30 Days. Here Are the Results.

Dextra Labs — Thu, 16 Jul 2026 04:39:19 +0000

The comparison posts flooding your feed right now have a problem. They test each tool on different codebases, different tasks, and different complexity levels, then draw conclusions as though they conducted a controlled experiment. They didn't. They tested four different things and called it a comparison.

We decided to do it properly.

Same codebase. Same four tasks. Same evaluation criteria. Thirty days. No sponsorships, no affiliate links, no predetermined conclusion. We run these tools daily on client engagements at Dextra Labs and wanted to know, with actual data, which tool performs best on which type of work.

The codebase: a production Python/FastAPI backend with 47 modules, PostgreSQL, Redis caching, Celery task queues, and roughly 38,000 lines of code. Real-world complexity, not a tutorial project, not a greenfield demo. An existing system with technical debt, inconsistent patterns, and the kind of accumulated decisions that real production codebases carry.

The tools: Cursor Pro ($20/month), Claude Code on Max ($100/month), Windsurf Pro ($20/month), and OpenAI Codex via ChatGPT Plus ($20/month). All tested during June-July 2026 on their latest available models.

The four tasks

We chose tasks that represent the actual work developers do daily, not synthetic benchmarks.

Task 1 - Refactor: Extract the authentication middleware from a monolithic route file into a separate module with proper dependency injection, updating all 14 files that import from it. This tests multi-file understanding, import chain management, and architectural coherence.

Task 2 - Greenfield feature: Add rate limiting to the API with Redis backing, configurable per-endpoint limits, proper HTTP headers (X-RateLimit-Remaining, Retry-After) and comprehensive tests. This tests feature implementation from scratch, including edge case handling.

Task 3 - Bug fix: A reported production bug where concurrent Celery tasks were occasionally double-processing the same job. Root cause was a race condition in the task deduplication logic. This tests debugging ability, concurrency understanding, and fix quality.

Task 4 - Test generation: Generate meaningful tests for the payment processing module, which had 11% coverage. Not just coverage, tests that catch real regressions. We evaluated generated tests by running mutation testing against them.

Scoring methodology

We scored each tool on five dimensions for each task, each on a 1-5 scale.
Correctness: does the output work without modification? Completeness: does it handle edge cases and error paths? Code quality: is the code clean, maintainable, and consistent with existing patterns? Time-to-PR-ready: total time from prompt to a PR we'd actually merge, including our review and fixes. Review overhead: how much human correction was needed after the tool finished.
Total possible score per task: 25. Total possible across all four tasks: 100.

Task 1 results: Refactor

Claude Code dominated this task. It loaded the entire codebase context, identified all 14 files that needed updating, and produced a complete refactor in a single planning pass. The import chain was handled correctly. The dependency injection pattern it chose was consistent with the existing codebase conventions. One minor issue: it renamed an internal helper function unnecessarily, which we caught in review. Time-to-PR-ready: 23 minutes. Score: 22/25.

Cursor handled the refactor well through its Composer agent mode but required two iterations. The first pass correctly extracted the middleware but missed two import updates in files that imported indirectly through a barrel export. After pointing this out, the second pass fixed it cleanly. Time-to-PR-ready: 38 minutes. Score: 19/25.

Codex produced a clean initial extraction but struggled with the import chain complexity. It correctly updated 11 of 14 files. The three it missed were in subdirectories that imported the auth module through a re-export pattern. We fixed these manually. Time-to-PR-ready: 52 minutes. Score: 16/25.

Windsurf's Cascade mode completed the extraction with good coherence, it maintained a single context throughout and the changes were internally consistent. It missed one edge case where a test file imported the middleware directly rather than through the application module. Time-to-PR-ready: 41 minutes. Score: 18/25.

Task 2 results: Greenfield feature

This is where the tools were most evenly matched, and where Cursor's IDE integration gave it a practical edge.

Cursor delivered the most complete implementation. The rate limiting middleware included Redis backing, per-endpoint configuration, proper HTTP headers, sliding window algorithm, and tests that covered the happy path, rate-exceeded path, and Redis connection failure fallback. The inline development experience, writing code, testing in the integrated terminal, iterating on failures, was the most fluid. Time-to-PR-ready: 34 minutes. Score: 23/25.

Claude Code produced the highest quality code architecturally, cleaner separation, better error handling, more thorough documentation, but the terminal-only workflow meant more manual verification. The implementation was excellent but took longer to validate. Time-to-PR-ready: 42 minutes. Score: 22/25.

Windsurf delivered a solid implementation with good test coverage. The sliding window logic had a subtle off-by-one error in the expiry calculation that we caught during review. Time-to-PR-ready: 48 minutes. Score: 19/25.

Codex produced a working implementation but made a security-relevant choice we disagreed with, storing rate limit counters with a fixed TTL rather than a sliding window, which creates a predictable reset point an attacker could time around. Functionally correct, strategically questionable. Time-to-PR-ready: 55 minutes. Score: 17/25.

Task 3 results: Bug fix

The race condition bug was the hardest task and produced the widest performance gap.

Claude Code found the root cause in its first analysis pass. It identified the deduplication logic, recognised the TOCTOU (time-of-check-to-time-of-use) vulnerability, and proposed a fix using Redis SETNX for atomic deduplication. The fix included a test that specifically reproduced the race condition using concurrent task execution. Time-to-PR-ready: 19 minutes. Score: 24/25.

Cursor identified the area of the codebase where the bug likely lived but proposed adding a database lock rather than fixing the deduplication logic. The fix would have worked but was heavier than necessary and would have introduced latency on every task execution. After pointing this out, it proposed the SETNX approach as an alternative. Time-to-PR-ready: 44 minutes. Score: 17/25.

Windsurf struggled with this task. It identified the deduplication module but proposed adding retry logic rather than addressing the race condition itself. The proposed fix would have reduced the symptom frequency without eliminating the root cause. Time-to-PR-ready: 62 minutes after we redirected it to the actual problem. Score: 14/25.

Codex correctly identified the race condition but the proposed fix had its own concurrency issue, it used a Python threading lock in a Celery context where tasks may run in separate processes. The fix wouldn't have worked in the production configuration. Time-to-PR-ready: 58 minutes after we caught and corrected this. Score: 15/25.

Task 4 results: Test generation

Cursor generated tests the fastest and with the highest raw coverage, 87% on the payment module. However, mutation testing revealed that 31% of the generated tests were implementation-detail tests that would pass even with deliberate bugs injected. Effective mutation kill rate: 62%. Time-to-PR-ready: 28 minutes. Score: 18/25.

Claude Code generated fewer tests but with notably higher quality. Coverage reached 78%, but the mutation kill rate was 81%, meaning the tests were genuinely catching bugs, not just executing code paths. It also generated property-based tests for the amount calculation logic, which none of the other tools attempted. Time-to-PR-ready: 35 minutes. Score: 22/25.

Codex generated comprehensive tests with decent coverage (82%) and reasonable mutation kill rate (68%). The tests were conventional and well-structured. No surprises, positive or negative. Time-to-PR-ready: 32 minutes. Score: 19/25.

Windsurf generated tests with 79% coverage but the lowest mutation kill rate at 54%. Many tests asserted that functions returned specific values without testing the business logic that produced those values. Time-to-PR-ready: 40 minutes. Score: 16/25.

The aggregate scorecard

What the numbers actually mean

Claude Code won on total score, but Cursor won on the task most developers do most frequently, greenfield feature implementation within an IDE workflow. The daily experience of writing new code in Cursor is more fluid than any other tool we tested. The tab autocomplete alone is worth the subscription for inline development speed.

Claude Code's advantage is concentrated in tasks that require deep codebase understanding, the refactor, the race condition bug, the high-quality test generation. These are the tasks where the 1M token context window and terminal-native agentic architecture create a measurable difference. If your bottleneck is complex architectural work, Claude Code is the clear winner.

Codex and Windsurf performed similarly in aggregate but for different reasons. Codex is strong on well-defined, straightforward tasks. Windsurf's Cascade provides better contextual coherence on multi-file edits but struggles when the task requires reasoning about concurrent systems or subtle bugs.

What we actually use daily

After thirty days of controlled testing, here's what we landed on: Cursor for daily inline development, feature implementation, quick edits, code exploration. Claude Code for complex refactors, bug investigations, test suite improvements, and any task that benefits from holding the entire codebase in context. We don't use them interchangeably. They're complementary tools for different cognitive tasks.

For teams evaluating these tools, the comparison of Claude Code vs Cursor vs Windsurf covers the feature-by-feature breakdown with recommendations by team size and workflow type.

For enterprise teams evaluating these at scale, the criteria change, data residency, SSO, audit logs, and centralised cost management matter more than accuracy on any single task. We wrote the enterprise-focused breakdown separately because the procurement decision is fundamentally different from the developer experience decision.

Published by Dextra Labs, AI Consulting and Enterprise Agent Development

How to Architect AI Agents That Pass Banking Compliance Audits (Real Patterns, Not Theory)

Dextra Labs — Sun, 28 Jun 2026 19:51:23 +0000

Building agents for banking is 30% AI work and 70% compliance plumbing. The 30% is the easy part. Here's how to handle the 70%.

The first time you build an underwriting agent for a bank, you'll write the credit logic in about a week. Then you'll spend the next two months on audit logging, explainability, human-in-the-loop checkpoints, and data residency and you'll understand why fintech AI projects take longer than the demo suggests.

This is the architecture that gets through an audit, using loan underwriting as the running example.

The Core Requirement: Every Decision Must Be Traceable

Regulators don't accept "the model gave it a score of 0.82." They want to know what data was used, what reasoning was applied, and what a human would need to review to understand and challenge the decision.

class AuditableDecision:
    """
    Every agent decision in a regulated context must
    produce this structure. Not optional. Not added later.
    """
    def __init__(self, decision_id: str):
        self.decision_id = decision_id
        self.inputs_used = {}
        self.reasoning_steps = []
        self.data_sources_consulted = []
        self.model_version = AGENT_VERSION
        self.timestamp = datetime.utcnow().isoformat()
        self.outcome = None
        self.confidence = None
        self.human_reviewable_explanation = None

    def add_reasoning_step(self, step_description: str, evidence: dict):
        self.reasoning_steps.append({
            "step": len(self.reasoning_steps) + 1,
            "description": step_description,
            "evidence": evidence,
            "timestamp": datetime.utcnow().isoformat()
        })

    def finalise(self, outcome: str, confidence: float, explanation: str):
        self.outcome = outcome
        self.confidence = confidence
        self.human_reviewable_explanation = explanation

    def to_audit_record(self) -> dict:
        return {
            "decision_id": self.decision_id,
            "inputs": self.inputs_used,
            "reasoning_chain": self.reasoning_steps,
            "data_sources": self.data_sources_consulted,
            "model_version": self.model_version,
            "outcome": self.outcome,
            "confidence": self.confidence,
            "explanation": self.human_reviewable_explanation,
            "timestamp": self.timestamp
        }

Every step in the agent's reasoning gets appended to this structure as it happens, not reconstructed afterward from logs, which is unreliable and which auditors specifically check for.

The Underwriting Decision Pipeline

async def run_underwriting_agent(application: dict) -> dict:
    decision = AuditableDecision(decision_id=application["application_id"])

    # Step 1: Document verification
    doc_result = await verify_documents(application["documents"])
    decision.data_sources_consulted.append("document_verification_service")
    decision.add_reasoning_step(
        "Verified submitted financial documents",
        {"documents_checked": doc_result["count"], 
         "verification_status": doc_result["status"]}
    )

    if doc_result["status"] != "verified":
        decision.finalise(
            outcome="ESCALATE_TO_HUMAN",
            confidence=1.0,
            explanation=f"Document verification failed: {doc_result['reason']}. "
                        f"Routed to human reviewer for manual document check."
        )
        await audit_store.append(decision.to_audit_record())
        return {"status": "escalated", "reason": "document_verification"}

    # Step 2: Risk scoring via Claude with explicit reasoning
    risk_response = await client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1500,
        system="""Analyse this loan application's risk factors. 
        You MUST cite specific data points for every risk factor 
        identified. Vague reasoning fails compliance review.

        Return JSON: {
            "risk_score": float,
            "risk_factors": [{"factor": str, "data_point": str, "weight": str}],
            "recommendation": "APPROVE" | "DENY" | "HUMAN_REVIEW",
            "explanation": "2-3 sentences a non-technical auditor can verify"
        }""",
        messages=[{"role": "user", "content": json.dumps(application["financials"])}]
    )

    risk_analysis = json.loads(risk_response.content[0].text)
    decision.add_reasoning_step(
        "Risk assessment based on financial data",
        risk_analysis
    )

    # Step 3: Threshold-based routing - NOT autonomous final decisions
    if risk_analysis["risk_score"] > 0.7:
        outcome = "HUMAN_REVIEW"  # high risk always escalates
    elif risk_analysis["risk_score"] < 0.3:
        outcome = "AUTO_APPROVE"  # only very low risk auto-decides
    else:
        outcome = "HUMAN_REVIEW"  # middle ground escalates too

    decision.finalise(
        outcome=outcome,
        confidence=1 - abs(0.5 - risk_analysis["risk_score"]),
        explanation=risk_analysis["explanation"]
    )

    await audit_store.append(decision.to_audit_record())
    return {"status": outcome, "decision_record": decision.to_audit_record()}

Note the threshold structure: only very low risk applications auto-approve. Everything else, high risk and the ambiguous middle ground, escalates to a human. This isn't conservative for the sake of it; it's the threshold design that auditors expect and that protects the institution from autonomous decisions on the cases that matter most.

Human-in-the-Loop: Not a Button, a Checkpoint

A human-in-the-loop checkpoint needs to give the reviewer enough information to make a genuine decision, not just a button to click.

def build_human_review_package(decision: AuditableDecision, 
                                 application: dict) -> dict:
    return {
        "application_summary": application["summary"],
        "agent_reasoning_chain": decision.reasoning_steps,
        "agent_recommendation": decision.outcome,
        "confidence_level": decision.confidence,
        "specific_concerns": [
            rf for rf in decision.reasoning_steps 
            if rf.get("evidence", {}).get("weight") == "high"
        ],
        "override_requires_justification": True,
        "regulatory_basis": get_applicable_lending_regulations(application)
    }

override_requires_justification: True matters because a reviewer who can override without explaining why produces rubber-stamp approval rates, not genuine oversight and that's specifically what examiners check for.

PII Handling in Prompts

Financial data going into LLM prompts needs careful handling, especially for cross-border banks with data residency requirements.

def sanitise_for_prompt(application: dict, residency_zone: str) -> dict:
    """
    Strip or tokenise PII before it reaches the LLM prompt,
    based on the data residency requirements for this customer.
    """
    sanitised = application.copy()

    # Replace direct identifiers with tokens
    sanitised["applicant_name"] = f"APPLICANT_{hash_id(application['applicant_id'])}"
    sanitised["ssn_or_national_id"] = "[REDACTED]"

    if residency_zone == "EU":
        # GDPR-specific handling
        sanitised["address"] = tokenise_address(application["address"])

    return sanitised

For cross-border banks, the model call itself may need to happen in a specific region, or via a locally-hosted model, depending on the residency zone, this is an architectural decision made before any agent code is written, not a configuration flag added later.

What Auditors Actually Check

Decision immutability, audit records can't be modified after creation. Reasoning chain completeness, every step that influenced the outcome must be in the record, not reconstructed from application logs after the fact. Override documentation, every human override of an agent recommendation requires a written justification. Threshold justification, the bank must be able to explain why the auto-approve threshold is set where it is, with evidence it doesn't create disparate impact.

The full AI agents for underwriting in banking architecture guide covers the complete pipeline including the specific regulatory frameworks (FCRA, ECOA, FCA lending standards) that shape these design decisions.

Underwriting Is One Workflow

The patterns here, audit trails, explainability, human-in-the-loop thresholds, generalise across banking AI use cases, but the specific requirements differ meaningfully by workflow. Fraud detection has different latency requirements (you need decisions in milliseconds, not minutes) and different audit needs (pattern-based reasoning rather than document-based reasoning). Worth reading the fraud detection architecture guide before you design either system, because the architectural choices you make for one affect how cleanly you can extend to the other.

Published by Dextra Labs | AI Consulting & Enterprise Agent Development

RAG vs Agentic AI: A Developer's Decision Tree (With Code Examples for Both)

Dextra Labs — Wed, 24 Jun 2026 20:57:33 +0000

Two different problems wearing similar clothes. Here's how to tell them apart in thirty seconds, with working code for both.

I see this confusion in almost every project kickoff: "We need RAG" when the actual requirement is agentic, or "we need an agent" when RAG would be simpler, cheaper, and faster to ship.

Let's fix that with a decision tree you can actually use, plus working code for each path.

The Decision Tree

Does your system need to ANSWER QUESTIONS from documents?
├── YES, and that's the whole job → RAG
└── YES, but it also needs to TAKE ACTIONS across systems
    └── → Agent that uses RAG as a tool

Does your system need to TAKE ACTIONS across multiple systems?
├── YES, with no document retrieval needed → Plain Agent
└── YES, and it needs grounded knowledge from documents → 
    → Agent that uses RAG as a tool

The test question that resolves most confusion: "Does this system need to decide what to do, or does it need to find and synthesise information?" Finding and synthesising → RAG. Deciding and acting → agent.

Path 1: Pure RAG

RAG is the right architecture when your job is grounding LLM responses in a specific document set, answering questions, summarising content, finding relevant passages.

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import Chroma
from langchain.chains import RetrievalQA
from langchain_anthropic import ChatAnthropic

# 1. Load and chunk documents
splitter = RecursiveCharacterTextSplitter(
    chunk_size=800, 
    chunk_overlap=100
)
chunks = splitter.split_documents(documents)

# 2. Embed and store
embeddings = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-mpnet-base-v2"
)
vectorstore = Chroma.from_documents(chunks, embeddings)

# 3. Build the retrieval chain
llm = ChatAnthropic(model="claude-sonnet-4-5")
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever(search_kwargs={"k": 4}),
    return_source_documents=True
)

# 4. Query
result = qa_chain({"query": "What is our refund policy for enterprise customers?"})
print(result["result"])
print(result["source_documents"])  # Always show sources

This is the whole job: retrieve relevant chunks, ground the LLM's answer in them, return a response with citations. No planning loop, no tool orchestration, no multi-step decision-making. If your use case stops here, building agent infrastructure on top of this is unnecessary complexity.

Path 2: Pure Agent (No RAG)

An agent is right when the job is taking actions, checking systems, executing operations, making decisions that span multiple steps and there's no document knowledge base involved.

import anthropic

client = anthropic.Anthropic()

tools = [
    {
        "name": "check_inventory",
        "description": "Check current stock level for a SKU",
        "input_schema": {
            "type": "object",
            "properties": {"sku": {"type": "string"}},
            "required": ["sku"]
        }
    },
    {
        "name": "create_purchase_order",
        "description": "Create a PO with a supplier",
        "input_schema": {
            "type": "object",
            "properties": {
                "supplier_id": {"type": "string"},
                "sku": {"type": "string"},
                "quantity": {"type": "integer"}
            },
            "required": ["supplier_id", "sku", "quantity"]
        }
    }
]

def run_inventory_agent(goal: str) -> str:
    messages = [{"role": "user", "content": goal}]

    for _ in range(6):
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1500,
            tools=tools,
            messages=messages
        )

        if response.stop_reason == "end_turn":
            return next(b.text for b in response.content if hasattr(b, 'text'))

        messages.append({"role": "assistant", "content": response.content})
        tool_results = []

        for block in response.content:
            if block.type == "tool_use":
                result = execute_inventory_tool(block.name, block.input)
                tool_results.append({
                    "type": "tool_result",
                    "tool_use_id": block.id,
                    "content": result
                })

        messages.append({"role": "user", "content": tool_results})

    return "Reached max iterations."

run_inventory_agent(
    "Check stock for SKU-4471. If below 50 units, "
    "create a PO with our primary supplier for 200 units."
)

No documents involved. The agent checks inventory, reasons about the threshold, and conditionally creates a purchase order. This is pure action orchestration.

Path 3: The Hybrid Agent Using RAG as a Tool

This is where most real enterprise systems actually land: an agent that needs to take actions, and one of the things it needs to do along the way is look something up in a document knowledge base.

import anthropic

client = anthropic.Anthropic()

def rag_lookup(query: str) -> str:
    """RAG retrieval wrapped as a tool the agent can call."""
    result = qa_chain({"query": query})  # the RAG chain from Path 1
    return json.dumps({
        "answer": result["result"],
        "sources": [doc.metadata.get("source") for doc in result["source_documents"]]
    })

tools = [
    {
        "name": "search_policy_documents",
        "description": "Search company policy documents for relevant information",
        "input_schema": {
            "type": "object",
            "properties": {"query": {"type": "string"}},
            "required": ["query"]
        }
    },
    {
        "name": "issue_refund",
        "description": "Process a refund for a customer order",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "amount": {"type": "number"}
            },
            "required": ["order_id", "amount"]
        }
    }
]

def execute_tool(name: str, input_data: dict) -> str:
    if name == "search_policy_documents":
        return rag_lookup(input_data["query"])
    elif name == "issue_refund":
        return process_refund(input_data["order_id"], input_data["amount"])

def run_refund_agent(customer_request: str) -> str:
    messages = [{"role": "user", "content": customer_request}]

    for _ in range(6):
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1500,
            tools=tools,
            messages=messages
        )

        if response.stop_reason == "end_turn":
            return next(b.text for b in response.content if hasattr(b, 'text'))

        messages.append({"role": "assistant", "content": response.content})
        tool_results = [
            {"type": "tool_result", "tool_use_id": block.id,
             "content": execute_tool(block.name, block.input)}
            for block in response.content if block.type == "tool_use"
        ]
        messages.append({"role": "user", "content": tool_results})

    return "Reached max iterations."

run_refund_agent(
    "Customer wants a refund on order #8821 for $340. "
    "Check our refund policy first to see if this qualifies."
)

The agent decides to call search_policy_documents to check eligibility before deciding whether to call issue_refund. The RAG system is doing exactly what it's good at, grounded retrieval, but it's a tool in service of the agent's broader decision-making, not the entire system.

The Cost and Complexity Reality

RAG-only systems are cheaper to build and run. Single retrieval call, single generation call, predictable latency, easier to evaluate (you can measure retrieval precision and answer accuracy independently).

Agentic systems are more expensive and harder to debug. Multiple LLM calls per task, unpredictable latency (depends how many iterations the agent takes), harder to evaluate because failure can happen at the planning stage or the execution stage. They're also the only option when the task genuinely requires multi-step action across systems.

The mistake we see most often: teams building agentic infrastructure for what's fundamentally a question-answering problem, paying the complexity cost for capability they don't need.

The full RAG vs agentic AI comparison covers the cost modelling, latency benchmarks, and evaluation methodology differences in more depth.

What Comes After This Decision

Once you've picked your architecture, the next question is build vs buy, do you build this RAG pipeline or agent loop yourself, or do you use a managed platform? The answer depends on your timeline, your team's capacity, and how differentiated your specific use case actually is. We wrote the framework with cost models, time estimates, and decision criteria for exactly this question, worth reading before you commit engineering time to either path.

Published by Dextra Labs | AI Consulting & Enterprise Agent Development