DEV Community: Dextra Labs

Building a Customer Service AI Agent That Executes Workflows (Not Just Answers Questions) - Full Architecture

Dextra Labs — Sun, 16 Aug 2026 21:59:41 +0000

Most customer service AI implementations answer questions. They retrieve relevant information from a knowledge base, synthesize a response, and hand the conversation back to the customer.

That's a chatbot. A good one, in 2026, but still a chatbot.

A customer service agent that executes workflows does something different. It processes the refund. It updates the account. It triggers the return label. It does what the customer asked, inside the systems that matter, and sends confirmation when it's done. The difference isn't the model, it's the architecture around the model, specifically the tool-use layer and how everything connecting to it is designed.

This is the architecture we use for production customer service agents. It's the part that most tutorials skip.

The Core Architecture Shift

A Q&A agent has one primary operation: retrieve context, generate response. The loop is simple.

A workflow-execution agent has three: classify intent, execute tools, generate response. The middle step is where production complexity lives.

Here's the agent loop that governs everything:

from anthropic import AsyncAnthropic
from typing import Optional
import asyncio

client = AsyncAnthropic()

async def agent_loop(conversation: Conversation) -> AgentResponse:
    # Step 1: Classify customer intent
    intent = await classify_intent(
        message=conversation.latest_message,
        history=conversation.history
    )

    # Step 2: Retrieve customer context from backend systems
    context = await retrieve_context(
        customer_id=conversation.customer_id,
        intent=intent
    )

    # Step 3: Plan actions based on intent + context
    action_plan = await plan_actions(
        intent=intent,
        context=context,
        policy=load_policy(intent.type)
    )

    # Step 4: Execute tools if the intent requires action
    tool_results = {}
    if action_plan.requires_tools:
        tool_results = await execute_tools(action_plan.tools)

        # Check escalation conditions before proceeding
        if should_escalate(tool_results, intent, context):
            return await escalate_to_human(
                conversation=conversation,
                context=context,
                tool_results=tool_results,
                reason=determine_escalation_reason(intent, tool_results)
            )

    # Step 5: Generate grounded response from results
    response = await generate_response(
        intent=intent,
        context=context,
        tool_results=tool_results
    )

    # Step 6: Persist updated conversation state
    await persist_context(conversation, response, tool_results)

    return response

The key difference from a retrieval-only agent: step 4 executes real operations against real systems. The agent isn't describing what should happen. It's making it happen.

Intent Classification

Before any tool call happens, the agent needs to know what category of request it's dealing with. Intent classification determines which tools get considered and which policy rules apply.

INTENT_CATEGORIES = [
    "order_status",
    "return_request",
    "refund_request",
    "account_update",
    "billing_dispute",
    "product_question",
    "complaint",
    "explicit_escalation"
]

async def classify_intent(message: str, history: list) -> Intent:
    response = await client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=256,
        system="""Classify the customer message into exactly one intent category.
        Return JSON: {"type": <category>, "confidence": <0-1>, "entities": {}}
        Categories: order_status, return_request, refund_request, account_update,
        billing_dispute, product_question, complaint, explicit_escalation""",
        messages=[
            {"role": "user", "content": f"History: {history[-3:]}\nMessage: {message}"}
        ]
    )
    return Intent.from_json(response.content[0].text)

Confidence scoring here matters beyond routing. When the intent classification confidence falls below 0.70, that's a signal for tighter tool permission scoping and a lower escalation threshold, the agent is operating with more uncertainty about what the customer actually needs.

The Tool-Use Layer

This is where the architecture diverges from a retrieval-only design. Tools are the interface between the agent and your backend systems, CRM, order management, payment processor, helpdesk.

CUSTOMER_SERVICE_TOOLS = [
    {
        "name": "lookup_order",
        "description": "Retrieve current order status, tracking, and line items for a customer order",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "customer_id": {"type": "string"}
            },
            "required": ["order_id", "customer_id"]
        }
    },
    {
        "name": "process_refund",
        "description": "Initiate a refund for an eligible order within policy parameters",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "refund_amount": {"type": "number"},
                "reason": {"type": "string"},
                "policy_check_passed": {"type": "boolean"}
            },
            "required": ["order_id", "refund_amount", "reason", "policy_check_passed"]
        }
    },
    {
        "name": "create_return_label",
        "description": "Generate a prepaid return shipping label and initiate return workflow",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "return_reason": {"type": "string"}
            },
            "required": ["order_id", "return_reason"]
        }
    },
    {
        "name": "update_account_field",
        "description": "Update a customer account detail after identity verification",
        "input_schema": {
            "type": "object",
            "properties": {
                "customer_id": {"type": "string"},
                "field": {"type": "string", "enum": ["email", "address", "phone"]},
                "new_value": {"type": "string"},
                "verified": {"type": "boolean"}
            },
            "required": ["customer_id", "field", "new_value", "verified"]
        }
    }
]

Notice policy_check_passed and verified as required fields in the refund and account update tools. The agent cannot call these tools without explicitly confirming that policy eligibility has been checked and identity has been verified. This is enforcement at the tool signature level, not at the prompt level, a much harder constraint to bypass.

Context Persistence Across Channels

A customer who starts a conversation in web chat, follows up by email, and then calls voice support should not have to re-explain their situation at each handoff. Context persistence is what makes multi-channel support feel like a single conversation rather than three separate ones.

from dataclasses import dataclass, field
from typing import Dict, List, Optional, Any

@dataclass
class ConversationContext:
    customer_id: str
    conversation_id: str
    channel: str  # "web_chat" | "email" | "voice" | "whatsapp"

    # Verified information (survives channel switches)
    verified_identity: bool = False
    verified_order_id: Optional[str] = None
    verified_fields: Dict[str, Any] = field(default_factory=dict)

    # Resolution tracking
    attempted_resolutions: List[Dict] = field(default_factory=list)
    current_intent: Optional[str] = None

    # Escalation state
    escalation_reason: Optional[str] = None
    escalation_priority: str = "standard"  # "standard" | "high" | "critical"

    # Cross-channel history
    prior_conversations: List[str] = field(default_factory=list)
    unresolved_issues: List[str] = field(default_factory=list)

async def load_or_create_context(customer_id: str, channel: str) -> ConversationContext:
    # Check for existing unresolved context across channels
    existing = await redis_client.get(f"context:{customer_id}:active")

    if existing:
        context = ConversationContext.from_json(existing)
        context.channel = channel  # Update current channel
        return context

    # Load customer history to pre-populate context
    customer = await crm.get_customer(customer_id)
    return ConversationContext(
        customer_id=customer_id,
        conversation_id=generate_id(),
        channel=channel,
        prior_conversations=customer.recent_conversation_ids,
        unresolved_issues=customer.open_tickets
    )

The verified_fields dictionary matters specifically. When a customer verifies their identity in the web chat session, that verification persists into the voice session. The agent in the new channel knows what's already been confirmed and doesn't ask the customer to reverify.

Escalation Routing

Escalation logic is where production quality separates from demo quality. An agent that escalates incorrectly frustrates customers. An agent that fails to escalate when it should creates liability.

LEGAL_KEYWORDS = [
    "attorney", "lawyer", "lawsuit", "legal action",
    "sue", "court", "fraud", "chargeback dispute"
]

SECURITY_KEYWORDS = [
    "hacked", "unauthorized", "data breach",
    "identity theft", "fraud", "compromised"
]

def should_escalate(
    tool_results: Dict,
    intent: Intent,
    context: ConversationContext
) -> bool:

    # Hard rules, always escalate regardless of confidence
    if intent.type == "explicit_escalation":
        return True

    if contains_any(context.latest_message, LEGAL_KEYWORDS):
        return True

    if contains_any(context.latest_message, SECURITY_KEYWORDS):
        return True

    # VIP / high-value customer handling
    if context.customer_tier == "enterprise":
        if len(context.attempted_resolutions) >= 2:
            return True

    # Confidence-based escalation
    if intent.confidence < 0.65:
        return True

    if tool_results.get("resolution_confidence", 1.0) < 0.70:
        return True

    # Time and turn limits
    if context.turn_count > 8:
        return True

    if context.elapsed_seconds > 600:
        return True

    return False

The tiered escalation threshold for enterprise customers (>= 2 attempts vs the standard > 8 turns) reflects a business decision that high-value customers get faster human access. That policy lives in code, not in a prompt, which means it's enforceable and auditable.

Human Handoff Design

The handoff payload is the most underbuilt piece of most agent architectures. If the human agent who picks up the escalation has to re-read the transcript and reconstruct context from scratch, the escalation experience is worse than if the customer had just called a human from the start.

async def escalate_to_human(
    conversation: Conversation,
    context: ConversationContext,
    tool_results: Dict,
    reason: str
) -> AgentResponse:

    # Generate AI summary of conversation for the human agent
    summary = await generate_handoff_summary(conversation, context, tool_results)

    handoff_payload = {
        "conversation_id": conversation.id,
        "customer": {
            "id": context.customer_id,
            "name": context.customer_name,
            "tier": context.customer_tier,
            "lifetime_value": context.customer_ltv,
            "sentiment": context.sentiment_score
        },
        "summary": summary,
        "escalation_reason": reason,
        "escalation_priority": context.escalation_priority,
        "attempted_resolutions": context.attempted_resolutions,
        "verified_context": context.verified_fields,
        "recommended_action": await suggest_next_action(context, tool_results),
        "full_transcript": conversation.messages,
        "open_tickets": context.unresolved_issues
    }

    # Route to appropriate queue based on priority and type
    queue = determine_queue(reason, context.customer_tier)
    ticket_id = await helpdesk.create_escalation(queue, handoff_payload)

    # Inform the customer honest about what's happening
    return AgentResponse(
        message=f"I'm connecting you with a specialist who can help with this. "
                f"They'll have the full context of our conversation "
                f"you won't need to repeat anything. Reference: {ticket_id}",
        action="escalate",
        ticket_id=ticket_id
    )

The recommended_action field in the payload is the piece that changes average handle time most significantly. When the human agent opens the case, they get a structured recommendation, not a transcript to reconstruct. The AI hasn't transferred the problem, it has transferred a decision-ready package.

What This Architecture Doesn't Solve

Building this correctly takes the agent from Q&A to workflow execution. What it doesn't solve is the integration layer beneath the tools, getting process_refund() to connect reliably to your actual payment processor, with your actual permission model, handling your actual error states.

The tool signatures above are intentionally clean. The implementations behind them are where the production complexity lives: rate limiting, retry logic, circuit breakers for when downstream APIs fail mid-conversation, audit logging for every write operation.

That gap between a tool defined and a tool that works under production load is where most enterprise agent projects either invest properly or discover they should have.

The chat is the easy part. The workflow execution and backend integration is where production agents are made or broken. We wrote the full build guide covering the complete eight-layer architecture, reliability patterns, cost modeling, and the failure modes that break agents at scale.

How to Build a 24/7 AI Customer Service Agent, Enterprise Guide

Dextra Labs builds production AI agent systems for enterprise customer service, finance, and operations. If your agent architecture is at the integration layer and you want a technical review, hello@dextralabs.com

How to Cut Claude Code Token Costs by 60% Without Losing Context (Practical Guide)

Dextra Labs — Thu, 13 Aug 2026 04:44:42 +0000

We were running Claude Code across a mid-sized TypeScript monorepo. Twelve engineers, active development across five services. The token bills were climbing fast, not because the team was being careless, but because nobody had thought carefully about context hygiene.

After spending two weeks systematically measuring and fixing token consumption, we cut the daily spend by 62% without any degradation in output quality. Most of the savings came from four things. This post covers all of them with before/after numbers.

What Actually Burns Your Tokens

Before fixing the problem, you need to understand where the tokens go.

Every Claude Code session has a context window, up to 200K tokens for Claude Sonnet models. But the cost isn't about the maximum. It's about what you're loading into every interaction.

Three things inflate context silently:

Your CLAUDE.md

It's loaded at the start of every session. If it's verbose, every interaction pays for that verbosity upfront.

File references

Every file you add to context stays in context. Loading broad directories when you need specific files is expensive fast.

Conversation history

Long sessions accumulate. By turn 30 of a debugging session, you may be paying for thousands of tokens of context that are no longer relevant to the current question.

The fix for each is specific. Let's go through them.

Fix 1: CLAUDE.md Surgery

Most CLAUDE.md files we've audited have the same problem, they were written incrementally, with something new added every time someone thought of a useful constraint. After a few months, they read like a junior developer trying to explain the entire codebase from scratch at the start of every session.

This is the pattern we commonly see:

# Project Overview
This is an e-commerce platform built in 2019 using Next.js, TypeScript, 
and PostgreSQL. We use React for the frontend, Express for the backend, 
Redis for caching, Stripe for payments, SendGrid for email, AWS S3 for 
file storage, Algolia for product search, and we follow a microservices 
architecture. Our database has 45 tables including users, products, orders, 
inventory, shipping, payments, reviews, categories, tags, suppliers...
[continues for another 2,000 tokens]

The problem: 90% of that context is irrelevant to the task in front of Claude at any given moment.

The fix is to strip CLAUDE.md down to what Claude actually uses in every interaction, the hard constraints that apply across all tasks and move the rest to reference files that get loaded on demand.

# Stack: Next.js 14 / TypeScript / PostgreSQL 16
# Monorepo: packages/web, packages/api, services/

## Active constraints
- Repository pattern: no direct DB calls from API routes
- Stripe SDK v13 only, no direct API calls
- All types in /shared/types, no inline interface declarations
- Test coverage required for services/ changes

## Load when needed
- Architecture: /docs/architecture.md
- API contracts: /docs/api-contracts.md
- DB schema: /docs/schema.md

Before: 3,200 tokens per session startup. After: 380 tokens per session startup.

That difference compounds across every single interaction in a session. On a typical 20-turn session, we were saving 55,000+ tokens just from the startup context reduction.

Fix 2: Selective File Loading

The second major source of token waste is file loading habits. Developers tend to add broad context out of habit, "load the entire services directory so Claude has the full picture", when most tasks only need two or three files.

The cost difference is not marginal. Loading a directory with 40 TypeScript files at an average of 200 lines each adds roughly 80,000 tokens to your context before you've written a single message. Loading the three files relevant to your task adds 6,000 tokens.

What we did instead:

Instead of loading broad directories, we started specifying exactly which files were needed for each task.

# Before: loading the entire payment service
@services/payment/

# After: loading only what's needed to fix the Stripe webhook handler
@services/payment/webhooks/stripe.ts
@services/payment/types/stripe.types.ts
@tests/payment/webhooks/stripe.test.ts

For tasks that genuinely require broader context, refactoring across services, finding patterns across the codebase, we created task-specific context bundles that could be loaded together.

# Context bundle for auth-related work
@services/auth/
@packages/web/src/lib/auth.ts
@packages/api/middleware/auth.ts
@docs/auth-flow.md

The bundle approach gives you the broad context when you actually need it, without loading it for tasks that don't.

Before: Average 45,000 tokens of file context per session. After: Average 11,000 tokens of file context per session.

Fix 3: Structure Your Prompts for Cache Hits

This one applies when you're using the Claude API directly alongside Claude Code, which most teams building AI-assisted workflows end up doing.

Claude's API supports prompt caching. Cached prompt portions cost roughly 10% of the standard input price. The catch: only content that appears consistently at the start of your prompt gets cached reliably.

The structural change that makes this work is separating stable context from dynamic context explicitly:

import anthropic

client = anthropic.Anthropic()

STABLE_SYSTEM_CONTEXT = """
You are a TypeScript engineer working on an e-commerce platform.
Stack: Next.js 14, PostgreSQL, Stripe v13, repository pattern.
Constraints: [your core constraints here]
"""

def get_completion(task_description, relevant_files):
    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=4096,
        system=[
            {
                "type": "text",
                "text": STABLE_SYSTEM_CONTEXT,
                "cache_control": {"type": "ephemeral"}
            },
            {
                "type": "text",
                "text": f"Current task context:\n{relevant_files}"
                # No cache_control here, this changes per request
            }
        ],
        messages=[{"role": "user", "content": task_description}]
    )
    return response

The stable system context gets cached after the first call. Every subsequent call that hits that cache costs 10 cents per million tokens instead of the standard $3. For teams running hundreds of API calls per day against the same system context, this alone reduces the API cost significantly.

Cache hits compound with conversation length. A 20-turn session where the first 5,000 tokens are consistently cached looks very different on a cost dashboard than the same session without caching.

Fix 4: Session Management and When to Use /compact

Long sessions are expensive for a non-obvious reason: the entire conversation history stays in context for every new message. By turn 40, you're paying for everything from turn 1 forward, including the debugging rabbit holes you abandoned at turn 8 and the context that became irrelevant when you solved the first problem and moved to the second.

Two strategies here.

Compacting long sessions

Claude Code's /compact command summarizes the conversation history into a condensed representation. Run it when:

You're 20+ turns into a session and switching to a different task
You've solved one problem and are starting a new one in the same session
The conversation has accumulated significant exploratory context that's no longer relevant

The tradeoff is real: compacted history loses some nuance. Don't compact in the middle of a complex debugging session where earlier turns contain important error context. Do compact when you're pivoting to a new problem where the old context is just overhead.

Starting fresh sessions strategically

The common instinct is to keep sessions alive for continuity. The better approach is to keep session-specific notes in a scratch file and start fresh for distinct tasks.

# session-notes.md (gitignored)
## Task: Fix Stripe webhook signature validation
Status: Solved, issue was in middleware ordering, not validation logic
Relevant files: services/payment/webhooks/stripe.ts, middleware/raw-body.ts
Solution: raw-body middleware must run before body-parser

A fresh session loaded with three relevant files and a one-paragraph task description in your CLAUDE.md is almost always cheaper than continuing a 30-turn session carrying all the context of the previous problem.

The Before/After Numbers

Here's what the full optimization set looked like across our client's engineering team over 30 days.

The output quality metric is the one that matters most. These weren't cuts that traded quality for cost, they were cuts that removed context that wasn't contributing to quality in the first place. Verbosity in your system prompt doesn't make Claude smarter about your codebase. Relevant, targeted context does.

The biggest single win was CLAUDE.md restructuring. It required one afternoon of cleanup and delivered persistent savings across every session from that point forward.

These techniques cut our token spend without degrading output. Context hygiene is not a one-time fix, it's a discipline. Once you build it into how your team works with Claude Code, the savings compound every day.

Full guide with measured before/after, the prompt caching implementation patterns, and the CLAUDE.md template we use across client deployments:

Claude Code Token Optimization - Full Guide

Dextra Labs helps engineering teams deploy and optimize Claude Code and enterprise LLM systems. If your token spend is climbing faster than your output value, it's usually a context architecture problem. hello@dextralabs.com

The AI Coding Tool Stack We Actually Use at a 15-Person Series B Startup in 2026 (With Cost Breakdown)

Dextra Labs — Mon, 27 Jul 2026 05:13:26 +0000

This isn't a "top 10 AI coding tools" listicle. This is the specific tools we use, on specific tasks, at a specific team size, with exact monthly costs. The kind of post we wished existed when we were building our own stack.

The context: Dextra Labs runs a 15-person engineering team. We build AI systems for enterprise clients, LLM deployments, agent architectures, RAG pipelines, and technical due diligence assessments. Our codebase is primarily Python (backend services, ML pipelines) and TypeScript (client-facing applications). We deploy to AWS. We ship to production multiple times daily.

We've tried most of the AI coding tools on the market over the past eighteen months. Some stuck. Some didn't. Here's what we actually use today, why we use each one, and what we tried and dropped.

The daily driver: Cursor Pro

Cost: $20/month per developer × 15 = $300/month

Every developer on the team uses Cursor as their primary IDE. The tab autocomplete alone justifies the cost, it's noticeably faster and more accurate than GitHub Copilot's completions, and it understands our codebase patterns well enough that the completions frequently produce multi-line suggestions that are exactly what we'd type.

Composer agent mode handles our most common development tasks: implementing features from specifications, writing boilerplate, and making consistent changes across multiple files. For a typical feature implementation, "add this endpoint with these fields and this validation", Composer produces a mergeable first draft in under 10 minutes.

What Cursor doesn't do well for us: complex refactors across many files, debugging subtle concurrency issues, and any task where understanding the entire codebase context matters more than quick inline edits. For those tasks, we reach for something else.

The heavy lifter: Claude Code on Max

Cost: $100/month × 3 senior engineers = $300/month

Three of our senior engineers have Claude Code Max subscriptions. They use it for the work that Cursor can't handle: large-scale refactors, architectural analysis, complex debugging sessions, comprehensive test suite generation, and code review preparation.

The 1M token context window is the capability that justifies the price. When we're refactoring a service that touches twelve modules, Claude Code can hold all twelve in context simultaneously and produce changes that are internally consistent across the entire scope. Cursor's context is limited by what fits in the IDE's context window, which means large refactors require breaking the work into smaller scoped pieces.

We also use Claude Code for pre-PR code analysis on critical modules, running it against a branch before submitting the PR to catch issues before human review. This has measurably reduced review cycle time because the PR arrives with obvious issues already addressed.

Not every engineer needs Claude Code. For mid-level developers working on well-scoped tasks within established patterns, Cursor handles everything they need. Claude Code's value scales with task complexity and codebase scope, it's most valuable for the engineers doing the most complex work.

The review layer: CodeRabbit

Cost: $15/month per developer × 15 = $225/month

CodeRabbit runs on every PR automatically via our GitHub integration. It provides AI-powered code review comments that catch common issues before human reviewers see the PR.

We chose CodeRabbit over building our own review pipeline because the maintenance burden of a custom pipeline wasn't justified at our team size. CodeRabbit's detection rate isn't perfect, in our internal testing it catches roughly 60% of the bug categories we care about, but it consistently catches the mechanical issues (missing error handling, potential null references, inconsistent naming) that human reviewers would otherwise spend time on.

The false positive rate was initially too high. We spent two weeks tuning the configuration, adding our project-specific rules, and excluding certain file patterns that generated noise. After tuning, the false positive rate dropped to a level the team finds acceptable, roughly one in five comments is irrelevant, which isn't great but is liveable.

The testing companion: Qodo (formerly CodiumAI)

Cost: Free tier for individual use, team features at $19/month × 5 = $95/month

Five engineers use Qodo's paid tier for test generation. The free tier handles basic test generation well enough for simple modules. The paid tier adds better context awareness and the ability to generate tests that respect our specific testing patterns and fixture setup.

We use Qodo specifically for generating initial test scaffolding, the boilerplate test structure, the basic happy-path tests, the error-path tests. Then we review and enhance by hand, adding the edge cases and business-logic tests that the generator doesn't have context for.

The honest assessment: Qodo saves about 40% of the time we'd spend writing tests from scratch. The generated tests need review and frequently need modification, but starting from a generated scaffold is faster than starting from an empty file.

What we tried and dropped

GitHub Copilot was our starting point eighteen months ago. We switched to Cursor when Cursor's autocomplete quality surpassed Copilot's and the Composer agent mode added a capability Copilot didn't offer. We kept one Copilot Business license for three months as a fallback before canceling entirely.

Windsurf (now Devin Desktop) we trialed for two months. Cascade's contextual awareness was impressive but the ownership changes, OpenAI deal, Google founders departure, Cognition acquisition, made us uncomfortable about building workflows around it. The product is good. The corporate stability isn't there yet for us to commit.

Amazon CodeGuru we tested for code review. The detection quality was decent but the AWS-specific integration requirements didn't fit our workflow, and the pricing at scale was higher than CodeRabbit for comparable capability.

A custom Claude API review pipeline we built ourselves and ran for three months. Detection quality was the best we tested, higher than any commercial tool. But maintaining the prompts, keeping the review rules current, managing the API costs, and debugging edge cases consumed about 8 hours per week of senior engineering time. At 15 engineers, the maintenance burden exceeded the marginal quality improvement over CodeRabbit. We shut it down, though we'd reconsider at 50+ engineers where the per-developer maintenance cost amortises better.

The monthly cost breakdown

$61 per developer per month for the full AI-assisted development stack. Against a fully-loaded engineering cost of roughly $12,000-$15,000 per developer per month in our market, the tooling cost is trivially small relative to even modest productivity gains.

We estimate the stack saves each developer 6-8 hours per week, primarily through faster feature implementation (Cursor autocomplete and Composer), reduced review overhead (CodeRabbit handling mechanical review), faster test writing (Qodo scaffolding), and eliminated debugging dead ends (Claude Code's architectural analysis).

At conservative estimates, 6 hours saved per developer per week × 15 developers × 4 weeks, that's 360 engineering hours per month recaptured. At our loaded cost rate, that's approximately $30,000-$40,000 per month in recaptured engineering capacity against $920 in tooling cost.

The ROI math isn't close.

What would change at 50 or 200 engineers

At 15 engineers, individual tool subscriptions work fine. The total cost is manageable. The configuration is per-developer. Nobody needs centralised management.

At 50+ engineers, the calculus changes. You need SSO integration so developers don't manage individual credentials. You need audit logging for compliance, who used which tool, what code was processed, what suggestions were accepted. You need centralised cost management so the CFO can see one line item instead of seventy-five subscriptions. You need usage analytics to understand whether the tools are actually being used effectively across the team.

At 200+ engineers, you're in enterprise procurement territory. Data residency matters, where does the code go when it's processed? Model selection governance matters, which models are approved for use on which codebases? Custom model training on your proprietary codebase becomes viable and valuable at this scale.

This stack works for a 15-person team. At 200+ engineers, the calculus changes, you need SSO, audit logs, and centralised cost management. We wrote the enterprise breakdown for teams at that scale because the evaluation criteria and the procurement process are fundamentally different from picking tools for a small team.

Published by Dextra Labs, AI Consulting and Enterprise Agent Development

Devin AI vs Claude Code vs Cursor Agent: Which One Actually Ships Production Code? (Benchmarks Inside)

Dextra Labs — Mon, 27 Jul 2026 05:09:11 +0000

"It writes code" is no longer impressive. The question in 2026 is whether it writes code you'd actually merge into a production codebase without spending more time reviewing and fixing than you'd spend writing it yourself.

We gave three autonomous coding agents the same production task and evaluated the output on the only metric that matters: would we ship this?

The task: add rate limiting to an existing Express.js API with Redis backing, configurable per-endpoint limits, proper HTTP response headers, graceful degradation when Redis is unavailable, and comprehensive tests.
This isn't a toy benchmark. It's a feature request that lands in a real team's sprint every quarter.

The agents: Devin (via Devin Desktop, Cognition's rebranded Windsurf platform at $20/month), Claude Code (on Max plan at $100/month with Opus 4.8), and Cursor Agent mode (Pro at $20/month with Composer 2.5).

Same repo. Same requirements document. Same evaluation rubric. No hand-holding during execution, we gave each agent the task and let it work.

The task specification

We provided each agent with identical requirements:

Add rate limiting middleware to the Express API.
Requirements:
- Redis-backed sliding window algorithm
- Configurable limits per endpoint via route decorator
- Standard HTTP headers: X-RateLimit-Limit, 
  X-RateLimit-Remaining, X-RateLimit-Reset, Retry-After
- Graceful fallback when Redis is unavailable 
  (allow requests with warning log, don't block)
- Tests covering: normal operation, rate exceeded, 
  Redis failure, concurrent requests, config validation
- Follow existing project patterns for middleware, 
  error handling, and test structure

The existing codebase had established patterns for middleware registration, error response formatting, and test organisation. We wanted to see which agents respected those patterns versus imposing their own conventions.

Devin's output

Devin worked autonomously for approximately 22 minutes. It explored the codebase, identified the existing middleware patterns, created the implementation files, and ran the test suite to verify its work.

The implementation used a sliding window algorithm with Redis sorted sets, a solid technical choice. The per-endpoint configuration used a decorator pattern that was consistent with the codebase's existing middleware approach.

What worked well: the Redis interaction code was clean, the sorted set implementation was correct, and the HTTP headers were properly formatted. The decorator pattern matched the existing codebase conventions. Devin clearly analysed the existing middleware structure before writing its own.

What didn't ship: the graceful degradation on Redis failure had a critical issue. When Redis was unavailable, the middleware threw an unhandled promise rejection instead of falling through to the allow-with-warning behaviour the requirements specified. The error handling path was partially implemented, there was a try-catch, but the catch block called a logging function that itself tried to write to Redis, creating a recursive failure.

The tests covered the happy path and the rate-exceeded path well. The Redis failure test existed but tested the wrong behaviour, it asserted that requests were blocked when Redis was down, contradicting the requirement for graceful degradation. Two tests had hardcoded timing assertions that would produce flaky failures on slower CI runners.

Code quality score: 3.5/5. Time-to-PR-ready: 45 minutes after fixing the Redis fallback, updating the test assertions, and removing the timing dependencies.

Claude Code's output

Claude Code completed the task in approximately 18 minutes. It loaded the full project context, asked no clarifying questions (the specification was detailed enough), and produced the implementation in a single pass.

The sliding window implementation used Redis MULTI/EXEC transactions for atomicity, a more robust approach than Devin's sorted set method for high-concurrency scenarios. The per-endpoint configuration used the existing decorator pattern.

What worked well: the Redis failure handling was the best of the three. A circuit breaker pattern, after three consecutive Redis failures, the middleware switches to allow-all mode for 30 seconds before retrying. This is more sophisticated than the requirements asked for, and it's the correct production pattern because it prevents cascading retry storms when Redis is genuinely down. The tests included a concurrent request test using Promise.all that actually exercised the race condition path.

What needed review: the circuit breaker added complexity that the requirements didn't specify. We debated whether to keep it. We kept it, it's the right engineering decision even though it wasn't requested. One test used a mock Redis client that didn't perfectly replicate the real client's error throwing behaviour, causing a false positive on the error path test.

Code quality score: 4.5/5. Time-to-PR-ready: 15 minutes, fixing the mock client issue and adding a comment explaining the circuit breaker design decision.

Cursor Agent's output

Cursor's Composer agent mode completed the task in approximately 14 minutes, the fastest of the three. The inline IDE experience meant we could watch the implementation happen in real time and the iteration cycle was the most fluid.

The sliding window used a Redis INCR with TTL approach, simpler than the other two implementations, which is both a strength (less code, easier to understand) and a limitation (less precise window boundaries at high concurrency).

What worked well: the code was the most readable of the three implementations. Clear variable names, good comments, consistent with the existing codebase style. The test structure followed the project's existing patterns exactly. The HTTP headers were correctly implemented.

What didn't ship: the rate limiting algorithm had a subtle issue at window boundaries. Because it used INCR with a fixed TTL rather than a true sliding window, a burst of requests at the end of one window and the beginning of the next could exceed the intended limit. At low traffic this doesn't matter. At high traffic with strict rate requirements, it's a bug.

The Redis failure handling worked but was simplistic, a try-catch that logged a warning and allowed the request through. No circuit breaker, no backoff, no tracking of Redis health. Functional but not production-hardened.

The tests were well-structured and covered the main paths but didn't include a concurrent request test. The boundary condition issue wouldn't have been caught by the generated tests.

Code quality score: 3.5/5. Time-to-PR-ready: 35 minutes after replacing the INCR approach with a proper sliding window and adding a concurrent test.

The comparison matrix

What this actually tells us

Claude Code produced the only output we would have merged with minimal changes. The circuit breaker pattern, the atomic transactions, and the concurrent test demonstrated understanding of production concerns that the other two agents didn't exhibit.

Cursor was fastest to initial output but the window boundary issue meant more human review time. The code was the most readable and the most consistent with existing style, which matters for long-term maintainability.

Devin's recursive Redis failure is the kind of bug that would have caused a production incident if it shipped. It's also the kind of bug that's hard to catch in review because the error handling code looks correct at first glance, you have to trace the execution path through the logging function to see that it re-enters the failing Redis connection.

The pattern across all three: the "happy path" implementation is solid from every agent. The differentiation is in error handling, edge cases, and production hardening. These are the dimensions where engineering experience matters most, and they're the dimensions where the gap between agents is widest.

Autonomous coding agents are advancing fast, but "shipping production code" is a much higher bar than passing SWE-bench. The benchmarks test whether the agent can solve a defined problem. Production readiness tests whether it handles the problems that aren't defined, the edge cases, the failure modes, the concurrent access patterns that the requirements didn't specify because experienced engineers handle them implicitly.

We wrote the full CTO comparison with ten tools for teams evaluating autonomous coding agents at enterprise scale, covering not just output quality but the procurement criteria that matter for teams of 20-200 engineers.

Published by Dextra Labs, AI Consulting and Enterprise Agent Development

AI Code Review Tools Are Missing 40% of Real Bugs, Here's What We Found After Testing 8 of Them

Dextra Labs — Tue, 21 Jul 2026 05:11:21 +0000

We planted 23 known bugs in a production-realistic codebase and ran eight AI code review tools against it.

The best tool caught 14 of them. The worst caught 7. The average detection rate across all eight tools was 60.3%. Which means the average AI code review tool missed roughly 40% of real, exploitable, production-impacting bugs that a human reviewer would be expected to catch.

Before the pitchforks come out, this isn't an indictment of AI code review. These tools catch genuine issues, improve code quality, and save reviewer time. But the marketing claims of "catches bugs before they ship" need calibration against what they actually catch and what they miss.

We published this because nobody else is testing these tools on realistic bugs. The existing comparisons test detection of toy examples or run tools against their own cherry-picked test cases. We wanted to know: if you're a senior engineer who would catch a race condition in review, does the AI tool catch it too?

The answer, for 40% of realistic bugs, is no.

The test setup

We created a deliberately buggy version of a mid-sized Python/FastAPI application, approximately 15,000 lines of code across 34 modules. The application handles user authentication, payment processing, order management, and a notification system. It's representative of the kind of B2B SaaS backend that most of these tools are marketed for.

We planted 23 bugs across eight categories. Every bug was a real bug pattern we've encountered in production systems during our code review and technical due diligence work. None were synthetic puzzles. All would have real consequences if deployed.

The categories and planted bugs:

SQL injection vulnerabilities: 3 bugs. One in a search endpoint using string concatenation. One in a reporting query with dynamic column selection. One in an admin tool with parameterised query syntax that was assembled incorrectly.

Authentication and authorisation bypasses: 3 bugs. One IDOR (Insecure Direct Object Reference) where user A could access user B's data by modifying the resource ID. One missing authentication check on an internal API endpoint. One JWT validation that checked expiry but not signature.

Race conditions: 3 bugs. One TOCTOU in payment processing where balance check and deduction weren't atomic. One in a coupon redemption where concurrent requests could redeem beyond the limit. One in a task queue where concurrent workers could pick up the same job.

Memory and resource leaks: 3 bugs. One database connection not returned to pool on exception path. One file handle left open in an error branch. One Redis connection created per request without pooling.

Off-by-one and boundary errors: 3 bugs. One pagination that skipped the last page. One date range that excluded the end date. One array index that would crash on empty input.

Input validation failures: 3 bugs. One path traversal in a file upload endpoint. One XXE vulnerability in XML parsing. One ReDoS-susceptible regex pattern.

Logic errors: 3 bugs. One pricing calculation that applied discount after tax instead of before. One notification that was sent on failed status instead of success. One sorting that reversed the intended order.

Data handling violations: 2 bugs. One endpoint that logged PII in plaintext. One that returned deleted user data in API responses.

The eight tools tested

We ran eight tools that represent the current market spectrum: GitHub Copilot code review, Cursor's built-in review, CodeRabbit, Qodo (formerly CodiumAI), Amazon CodeGuru, Sourcery, DeepSource, and a custom Claude-based review pipeline we built for client work.

Each tool was given the same codebase in the same state. We submitted the buggy code as a PR against the clean baseline and captured every issue each tool flagged.

The detection matrix

What the data reveals

The category breakdown is more interesting than the aggregate numbers.

SQL injection and input validation were the best-detected categories across all tools. Most tools caught the obvious string concatenation injection and the path traversal. These are well-represented in training data and well-documented as vulnerability patterns. If your concern is common OWASP-style vulnerabilities, most AI review tools provide decent coverage.

Race conditions were catastrophically underdetected. Only two tools caught any race conditions at all. The TOCTOU in payment processing, arguably the highest-impact bug in the entire test set, was caught by only two of eight tools. Race conditions require understanding concurrent execution flows, which most AI review tools don't reason about effectively. They analyse the code path sequentially and don't model what happens when two threads execute the same function simultaneously.

This matters because race conditions in payment processing, inventory management, and resource allocation are among the most costly production bugs. They're also among the hardest for human reviewers to catch, which means they're the category where AI assistance would be most valuable and they're the category where AI assistance is currently weakest.

Logic errors were nearly invisible to most tools. The pricing calculation bug, discount applied after tax instead of before, was caught by only three tools. This is because detecting logic errors requires understanding business intent, not just code structure. The code was syntactically correct, followed good patterns, and would pass any static analysis. It just didn't do what the business required. No tool can catch this without understanding what the code is supposed to do, which requires context that most review tools don't have.

Authentication bypasses had moderate detection but the patterns that were caught were the obvious ones. The missing auth check on an endpoint was caught by most tools. The subtle JWT signature validation issue, where the code checked expiry but not signature, was caught by only two. The IDOR was caught by three. The pattern: obvious security omissions get caught, subtle security logic errors get missed.

The false positive problem

Detection rate is only half the story. False positive rate determines whether developers actually use the tool or start ignoring its output.

CodeRabbit had the highest detection rate among commercial tools but also the highest false positive count. Sourcery had the worst ratio, more false positives than true positives. The custom Claude pipeline had the best balance of high detection with low false positives, largely because we tuned the review prompt specifically for this codebase's patterns and explicitly instructed the model to flag only issues with concrete impact.

False positives above 40% are adoption killers. When developers see AI review comments that are wrong or irrelevant more than once in every three comments, they stop reading the comments. The tool becomes noise that the team learns to ignore, which is worse than not having the tool at all because it creates a false sense of security.

What this means for teams using AI code review

AI code review is valuable. It catches real bugs, enforces consistency, and frees human reviewers from mechanical checks. But it is not a replacement for human review on the bug categories that matter most, race conditions, logic errors, subtle security vulnerabilities, and business rule violations.

The effective deployment pattern: AI review handles the first pass. It catches the common vulnerabilities, the style issues, the obvious errors. Human reviewers then focus their attention on the categories AI consistently misses, concurrency, business logic, architectural decisions, and the subtle security patterns that require understanding intent rather than just structure.

Tool selection is only half the battle. The other half is workflow integration, where in your PR cycle does AI review sit, what does it escalate to human reviewers, and how do you prevent the team from treating AI review as sufficient review? The AI code review enterprise guide covers the full workflow design, including the organisational changes that make AI review additive rather than a false safety net.

Published by Dextra Labs, AI Consulting and Enterprise Agent Development

On-Premise AI Code Review: How We Deployed Claude Locally in an Air-Gapped Environment (Setup Guide)

Dextra Labs — Sun, 19 Jul 2026 18:44:52 +0000

The security officer said no.

Not "maybe with some modifications." Not "let us review the data handling policy." No. Cloud-hosted AI code review was not going to happen in their environment. The codebase contained classified material. The network was air-gapped. No data leaves the building. End of conversation.

This was a defence contractor. Their development team was writing code that couldn't touch any external API, any cloud service, any SaaS platform. But they were also writing 200,000+ lines of code per year with a team that was struggling to maintain review quality across twelve engineers, the exact problem that AI code review solves.

The challenge: deploy AI code review capability that runs entirely on-premise, in an air-gapped network, with no external connectivity, while meeting the security and compliance requirements of a classified computing environment.

This is the guide we wish existed when we started that project. It covers hardware requirements, model selection, inference server setup, CI/CD integration and the security architecture that made it pass the compliance review.

Why air-gapped AI code review is different from everything else

Most AI code review guides assume cloud connectivity. The model runs on the provider's infrastructure. The code is sent via API. The review comes back. The security question is about data handling policies, encryption in transit and vendor trust.

In an air-gapped environment, none of that applies. The model runs on hardware you control. The code never leaves your network. There is no vendor to trust because there is no vendor in the loop.

This sounds simpler from a security perspective. In some ways it is. In other ways it's significantly more complex because you're now responsible for the entire stack, model selection, quantisation, inference hardware, deployment, monitoring, updating and maintaining the system that in a cloud deployment is someone else's problem.

Hardware requirements

The hardware question is the first decision and the one that constrains everything else.

For on-premise AI code review, the model needs to be large enough to understand code semantics, identify patterns and generate meaningful review comments. Small models (7B parameters) can handle basic linting and style checking but lack the reasoning capability for substantive code review. Large models (70B+ parameters) provide review quality approaching cloud-hosted frontier models but require significant GPU infrastructure.

Our recommendation for production on-premise code review:

For teams under 25 engineers with moderate review volume: a single server with 2x NVIDIA A100 80GB GPUs running a 70B parameter model at 4-bit quantisation. This handles the inference load for a team producing 15-25 PRs per day with acceptable latency (30-60 seconds per review). Total hardware cost: approximately $30,000-$45,000 for the GPU server, which amortises over three years to roughly $10,000-$15,000 per year, comparable to cloud-hosted review tool licensing for a team of this size.

For teams of 25-75 engineers: two inference servers with 4x A100 80GB GPUs each, behind a load balancer. This handles 50-80 PRs per day with consistent latency. The second server provides redundancy, if one goes down, review capability continues at reduced throughput rather than stopping entirely.

For teams above 75 engineers: a dedicated inference cluster with horizontal scaling. At this scale, the infrastructure management becomes a full-time operations concern and we recommend dedicated MLOps support.
CPU-only inference is technically possible using llama.cpp with heavily quantised models (4-bit or lower) but the latency is prohibitive for production use, 3-5 minutes per review versus 30-60 seconds on GPU. We tested it and abandoned it after the team stopped waiting for reviews and merged without them.

Model selection

In an air-gapped environment, you're limited to open-weight models that can be downloaded, transferred via physical media and deployed locally. The frontier closed models (Claude, GPT) are not available offline.

The models we've tested and can recommend for on-premise code review as of mid-2026:

Llama 3.3 70B provides the best balance of code understanding, review quality and inference efficiency. At 4-bit quantisation (GPTQ or AWQ), it fits on 2x A100 80GB GPUs with room for reasonable batch sizes. The review quality on Python, TypeScript, Java and C++ codebases is strong, it catches logical errors, security vulnerabilities and pattern inconsistencies at a level that meaningfully reduces human review burden.

DeepSeek Coder V3 is specialised for code tasks and provides excellent results on code-specific review. The model is smaller (33B in its most capable variant) and runs on less hardware, making it a good choice for smaller deployments. It outperforms Llama 3.3 70B on code-specific benchmarks but has weaker general reasoning for architectural review comments.

Qwen 2.5 Coder 32B is another strong code-specialised option that balances quality and resource requirements well.

Our recommendation: start with DeepSeek Coder V3 for code-focused review, add Llama 3.3 70B if you need broader architectural analysis and review comments that explain why something is a problem, not just that it is one.

Inference server setup

We use vLLM as the inference server for production deployments. It handles model loading, request queuing, continuous batching and provides an OpenAI-compatible API endpoint that makes integration with existing tools straightforward.

The deployment architecture:

┌─────────────────────────────────────────────┐
│              Air-gapped network             │
│                                             │
│  ┌───────────┐    ┌──────────────────────┐  │
│  │  GitLab   │───▶│  CI/CD Runner        │  │
│  │  Server   │    │  (review pipeline)   │  │
│  └───────────┘    └──────────┬───────────┘  │
│                              │              │
│                              ▼              │
│                   ┌──────────────────────┐  │
│                   │  Load Balancer       │  │
│                   └──────────┬───────────┘  │
│                    ┌─────────┴─────────┐    │
│                    ▼                   ▼    │
│           ┌──────────────┐   ┌──────────────┐
│           │  vLLM Node 1 │   │  vLLM Node 2 │
│           │  2x A100     │   │  2x A100     │
│           │  Llama 70B   │   │  DeepSeek    │
│           └──────────────┘   └──────────────┘
│                                             │
└─────────────────────────────────────────────┘

The dual-node setup serves two purposes: redundancy (either node can handle all traffic if the other is down) and model specialisation (one node runs the general-purpose model for architectural review, the other runs the code-specialised model for detailed code analysis). The review pipeline can route different review types to different models.

Setting up vLLM on each node:

# Install vLLM (transfer wheel files via approved media)
pip install vllm --no-index --find-links /media/packages/

# Start the inference server
python -m vllm.entrypoints.openai.api_server \
  --model /models/llama-3.3-70b-awq/ \
  --quantization awq \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --port 8000 \
  --host 0.0.0.0

The tensor-parallel-size 2 flag splits the model across both GPUs. The max-model-len 32768 sets the maximum context window, large enough to review substantial diffs but not so large that it exhausts GPU memory.
For the containerised deployment:

FROM nvidia/cuda:12.4.0-base-ubuntu22.04
COPY ./packages /packages
RUN pip install --no-index --find-links /packages vllm
COPY ./models /models
EXPOSE 8000
CMD ["python", "-m", "vllm.entrypoints.openai.api_server", \
     "--model", "/models/llama-3.3-70b-awq/", \
     "--quantization", "awq", \
     "--tensor-parallel-size", "2", \
     "--port", "8000"]

CI/CD integration

The review pipeline hooks into your existing CI/CD system. We'll use GitLab CI as the example since it's the most common on-premise Git platform in classified environments, but the pattern adapts to any CI system.

The pipeline stage triggers on every merge request. It extracts the diff, formats the review prompt with project-specific conventions, sends it to the vLLM endpoint, parses the response and posts review comments back to the merge request.

# .gitlab-ci.yml
ai-code-review:
  stage: review
  script:
    - python scripts/ai_review.py
      --diff-ref $CI_MERGE_REQUEST_DIFF_BASE_SHA
      --target-ref $CI_COMMIT_SHA
      --mr-iid $CI_MERGE_REQUEST_IID
      --api-url http://vllm-lb.internal:8000/v1
      --rules-file .review-rules.md
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

The review script:

import requests
import subprocess
import json

def get_diff(base_ref, target_ref):
    result = subprocess.run(
        ["git", "diff", base_ref, target_ref],
        capture_output=True, text=True
    )
    return result.stdout

def request_review(diff, rules, api_url):
    prompt = f"""Review this code diff against these project rules:

{rules}

Diff:
{diff}

Provide review comments as JSON array with fields:
file, line, severity (critical/warning/info), comment
Only flag genuine issues. No style-only comments unless
they violate a specific rule above."""

    response = requests.post(
        f"{api_url}/chat/completions",
        json={
            "model": "llama-3.3-70b-awq",
            "messages": [{"role": "user", "content": prompt}],
            "temperature": 0.1,
            "max_tokens": 4096
        }
    )
    return json.loads(
        response.json()["choices"][0]["message"]["content"]
    )

The temperature: 0.1 is deliberate, we want consistent, deterministic reviews, not creative suggestions. The rules file (.review-rules.md) contains project-specific conventions that the model uses as review criteria, similar to how cloud-hosted review tools use configuration files.

Security considerations for classified environments

The model files themselves need security classification review before deployment. Open-weight models trained on public data don't contain classified information, but the review process requires documentation that the model was trained on public data only and that no classified data has been used for fine-tuning or prompt construction.

The inference server must run in the same security zone as the codebase. No network paths between the inference server and any external network, even through proxy or gateway. The load balancer is internal only.

Audit logging captures every review request and response, the diff submitted, the review comments generated and the pipeline metadata. This audit trail is required for both security compliance and for monitoring the AI review quality over time.

Model updates require the same physical media transfer process as the initial deployment. When a new model version is released, it's downloaded on an unclassified system, transferred via approved media and deployed through the standard change management process. This means model updates happen quarterly at most, not continuously.

Air-gapped deployment is one piece of the puzzle. The full compliance and security framework for on-premise AI code review, covering SOC 2, HIPAA, PCI and government security requirements, is in our complete guide. It covers the policy documentation, access controls and audit procedures that the technical deployment needs to operate within.

Published by Dextra Labs, AI Consulting and Enterprise Agent Development

I Tested Cursor, Claude Code, Windsurf, and Codex on the Same Codebase for 30 Days. Here Are the Results.

Dextra Labs — Thu, 16 Jul 2026 04:39:19 +0000

The comparison posts flooding your feed right now have a problem. They test each tool on different codebases, different tasks, and different complexity levels, then draw conclusions as though they conducted a controlled experiment. They didn't. They tested four different things and called it a comparison.

We decided to do it properly.

Same codebase. Same four tasks. Same evaluation criteria. Thirty days. No sponsorships, no affiliate links, no predetermined conclusion. We run these tools daily on client engagements at Dextra Labs and wanted to know, with actual data, which tool performs best on which type of work.

The codebase: a production Python/FastAPI backend with 47 modules, PostgreSQL, Redis caching, Celery task queues, and roughly 38,000 lines of code. Real-world complexity, not a tutorial project, not a greenfield demo. An existing system with technical debt, inconsistent patterns, and the kind of accumulated decisions that real production codebases carry.

The tools: Cursor Pro ($20/month), Claude Code on Max ($100/month), Windsurf Pro ($20/month), and OpenAI Codex via ChatGPT Plus ($20/month). All tested during June-July 2026 on their latest available models.

The four tasks

We chose tasks that represent the actual work developers do daily, not synthetic benchmarks.

Task 1 - Refactor: Extract the authentication middleware from a monolithic route file into a separate module with proper dependency injection, updating all 14 files that import from it. This tests multi-file understanding, import chain management, and architectural coherence.

Task 2 - Greenfield feature: Add rate limiting to the API with Redis backing, configurable per-endpoint limits, proper HTTP headers (X-RateLimit-Remaining, Retry-After) and comprehensive tests. This tests feature implementation from scratch, including edge case handling.

Task 3 - Bug fix: A reported production bug where concurrent Celery tasks were occasionally double-processing the same job. Root cause was a race condition in the task deduplication logic. This tests debugging ability, concurrency understanding, and fix quality.

Task 4 - Test generation: Generate meaningful tests for the payment processing module, which had 11% coverage. Not just coverage, tests that catch real regressions. We evaluated generated tests by running mutation testing against them.

Scoring methodology

We scored each tool on five dimensions for each task, each on a 1-5 scale.
Correctness: does the output work without modification? Completeness: does it handle edge cases and error paths? Code quality: is the code clean, maintainable, and consistent with existing patterns? Time-to-PR-ready: total time from prompt to a PR we'd actually merge, including our review and fixes. Review overhead: how much human correction was needed after the tool finished.
Total possible score per task: 25. Total possible across all four tasks: 100.

Task 1 results: Refactor

Claude Code dominated this task. It loaded the entire codebase context, identified all 14 files that needed updating, and produced a complete refactor in a single planning pass. The import chain was handled correctly. The dependency injection pattern it chose was consistent with the existing codebase conventions. One minor issue: it renamed an internal helper function unnecessarily, which we caught in review. Time-to-PR-ready: 23 minutes. Score: 22/25.

Cursor handled the refactor well through its Composer agent mode but required two iterations. The first pass correctly extracted the middleware but missed two import updates in files that imported indirectly through a barrel export. After pointing this out, the second pass fixed it cleanly. Time-to-PR-ready: 38 minutes. Score: 19/25.

Codex produced a clean initial extraction but struggled with the import chain complexity. It correctly updated 11 of 14 files. The three it missed were in subdirectories that imported the auth module through a re-export pattern. We fixed these manually. Time-to-PR-ready: 52 minutes. Score: 16/25.

Windsurf's Cascade mode completed the extraction with good coherence, it maintained a single context throughout and the changes were internally consistent. It missed one edge case where a test file imported the middleware directly rather than through the application module. Time-to-PR-ready: 41 minutes. Score: 18/25.

Task 2 results: Greenfield feature

This is where the tools were most evenly matched, and where Cursor's IDE integration gave it a practical edge.

Cursor delivered the most complete implementation. The rate limiting middleware included Redis backing, per-endpoint configuration, proper HTTP headers, sliding window algorithm, and tests that covered the happy path, rate-exceeded path, and Redis connection failure fallback. The inline development experience, writing code, testing in the integrated terminal, iterating on failures, was the most fluid. Time-to-PR-ready: 34 minutes. Score: 23/25.

Claude Code produced the highest quality code architecturally, cleaner separation, better error handling, more thorough documentation, but the terminal-only workflow meant more manual verification. The implementation was excellent but took longer to validate. Time-to-PR-ready: 42 minutes. Score: 22/25.

Windsurf delivered a solid implementation with good test coverage. The sliding window logic had a subtle off-by-one error in the expiry calculation that we caught during review. Time-to-PR-ready: 48 minutes. Score: 19/25.

Codex produced a working implementation but made a security-relevant choice we disagreed with, storing rate limit counters with a fixed TTL rather than a sliding window, which creates a predictable reset point an attacker could time around. Functionally correct, strategically questionable. Time-to-PR-ready: 55 minutes. Score: 17/25.

Task 3 results: Bug fix

The race condition bug was the hardest task and produced the widest performance gap.

Claude Code found the root cause in its first analysis pass. It identified the deduplication logic, recognised the TOCTOU (time-of-check-to-time-of-use) vulnerability, and proposed a fix using Redis SETNX for atomic deduplication. The fix included a test that specifically reproduced the race condition using concurrent task execution. Time-to-PR-ready: 19 minutes. Score: 24/25.

Cursor identified the area of the codebase where the bug likely lived but proposed adding a database lock rather than fixing the deduplication logic. The fix would have worked but was heavier than necessary and would have introduced latency on every task execution. After pointing this out, it proposed the SETNX approach as an alternative. Time-to-PR-ready: 44 minutes. Score: 17/25.

Windsurf struggled with this task. It identified the deduplication module but proposed adding retry logic rather than addressing the race condition itself. The proposed fix would have reduced the symptom frequency without eliminating the root cause. Time-to-PR-ready: 62 minutes after we redirected it to the actual problem. Score: 14/25.

Codex correctly identified the race condition but the proposed fix had its own concurrency issue, it used a Python threading lock in a Celery context where tasks may run in separate processes. The fix wouldn't have worked in the production configuration. Time-to-PR-ready: 58 minutes after we caught and corrected this. Score: 15/25.

Task 4 results: Test generation

Cursor generated tests the fastest and with the highest raw coverage, 87% on the payment module. However, mutation testing revealed that 31% of the generated tests were implementation-detail tests that would pass even with deliberate bugs injected. Effective mutation kill rate: 62%. Time-to-PR-ready: 28 minutes. Score: 18/25.

Claude Code generated fewer tests but with notably higher quality. Coverage reached 78%, but the mutation kill rate was 81%, meaning the tests were genuinely catching bugs, not just executing code paths. It also generated property-based tests for the amount calculation logic, which none of the other tools attempted. Time-to-PR-ready: 35 minutes. Score: 22/25.

Codex generated comprehensive tests with decent coverage (82%) and reasonable mutation kill rate (68%). The tests were conventional and well-structured. No surprises, positive or negative. Time-to-PR-ready: 32 minutes. Score: 19/25.

Windsurf generated tests with 79% coverage but the lowest mutation kill rate at 54%. Many tests asserted that functions returned specific values without testing the business logic that produced those values. Time-to-PR-ready: 40 minutes. Score: 16/25.

The aggregate scorecard

What the numbers actually mean

Claude Code won on total score, but Cursor won on the task most developers do most frequently, greenfield feature implementation within an IDE workflow. The daily experience of writing new code in Cursor is more fluid than any other tool we tested. The tab autocomplete alone is worth the subscription for inline development speed.

Claude Code's advantage is concentrated in tasks that require deep codebase understanding, the refactor, the race condition bug, the high-quality test generation. These are the tasks where the 1M token context window and terminal-native agentic architecture create a measurable difference. If your bottleneck is complex architectural work, Claude Code is the clear winner.

Codex and Windsurf performed similarly in aggregate but for different reasons. Codex is strong on well-defined, straightforward tasks. Windsurf's Cascade provides better contextual coherence on multi-file edits but struggles when the task requires reasoning about concurrent systems or subtle bugs.

What we actually use daily

After thirty days of controlled testing, here's what we landed on: Cursor for daily inline development, feature implementation, quick edits, code exploration. Claude Code for complex refactors, bug investigations, test suite improvements, and any task that benefits from holding the entire codebase in context. We don't use them interchangeably. They're complementary tools for different cognitive tasks.

For teams evaluating these tools, the comparison of Claude Code vs Cursor vs Windsurf covers the feature-by-feature breakdown with recommendations by team size and workflow type.

For enterprise teams evaluating these at scale, the criteria change, data residency, SSO, audit logs, and centralised cost management matter more than accuracy on any single task. We wrote the enterprise-focused breakdown separately because the procurement decision is fundamentally different from the developer experience decision.

Published by Dextra Labs, AI Consulting and Enterprise Agent Development

How to Architect AI Agents That Pass Banking Compliance Audits (Real Patterns, Not Theory)

Dextra Labs — Sun, 28 Jun 2026 19:51:23 +0000

Building agents for banking is 30% AI work and 70% compliance plumbing. The 30% is the easy part. Here's how to handle the 70%.

The first time you build an underwriting agent for a bank, you'll write the credit logic in about a week. Then you'll spend the next two months on audit logging, explainability, human-in-the-loop checkpoints, and data residency and you'll understand why fintech AI projects take longer than the demo suggests.

This is the architecture that gets through an audit, using loan underwriting as the running example.

The Core Requirement: Every Decision Must Be Traceable

Regulators don't accept "the model gave it a score of 0.82." They want to know what data was used, what reasoning was applied, and what a human would need to review to understand and challenge the decision.

class AuditableDecision:
    """
    Every agent decision in a regulated context must
    produce this structure. Not optional. Not added later.
    """
    def __init__(self, decision_id: str):
        self.decision_id = decision_id
        self.inputs_used = {}
        self.reasoning_steps = []
        self.data_sources_consulted = []
        self.model_version = AGENT_VERSION
        self.timestamp = datetime.utcnow().isoformat()
        self.outcome = None
        self.confidence = None
        self.human_reviewable_explanation = None

    def add_reasoning_step(self, step_description: str, evidence: dict):
        self.reasoning_steps.append({
            "step": len(self.reasoning_steps) + 1,
            "description": step_description,
            "evidence": evidence,
            "timestamp": datetime.utcnow().isoformat()
        })

    def finalise(self, outcome: str, confidence: float, explanation: str):
        self.outcome = outcome
        self.confidence = confidence
        self.human_reviewable_explanation = explanation

    def to_audit_record(self) -> dict:
        return {
            "decision_id": self.decision_id,
            "inputs": self.inputs_used,
            "reasoning_chain": self.reasoning_steps,
            "data_sources": self.data_sources_consulted,
            "model_version": self.model_version,
            "outcome": self.outcome,
            "confidence": self.confidence,
            "explanation": self.human_reviewable_explanation,
            "timestamp": self.timestamp
        }

Every step in the agent's reasoning gets appended to this structure as it happens, not reconstructed afterward from logs, which is unreliable and which auditors specifically check for.

The Underwriting Decision Pipeline

async def run_underwriting_agent(application: dict) -> dict:
    decision = AuditableDecision(decision_id=application["application_id"])

    # Step 1: Document verification
    doc_result = await verify_documents(application["documents"])
    decision.data_sources_consulted.append("document_verification_service")
    decision.add_reasoning_step(
        "Verified submitted financial documents",
        {"documents_checked": doc_result["count"], 
         "verification_status": doc_result["status"]}
    )

    if doc_result["status"] != "verified":
        decision.finalise(
            outcome="ESCALATE_TO_HUMAN",
            confidence=1.0,
            explanation=f"Document verification failed: {doc_result['reason']}. "
                        f"Routed to human reviewer for manual document check."
        )
        await audit_store.append(decision.to_audit_record())
        return {"status": "escalated", "reason": "document_verification"}

    # Step 2: Risk scoring via Claude with explicit reasoning
    risk_response = await client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1500,
        system="""Analyse this loan application's risk factors. 
        You MUST cite specific data points for every risk factor 
        identified. Vague reasoning fails compliance review.

        Return JSON: {
            "risk_score": float,
            "risk_factors": [{"factor": str, "data_point": str, "weight": str}],
            "recommendation": "APPROVE" | "DENY" | "HUMAN_REVIEW",
            "explanation": "2-3 sentences a non-technical auditor can verify"
        }""",
        messages=[{"role": "user", "content": json.dumps(application["financials"])}]
    )

    risk_analysis = json.loads(risk_response.content[0].text)
    decision.add_reasoning_step(
        "Risk assessment based on financial data",
        risk_analysis
    )

    # Step 3: Threshold-based routing - NOT autonomous final decisions
    if risk_analysis["risk_score"] > 0.7:
        outcome = "HUMAN_REVIEW"  # high risk always escalates
    elif risk_analysis["risk_score"] < 0.3:
        outcome = "AUTO_APPROVE"  # only very low risk auto-decides
    else:
        outcome = "HUMAN_REVIEW"  # middle ground escalates too

    decision.finalise(
        outcome=outcome,
        confidence=1 - abs(0.5 - risk_analysis["risk_score"]),
        explanation=risk_analysis["explanation"]
    )

    await audit_store.append(decision.to_audit_record())
    return {"status": outcome, "decision_record": decision.to_audit_record()}

Note the threshold structure: only very low risk applications auto-approve. Everything else, high risk and the ambiguous middle ground, escalates to a human. This isn't conservative for the sake of it; it's the threshold design that auditors expect and that protects the institution from autonomous decisions on the cases that matter most.

Human-in-the-Loop: Not a Button, a Checkpoint

A human-in-the-loop checkpoint needs to give the reviewer enough information to make a genuine decision, not just a button to click.

def build_human_review_package(decision: AuditableDecision, 
                                 application: dict) -> dict:
    return {
        "application_summary": application["summary"],
        "agent_reasoning_chain": decision.reasoning_steps,
        "agent_recommendation": decision.outcome,
        "confidence_level": decision.confidence,
        "specific_concerns": [
            rf for rf in decision.reasoning_steps 
            if rf.get("evidence", {}).get("weight") == "high"
        ],
        "override_requires_justification": True,
        "regulatory_basis": get_applicable_lending_regulations(application)
    }

override_requires_justification: True matters because a reviewer who can override without explaining why produces rubber-stamp approval rates, not genuine oversight and that's specifically what examiners check for.

PII Handling in Prompts

Financial data going into LLM prompts needs careful handling, especially for cross-border banks with data residency requirements.

def sanitise_for_prompt(application: dict, residency_zone: str) -> dict:
    """
    Strip or tokenise PII before it reaches the LLM prompt,
    based on the data residency requirements for this customer.
    """
    sanitised = application.copy()

    # Replace direct identifiers with tokens
    sanitised["applicant_name"] = f"APPLICANT_{hash_id(application['applicant_id'])}"
    sanitised["ssn_or_national_id"] = "[REDACTED]"

    if residency_zone == "EU":
        # GDPR-specific handling
        sanitised["address"] = tokenise_address(application["address"])

    return sanitised

For cross-border banks, the model call itself may need to happen in a specific region, or via a locally-hosted model, depending on the residency zone, this is an architectural decision made before any agent code is written, not a configuration flag added later.

What Auditors Actually Check

Decision immutability, audit records can't be modified after creation. Reasoning chain completeness, every step that influenced the outcome must be in the record, not reconstructed from application logs after the fact. Override documentation, every human override of an agent recommendation requires a written justification. Threshold justification, the bank must be able to explain why the auto-approve threshold is set where it is, with evidence it doesn't create disparate impact.

The full AI agents for underwriting in banking architecture guide covers the complete pipeline including the specific regulatory frameworks (FCRA, ECOA, FCA lending standards) that shape these design decisions.

Underwriting Is One Workflow

The patterns here, audit trails, explainability, human-in-the-loop thresholds, generalise across banking AI use cases, but the specific requirements differ meaningfully by workflow. Fraud detection has different latency requirements (you need decisions in milliseconds, not minutes) and different audit needs (pattern-based reasoning rather than document-based reasoning). Worth reading the fraud detection architecture guide before you design either system, because the architectural choices you make for one affect how cleanly you can extend to the other.

Published by Dextra Labs | AI Consulting & Enterprise Agent Development

RAG vs Agentic AI: A Developer's Decision Tree (With Code Examples for Both)

Dextra Labs — Wed, 24 Jun 2026 20:57:33 +0000

Two different problems wearing similar clothes. Here's how to tell them apart in thirty seconds, with working code for both.

I see this confusion in almost every project kickoff: "We need RAG" when the actual requirement is agentic, or "we need an agent" when RAG would be simpler, cheaper, and faster to ship.

Let's fix that with a decision tree you can actually use, plus working code for each path.

The Decision Tree

Does your system need to ANSWER QUESTIONS from documents?
├── YES, and that's the whole job → RAG
└── YES, but it also needs to TAKE ACTIONS across systems
    └── → Agent that uses RAG as a tool

Does your system need to TAKE ACTIONS across multiple systems?
├── YES, with no document retrieval needed → Plain Agent
└── YES, and it needs grounded knowledge from documents → 
    → Agent that uses RAG as a tool

The test question that resolves most confusion: "Does this system need to decide what to do, or does it need to find and synthesise information?" Finding and synthesising → RAG. Deciding and acting → agent.

Path 1: Pure RAG

RAG is the right architecture when your job is grounding LLM responses in a specific document set, answering questions, summarising content, finding relevant passages.

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import Chroma
from langchain.chains import RetrievalQA
from langchain_anthropic import ChatAnthropic

# 1. Load and chunk documents
splitter = RecursiveCharacterTextSplitter(
    chunk_size=800, 
    chunk_overlap=100
)
chunks = splitter.split_documents(documents)

# 2. Embed and store
embeddings = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-mpnet-base-v2"
)
vectorstore = Chroma.from_documents(chunks, embeddings)

# 3. Build the retrieval chain
llm = ChatAnthropic(model="claude-sonnet-4-5")
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever(search_kwargs={"k": 4}),
    return_source_documents=True
)

# 4. Query
result = qa_chain({"query": "What is our refund policy for enterprise customers?"})
print(result["result"])
print(result["source_documents"])  # Always show sources

This is the whole job: retrieve relevant chunks, ground the LLM's answer in them, return a response with citations. No planning loop, no tool orchestration, no multi-step decision-making. If your use case stops here, building agent infrastructure on top of this is unnecessary complexity.

Path 2: Pure Agent (No RAG)

An agent is right when the job is taking actions, checking systems, executing operations, making decisions that span multiple steps and there's no document knowledge base involved.

import anthropic

client = anthropic.Anthropic()

tools = [
    {
        "name": "check_inventory",
        "description": "Check current stock level for a SKU",
        "input_schema": {
            "type": "object",
            "properties": {"sku": {"type": "string"}},
            "required": ["sku"]
        }
    },
    {
        "name": "create_purchase_order",
        "description": "Create a PO with a supplier",
        "input_schema": {
            "type": "object",
            "properties": {
                "supplier_id": {"type": "string"},
                "sku": {"type": "string"},
                "quantity": {"type": "integer"}
            },
            "required": ["supplier_id", "sku", "quantity"]
        }
    }
]

def run_inventory_agent(goal: str) -> str:
    messages = [{"role": "user", "content": goal}]

    for _ in range(6):
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1500,
            tools=tools,
            messages=messages
        )

        if response.stop_reason == "end_turn":
            return next(b.text for b in response.content if hasattr(b, 'text'))

        messages.append({"role": "assistant", "content": response.content})
        tool_results = []

        for block in response.content:
            if block.type == "tool_use":
                result = execute_inventory_tool(block.name, block.input)
                tool_results.append({
                    "type": "tool_result",
                    "tool_use_id": block.id,
                    "content": result
                })

        messages.append({"role": "user", "content": tool_results})

    return "Reached max iterations."

run_inventory_agent(
    "Check stock for SKU-4471. If below 50 units, "
    "create a PO with our primary supplier for 200 units."
)

No documents involved. The agent checks inventory, reasons about the threshold, and conditionally creates a purchase order. This is pure action orchestration.

Path 3: The Hybrid Agent Using RAG as a Tool

This is where most real enterprise systems actually land: an agent that needs to take actions, and one of the things it needs to do along the way is look something up in a document knowledge base.

import anthropic

client = anthropic.Anthropic()

def rag_lookup(query: str) -> str:
    """RAG retrieval wrapped as a tool the agent can call."""
    result = qa_chain({"query": query})  # the RAG chain from Path 1
    return json.dumps({
        "answer": result["result"],
        "sources": [doc.metadata.get("source") for doc in result["source_documents"]]
    })

tools = [
    {
        "name": "search_policy_documents",
        "description": "Search company policy documents for relevant information",
        "input_schema": {
            "type": "object",
            "properties": {"query": {"type": "string"}},
            "required": ["query"]
        }
    },
    {
        "name": "issue_refund",
        "description": "Process a refund for a customer order",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "amount": {"type": "number"}
            },
            "required": ["order_id", "amount"]
        }
    }
]

def execute_tool(name: str, input_data: dict) -> str:
    if name == "search_policy_documents":
        return rag_lookup(input_data["query"])
    elif name == "issue_refund":
        return process_refund(input_data["order_id"], input_data["amount"])

def run_refund_agent(customer_request: str) -> str:
    messages = [{"role": "user", "content": customer_request}]

    for _ in range(6):
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1500,
            tools=tools,
            messages=messages
        )

        if response.stop_reason == "end_turn":
            return next(b.text for b in response.content if hasattr(b, 'text'))

        messages.append({"role": "assistant", "content": response.content})
        tool_results = [
            {"type": "tool_result", "tool_use_id": block.id,
             "content": execute_tool(block.name, block.input)}
            for block in response.content if block.type == "tool_use"
        ]
        messages.append({"role": "user", "content": tool_results})

    return "Reached max iterations."

run_refund_agent(
    "Customer wants a refund on order #8821 for $340. "
    "Check our refund policy first to see if this qualifies."
)

The agent decides to call search_policy_documents to check eligibility before deciding whether to call issue_refund. The RAG system is doing exactly what it's good at, grounded retrieval, but it's a tool in service of the agent's broader decision-making, not the entire system.

The Cost and Complexity Reality

RAG-only systems are cheaper to build and run. Single retrieval call, single generation call, predictable latency, easier to evaluate (you can measure retrieval precision and answer accuracy independently).

Agentic systems are more expensive and harder to debug. Multiple LLM calls per task, unpredictable latency (depends how many iterations the agent takes), harder to evaluate because failure can happen at the planning stage or the execution stage. They're also the only option when the task genuinely requires multi-step action across systems.

The mistake we see most often: teams building agentic infrastructure for what's fundamentally a question-answering problem, paying the complexity cost for capability they don't need.

The full RAG vs agentic AI comparison covers the cost modelling, latency benchmarks, and evaluation methodology differences in more depth.

What Comes After This Decision

Once you've picked your architecture, the next question is build vs buy, do you build this RAG pipeline or agent loop yourself, or do you use a managed platform? The answer depends on your timeline, your team's capacity, and how differentiated your specific use case actually is. We wrote the framework with cost models, time estimates, and decision criteria for exactly this question, worth reading before you commit engineering time to either path.

Published by Dextra Labs | AI Consulting & Enterprise Agent Development

Building an ERP Integration Layer for AI Agents: SAP, Oracle, NetSuite Architecture Patterns

Dextra Labs — Mon, 22 Jun 2026 20:16:26 +0000

Nobody writes about this because it's not glamorous. It's also where most enterprise AI projects actually live or die.

Every enterprise AI agent demo I've seen looks the same: clean API calls, mock data, instant responses. Then someone tries to connect it to actual SAP S/4HANA running in a client's data centre with a service account that was provisioned in 2019, and the demo timeline becomes a different project entirely.

ERP integration is the unglamorous 70% of enterprise AI agent work that determines whether your agent actually functions in production. This is the architecture guide I wish existed when we built our first finance agent integration.

The Authentication Layer: Three Patterns, Three ERPs

SAP S/4HANA - OAuth 2.0 with SAML bridging
SAP's modern API layer (via SAP Business Technology Platform) uses OAuth 2.0, but most enterprise SAP environments still run a hybrid setup with legacy RFC connections behind the scenes.

import requests
from datetime import datetime, timedelta

class SAPAuthManager:
    def __init__(self, client_id, client_secret, token_url):
        self.client_id = client_id
        self.client_secret = client_secret
        self.token_url = token_url
        self._token = None
        self._expiry = None

    def get_token(self) -> str:
        if self._token and datetime.now() < self._expiry:
            return self._token

        response = requests.post(self.token_url, data={
            "grant_type": "client_credentials",
            "client_id": self.client_id,
            "client_secret": self.client_secret
        })
        response.raise_for_status()

        data = response.json()
        self._token = data["access_token"]
        # SAP tokens typically expire in 3600s - refresh at 90% lifetime
        self._expiry = datetime.now() + timedelta(
            seconds=data["expires_in"] * 0.9
        )
        return self._token

    def get_headers(self) -> dict:
        return {
            "Authorization": f"Bearer {self.get_token()}",
            "Content-Type": "application/json"
        }

Oracle Fusion - Service accounts with JWT

Oracle Fusion Cloud uses OAuth 2.0 with JWT bearer tokens, typically via a dedicated service account with scoped REST API roles.

import jwt
import time
import requests

class OracleFusionAuth:
    def __init__(self, client_id, private_key, token_url, scope):
        self.client_id = client_id
        self.private_key = private_key
        self.token_url = token_url
        self.scope = scope

    def build_jwt_assertion(self) -> str:
        now = int(time.time())
        payload = {
            "iss": self.client_id,
            "sub": self.client_id,
            "aud": self.token_url,
            "exp": now + 300,
            "iat": now
        }
        return jwt.encode(payload, self.private_key, algorithm="RS256")

    def get_token(self) -> str:
        assertion = self.build_jwt_assertion()
        response = requests.post(self.token_url, data={
            "grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer",
            "assertion": assertion,
            "scope": self.scope
        })
        response.raise_for_status()
        return response.json()["access_token"]

NetSuite - Token-based authentication (TBA)

NetSuite uses OAuth 1.0a-style token-based auth, which is older but well-documented.

import hashlib
import hmac
import base64
import time
import secrets

class NetSuiteTBA:
    def __init__(self, account_id, consumer_key, consumer_secret, 
                 token_id, token_secret):
        self.account_id = account_id
        self.consumer_key = consumer_key
        self.consumer_secret = consumer_secret
        self.token_id = token_id
        self.token_secret = token_secret

    def build_auth_header(self, method: str, url: str) -> str:
        nonce = secrets.token_hex(16)
        timestamp = str(int(time.time()))

        base_string = "&".join([
            method, url, 
            f"oauth_consumer_key={self.consumer_key}",
            f"oauth_nonce={nonce}",
            f"oauth_signature_method=HMAC-SHA256",
            f"oauth_timestamp={timestamp}",
            f"oauth_token={self.token_id}",
            f"oauth_version=1.0"
        ])

        signing_key = f"{self.consumer_secret}&{self.token_secret}"
        signature = base64.b64encode(
            hmac.new(signing_key.encode(), base_string.encode(), 
                     hashlib.sha256).digest()
        ).decode()

        return (f'OAuth realm="{self.account_id}", '
                f'oauth_consumer_key="{self.consumer_key}", '
                f'oauth_token="{self.token_id}", '
                f'oauth_signature_method="HMAC-SHA256", '
                f'oauth_timestamp="{timestamp}", '
                f'oauth_nonce="{nonce}", '
                f'oauth_version="1.0", '
                f'oauth_signature="{signature}"')

RBAC matters across all three: provision the service account with the minimum scope needed for your agent's actions, never broad admin access. We've seen integrations provisioned with full ERP admin rights because it was "easier to set up", this is a security finding waiting to happen in any audit.

Data Sync: Event-Driven vs Polling

The architecture decision that affects everything downstream is whether your agent reacts to ERP events or polls for changes.

# POLLING PATTERN - simpler, higher latency, works everywhere
class ERPPollingSync:
    def __init__(self, erp_client, poll_interval_seconds=60):
        self.erp_client = erp_client
        self.poll_interval = poll_interval_seconds
        self.last_sync_timestamp = None

    async def poll_for_changes(self):
        while True:
            changes = await self.erp_client.get_changed_records(
                since=self.last_sync_timestamp
            )
            for record in changes:
                await self.process_change(record)

            self.last_sync_timestamp = datetime.utcnow()
            await asyncio.sleep(self.poll_interval)


# EVENT-DRIVEN PATTERN - lower latency, requires ERP webhook support
class ERPEventSync:
    def __init__(self, webhook_secret):
        self.webhook_secret = webhook_secret

    async def handle_webhook(self, request_body: bytes, signature: str):
        if not self.verify_signature(request_body, signature):
            raise ValueError("Invalid webhook signature")

        event = json.loads(request_body)

        # Idempotency check - ERP webhooks can fire duplicates
        if await self.already_processed(event["event_id"]):
            return {"status": "duplicate_ignored"}

        await self.process_event(event)
        await self.mark_processed(event["event_id"])

SAP S/4HANA supports event-driven architecture via SAP Event Mesh. Oracle Fusion supports business event subscriptions. NetSuite's webhook support is more limited, polling is often the pragmatic choice there.

Idempotency: Non-Negotiable for Financial Transactions

Every write operation your agent performs against an ERP needs an idempotency key. Network failures, retries, and webhook duplicates will otherwise create duplicate financial records.

class IdempotentERPWriter:
    def __init__(self, erp_client, idempotency_store):
        self.erp_client = erp_client
        self.store = idempotency_store

    async def create_invoice(self, invoice_data: dict, 
                              idempotency_key: str) -> dict:
        existing = await self.store.get(idempotency_key)
        if existing:
            return existing  # Already processed - return cached result

        try:
            result = await self.erp_client.create_invoice(invoice_data)
            await self.store.set(idempotency_key, result, ttl_hours=24)
            return result
        except Exception as e:
            await self.store.set(idempotency_key, 
                                  {"status": "failed", "error": str(e)}, 
                                  ttl_hours=1)
            raise

Audit Logging: Built In, Not Bolted On

Every agent action against an ERP needs an immutable audit record, what changed, who (or which agent) initiated it, and why.

async def execute_erp_action(action: dict, agent_context: dict):
    audit_record = {
        "agent_id": agent_context["agent_id"],
        "action_type": action["type"],
        "erp_system": action["target_system"],
        "record_affected": action["record_id"],
        "reasoning": agent_context["decision_rationale"],
        "timestamp": datetime.utcnow().isoformat(),
        "idempotency_key": action["idempotency_key"]
    }

    await audit_store.append(audit_record)  # append-only, immutable
    result = await erp_client.execute(action)

    audit_record["result"] = result
    audit_record["completed_at"] = datetime.utcnow().isoformat()
    await audit_store.append(audit_record)

    return result

ERP Downtime: Plan for It

ERP systems have maintenance windows and occasional outages. Your agent needs graceful degradation, not crashes.

class ResilientERPClient:
    def __init__(self, erp_client, circuit_breaker):
        self.erp_client = erp_client
        self.circuit_breaker = circuit_breaker

    async def call_with_resilience(self, operation, *args):
        if self.circuit_breaker.is_open():
            return {"status": "deferred", 
                     "reason": "ERP circuit breaker open"}

        try:
            result = await operation(*args)
            self.circuit_breaker.record_success()
            return result
        except ERPTimeoutError:
            self.circuit_breaker.record_failure()
            return {"status": "queued_for_retry"}

What This Means for Your Project Timeline

ERP integration is consistently underestimated in enterprise AI projects. The authentication setup alone, coordinating with the client's ERP admin team, provisioning service accounts, testing RBAC scopes, regularly takes longer than the AI agent's core logic development.

The full ERP integration architecture for AI agent finance workflows covers the complete reference architecture for all three ERP systems, including the specific gotchas we've hit with each.

ERP integration is one piece of enterprise AI agent deployment. The full enterprise AI agents in finance guide covers the rest of the stack, orchestration, compliance, monitoring, for teams building the complete system.

Published by Dextra Labs | AI Consulting & Enterprise Agent Development

AI Agent vs LLM: Why Just Calling the OpenAI API Doesn't Give You an Agent (And What Does)

Dextra Labs — Fri, 19 Jun 2026 18:55:36 +0000

"We built an agent, we called the API in a loop." No. You built a chatbot with extra steps. Here's the actual difference.

I want to start with a confession: I've made this mistake myself, early on, with enough confidence that I told a client we'd built an "agent" when what we'd actually built was a chatbot that occasionally called a function.

The mistake is understandable because the surface-level pattern looks similar. You call an LLM. You give it a prompt. You get a response. Loop it a few times, add a tool call, and it feels agentic. But there's a real, technical distinction between an LLM wrapper and an actual agent and the distinction matters because it determines whether your system can handle the messy, multi-step, failure-prone reality of production tasks.

Let's define it precisely, then build both versions so you can see exactly where the line is.

What Makes Something an Agent (Not Just an LLM Call)

An agent has four properties that a simple LLM wrapper doesn't have.

Autonomous tool use: the system decides which tools to call and when, based on reasoning about the current state, not a hardcoded sequence you wrote.

Planning and replanning: the system forms a plan, executes a step, observes the result, and revises its plan based on what it learns. A single API call with a fixed prompt can't do this. It doesn't see its own output and decide what to do next.

Memory across steps: the system maintains state about what it's already tried, what worked, and what didn't, within a single task execution.

Error recovery: when a tool call fails or returns unexpected output, the system adapts rather than crashing or returning a generic failure.

If your system doesn't do at least the first two, you have an LLM wrapper, not an agent. That's not an insult, LLM wrappers are useful and often the right tool. But calling it an agent when it's not creates a mismatch between what you've built and what stakeholders think they're getting.

The LLM Wrapper (What Most People Build First)

import anthropic

client = anthropic.Anthropic()

def simple_llm_call(user_query: str) -> str:
    """
    This is an LLM call. It is not an agent.
    It has no memory, no planning, no tool use,
    no error recovery.
    """
    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": user_query}]
    )
    return response.content[0].text

# This works fine for simple Q&A
result = simple_llm_call("What's the capital of France?")

This is perfectly fine code for the right job, single-turn question answering. The problem is when people wrap this in a loop and call it agentic:

def "agent"_with_a_loop(tasks: list[str]) -> list[str]:
    """
    This is NOT an agent. It's an LLM call 
    executed multiple times. There's no reasoning
    about what to do next based on previous results.
    """
    results = []
    for task in tasks:
        result = simple_llm_call(task)
        results.append(result)
    return results

This is a batch processing loop. It doesn't reason about whether task 2 should change based on the outcome of task 1. It doesn't use tools. It doesn't recover from failure beyond whatever exception handling you bolt on externally. Calling this an agent is where the confusion starts.

The Actual Agent: ReAct Pattern With Tool Use

A real agent implements the ReAct pattern, Reasoning and Acting in a loop, where each action produces an observation that informs the next reasoning step.

import anthropic
import json

client = anthropic.Anthropic()

tools = [
    {
        "name": "search_database",
        "description": "Search the customer database by criteria",
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {"type": "string"}
            },
            "required": ["query"]
        }
    },
    {
        "name": "send_email",
        "description": "Send an email to a customer",
        "input_schema": {
            "type": "object",
            "properties": {
                "recipient": {"type": "string"},
                "subject": {"type": "string"},
                "body": {"type": "string"}
            },
            "required": ["recipient", "subject", "body"]
        }
    }
]

def execute_tool(tool_name: str, tool_input: dict) -> str:
    """Actual tool execution — this is what makes it agentic."""
    if tool_name == "search_database":
        # Real database call would go here
        return json.dumps({"customer_id": 4521, "status": "active", 
                           "last_order": "2026-05-12"})
    elif tool_name == "send_email":
        # Real email sending would go here
        return json.dumps({"sent": True, "message_id": "msg_8821"})
    return json.dumps({"error": "unknown tool"})


def run_agent(goal: str, max_iterations: int = 8) -> str:
    """
    This IS an agent. It reasons, acts, observes, 
    and replans based on what it learns at each step.
    This is the property a simple LLM call doesn't have.
    """

    messages = [{"role": "user", "content": goal}]

    for iteration in range(max_iterations):
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=2048,
            tools=tools,
            messages=messages
        )

        # The model decided whether to use a tool or finish.
        # This decision happens at every step based on 
        # everything that's happened so far — that's planning.
        if response.stop_reason == "end_turn":
            final_text = next(
                (b.text for b in response.content if hasattr(b, 'text')),
                "Task completed."
            )
            return final_text

        if response.stop_reason == "tool_use":
            messages.append({"role": "assistant", "content": response.content})

            tool_results = []
            for block in response.content:
                if block.type == "tool_use":
                    print(f"  [Agent decided to call] {block.name}({block.input})")

                    # Execute and observe — error recovery happens here
                    try:
                        result = execute_tool(block.name, block.input)
                    except Exception as e:
                        result = json.dumps({"error": str(e), 
                                              "recoverable": True})

                    tool_results.append({
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": result
                    })

            # The observation goes back into context.
            # The NEXT reasoning step sees what happened
            # and decides what to do based on it.
            messages.append({"role": "user", "content": tool_results})

    return "Max iterations reached without completing the goal."

Run this with a goal like "Find customer 4521's order status and email them an update if their last order was more than 30 days ago" and watch what happens. The agent calls search_database, observes the result, reasons about whether the date condition is met, and only then decides whether to call send_email. If the database call returned an error, the agent would see that in its context and could try a different search strategy, that's the replanning property.

This is the part a fixed-sequence script can't do. A script that calls search_database() then unconditionally calls send_email() is not reasoning about the result. It's executing a predetermined sequence that happens to involve an LLM somewhere in it.

The Test: Could You Replace the LLM Call With an If-Statement?

Here's the practical test I use when someone shows me their "agent" architecture.

If you could replace the LLM call with a deterministic if-statement and get the same behaviour, you don't have agentic reasoning, you have a script with an LLM bolted on for the parts that needed natural language generation.

# This is NOT agentic, even though it calls an LLM
def fake_agent(customer_id: int):
    customer = search_database(f"id:{customer_id}")
    if customer['days_since_order'] > 30:
        email_body = simple_llm_call(
            f"Write a follow-up email for customer {customer_id}"
        )
        send_email(customer['email'], "Update", email_body)

The LLM here is doing text generation, not reasoning about what action to take. The decision logic is all in your Python if statement. This is a perfectly reasonable architecture for plenty of use cases, but it's not an agent. It's a script that uses an LLM as a writing tool.

The real agent version lets the model decide whether to check the order date, whether the result warrants an email, what the email should say, and whether to retry if the email send fails, all through its own reasoning over the conversation history, not through your hardcoded control flow.

Memory: The Property Most Wrapper Implementations Skip

A genuine agent maintains state across the interaction, which means it can reference earlier observations when making later decisions.

class AgentSession:
    """
    Memory across a task execution. This is what 
    separates a stateful agent from a stateless 
    LLM call repeated in a loop.
    """

    def __init__(self):
        self.messages = []
        self.tool_call_history = []
        self.observations = {}

    def add_observation(self, tool_name: str, result: dict):
        self.tool_call_history.append({
            "tool": tool_name,
            "result": result,
            "step": len(self.tool_call_history)
        })
        # Agent can later ask "did I already check this?"
        self.observations[tool_name] = result

    def already_attempted(self, tool_name: str) -> bool:
        """
        This prevents the agent from repeating failed 
        actions indefinitely — a common failure mode
        in naive implementations.
        """
        return tool_name in self.observations

Without this, a naive agent loop can call the same failing tool repeatedly because it has no memory of having already tried it. This is one of the most common production bugs in early agent implementations, the agent gets stuck in a loop because nothing tells it "you already tried this and it didn't work."

Why This Distinction Actually Matters

This isn't pedantry. The architectural decision has real consequences.

If your task is single-turn, answer this question, summarise this document, classify this text, an LLM wrapper is simpler, cheaper, faster, and easier to debug. Building agent infrastructure for a task that doesn't need planning or tool orchestration is wasted complexity.

If your task requires multiple steps where each step's outcome affects the next decision, process this application by checking three systems and deciding what to do based on what you find, you need actual agentic architecture. An LLM wrapper will fail because it can't adapt mid-task to what it discovers.

The full breakdown of AI agent vs LLM architectures, including when each is the right call for specific production use cases, covers the decision framework in more depth than fits here.

What's Next: RAG vs Agentic Architecture

Now that the LLM vs agent distinction is clear, the next architectural decision most teams face is RAG vs agentic AI and these get confused almost as often as LLM vs agent does. RAG solves the problem of grounding LLM responses in your specific documents. Agentic architecture solves the problem of taking multi-step actions across systems. They're different tools for different problems, and the RAG vs agentic AI comparison we published covers exactly where each one fits, including the hybrid pattern where an agent uses RAG as one of its tools.

Published by Dextra Labs | AI Consulting & Enterprise Agent Development

Building AI Agents for Compliance Monitoring in Finance: Architecture That Passes Auditors

Dextra Labs — Wed, 27 May 2026 20:36:28 +0000

The compliance AI that can't explain its decisions is worse than no compliance AI. Here's how to build one that can.

There's a specific failure mode that kills fintech AI projects that traditional software projects don't have.

The system works. The accuracy is good. The false positive rate is acceptable. And then your compliance officer asks: "Why did this transaction get flagged?" And the answer is "the model gave it a score of 0.87", which is not an answer a regulator will accept.

Explainability in compliance AI isn't a nice-to-have. It's a regulatory requirement. FINRA, FCA, RBI, every major financial regulator has issued guidance making clear that automated compliance decisions require documented reasoning that a human auditor can review and challenge. "The AI said so" is not documented reasoning.

This tutorial covers how to build a compliance monitoring agent architecture that produces decisions an auditor can actually work with.

The Architecture Overview

REGULATORY DATA FEEDS
(OFAC, FATF, FinCEN, local watchlists)
         ↓
[INGESTION AGENT] — normalise, deduplicate, version
         ↓
TRANSACTION STREAM (real-time)
         ↓
[SCREENING AGENT] — rule-based + Claude analysis
         ↓ 
         ├── LOW RISK → auto-clear + audit log
         ├── MEDIUM RISK → flag + evidence package → analyst queue
         └── HIGH RISK → block + SAR draft → senior review
                    ↓
         [AUDIT TRAIL AGENT] — immutable decision log
                    ↓
         [REPORTING AGENT] — SAR generation, regulatory reporting

Every stage produces a structured, human-readable decision record. This isn't a post-processing step, it's built into every agent's output schema from day one.

Agent 1: Regulatory Data Ingestion

Regulatory watchlists change constantly. OFAC updates the SDN list multiple times a week. FATF grey/black lists update quarterly. Local regulators issue updates on irregular schedules.

from anthropic import Anthropic
from datetime import datetime
import hashlib
import json

client = Anthropic()

class RegulatoryIngestionAgent:
    def __init__(self, db_connection, audit_logger):
        self.db = db_connection
        self.audit = audit_logger

    async def ingest_watchlist_update(
        self, 
        source: str,
        raw_data: bytes,
        update_metadata: dict
    ) -> dict:
        """
        Ingests watchlist updates with full provenance tracking.
        Every entry gets a source, version and effective date.
        """

        # Parse with Claude for flexible format handling
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=4000,
            system="""Parse regulatory watchlist data into 
            structured entities. Handle variations in format
            across different regulatory sources.

            Extract for each entity:
            - canonical_name (primary identifier)
            - aliases (all alternative names)
            - entity_type (individual/organisation/vessel/aircraft)
            - identifiers (passport, tax ID, registration numbers)
            - addresses (with country codes)
            - listing_reason (sanctions program or crime category)
            - effective_date
            - source_reference (regulatory document ID)

            Return JSON array of entities.
            Flag any entries with ambiguous identity markers.""",
            messages=[{
                "role": "user",
                "content": f"Source: {source}\n\n{raw_data.decode('utf-8', errors='replace')}"
            }]
        )

        entities = json.loads(response.content[0].text)

        # Version control for watchlist entries
        for entity in entities:
            entity['_provenance'] = {
                'source': source,
                'ingest_timestamp': datetime.utcnow().isoformat(),
                'source_document_hash': hashlib.sha256(raw_data).hexdigest(),
                'regulatory_effective_date': update_metadata.get('effective_date'),
                'version_id': self.generate_version_id(entity, source)
            }

        await self.db.upsert_watchlist_entities(entities)

        self.audit.log({
            'event': 'watchlist_update_ingested',
            'source': source,
            'entities_added': len(entities),
            'timestamp': datetime.utcnow().isoformat()
        })

        return {
            'entities_processed': len(entities),
            'flagged_for_review': [e for e in entities if e.get('ambiguous')]
        }

The provenance tracking matters for audit purposes. When an auditor asks "was this entity on the watchlist at the time of this transaction?", you need to be able to answer precisely, not "yes, they're on the list now" but "this entity was added to the OFAC SDN list on [date] under [regulatory reference] and was active in our database from [timestamp]."

Agent 2: Real-Time Transaction Screening

This is the core compliance agent. It needs to be fast, blocking a payment for 30 seconds to run compliance checks is not acceptable in most contexts and it needs to produce explainable decisions.

class TransactionScreeningAgent:

    RISK_THRESHOLDS = {
        'auto_clear': 0.25,
        'analyst_review': 0.6,
        'block_and_escalate': 0.85
    }

    async def screen_transaction(
        self, 
        transaction: dict
    ) -> dict:
        """
        Screens transaction against watchlists and risk models.
        Returns decision with full reasoning chain for audit trail.
        """

        # Fast rule-based pre-screen
        rule_matches = await self.run_rule_engine(transaction)

        if rule_matches['exact_match']:
            return self.build_decision(
                transaction, 
                risk_score=0.95,
                decision='BLOCK',
                reasoning_type='exact_watchlist_match',
                evidence=rule_matches
            )

        # Claude analysis for fuzzy matching and context
        entity_context = await self.get_entity_context(
            transaction['counterparty']
        )

        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1500,
            system="""You are a compliance analyst screening 
            financial transactions. Analyse the transaction
            against the provided entity context and risk factors.

            Provide a structured risk assessment with:
            1. Risk score (0.0-1.0)
            2. Primary risk factors (list each with evidence)
            3. Mitigating factors (if any)
            4. Decision rationale (2-3 sentences, auditor-readable)
            5. Recommended action: AUTO_CLEAR / ANALYST_REVIEW / BLOCK
            6. Confidence level: HIGH / MEDIUM / LOW

            Be specific. Cite the exact data points that 
            influenced the score. Vague rationale fails audits.

            Return as JSON with schema:
            {
                "risk_score": float,
                "risk_factors": [{"factor": str, "evidence": str, "weight": str}],
                "mitigating_factors": [str],
                "decision_rationale": str,
                "recommended_action": str,
                "confidence": str,
                "additional_checks_required": [str]
            }""",
            messages=[{
                "role": "user",
                "content": f"""Transaction details:
Amount: {transaction['amount']} {transaction['currency']}
Counterparty: {transaction['counterparty_name']}
Counterparty country: {transaction['counterparty_country']}
Transaction type: {transaction['type']}
Reference: {transaction.get('reference', 'None')}
Originating account risk tier: {transaction['account_risk_tier']}

Entity context from watchlist database:
{json.dumps(entity_context, indent=2)}

Fuzzy name match results:
{json.dumps(rule_matches['fuzzy_matches'], indent=2)}"""
            }]
        )

        analysis = json.loads(response.content[0].text)

        return self.build_decision(
            transaction,
            risk_score=analysis['risk_score'],
            decision=analysis['recommended_action'],
            reasoning_type='claude_analysis',
            evidence=analysis
        )

    def build_decision(
        self, 
        transaction: dict,
        risk_score: float,
        decision: str,
        reasoning_type: str,
        evidence: dict
    ) -> dict:
        """
        Builds the decision record that goes to audit trail.
        Every field that an auditor might ask about is explicit.
        """
        return {
            'transaction_id': transaction['id'],
            'screening_timestamp': datetime.utcnow().isoformat(),
            'decision': decision,
            'risk_score': risk_score,
            'reasoning_type': reasoning_type,
            'evidence': evidence,
            'agent_version': AGENT_VERSION,
            'watchlist_versions_consulted': self.get_active_watchlist_versions(),
            'regulatory_basis': self.get_applicable_regulations(transaction),
            'human_review_required': risk_score >= self.RISK_THRESHOLDS['analyst_review']
        }

The watchlist_versions_consulted field is one of the most important for audit purposes. When a regulator asks "was this screened against the current OFAC list?", you can provide the exact version ID of the list that was active at screening time.

Agent 3: The Audit Trail Agent

The audit trail is not a log. It's an immutable, queryable record of every compliance decision with enough context to reconstruct the reasoning from scratch.

class AuditTrailAgent:

    def __init__(self, immutable_store):
        # Immutable store — append only, no updates, no deletes
        self.store = immutable_store

    async def record_decision(self, decision_record: dict) -> str:
        """
        Records a compliance decision with full provenance.
        Returns the immutable record ID for reference.
        """

        # Generate explainability summary for human review
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=800,
            system="""Generate a plain-language explanation of 
            this compliance decision suitable for regulator review.

            The explanation must:
            1. State the decision and its risk basis clearly
            2. Identify the specific factors that drove the decision
            3. Note any watchlist matches with regulatory references
            4. Explain what additional review was triggered, if any
            5. Be written so a non-technical compliance officer
               can understand and defend it

            Maximum 200 words. No jargon. No model internals.
            The reader is an auditor, not a data scientist.""",
            messages=[{
                "role": "user",
                "content": json.dumps(decision_record, indent=2)
            }]
        )

        human_readable_explanation = response.content[0].text

        audit_record = {
            **decision_record,
            'human_readable_explanation': human_readable_explanation,
            'record_created_at': datetime.utcnow().isoformat(),
            'record_id': self.generate_record_id(decision_record)
        }

        record_id = await self.store.append(audit_record)

        return record_id

    async def generate_examination_report(
        self,
        date_range: tuple,
        transaction_ids: list = None,
        include_auto_cleared: bool = False
    ) -> dict:
        """
        Generates examination-ready compliance report.
        Format designed for regulatory examination.
        """

        records = await self.store.query(
            date_range=date_range,
            transaction_ids=transaction_ids,
            include_auto_cleared=include_auto_cleared
        )

        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=3000,
            system="""Compile a compliance examination report 
            from transaction screening records.

            Structure the report as regulators expect:
            1. Executive summary (screening volume, decision distribution)
            2. High-risk transaction summary (blocked and escalated)
            3. Watchlist match analysis (by source, match type)
            4. False positive analysis (analyst overrides)
            5. System performance metrics
            6. Notable patterns or anomalies

            Be factual. Cite specific transaction IDs for examples.
            Format for readability — this goes to regulators.""",
            messages=[{
                "role": "user",
                "content": f"Records for period {date_range}:\n{json.dumps(records, indent=2)}"
            }]
        )

        return {
            'report': response.content[0].text,
            'record_count': len(records),
            'period': date_range,
            'generated_at': datetime.utcnow().isoformat()
        }

The Explainability Requirement in Practice

The human-readable explanation generation is the piece that compliance teams consistently cite as the most valuable. Not the risk score, the explanation.

When an analyst reviews a flagged transaction, they need to understand not just that the system flagged it but why, in terms they can defend to a regulator. "Risk score: 0.73" tells them nothing they can act on. "Transaction flagged: counterparty name 'Al-Rashid Trading LLC' returns 0.87 similarity to sanctioned entity 'Al-Rasheed Trading' on OFAC SDN list (added 2024-03-15, Program: SDGT). Transaction amount ($47,000) above standard trade threshold for counterparty country. Pattern consistent with structuring indicators from FinCEN Advisory FIN-2023-A001" tells them exactly what to investigate.

The AI agents for compliance monitoring in finance article covers the full regulatory framework mapping, which specific regulations require which types of documentation, in detail.

What Auditors Actually Check

Three things that compliance AI architectures consistently fail on during examination:

Decision immutability: Auditors check that compliance records can't be modified after the fact. Your audit trail store must be append-only. If your logging goes to a database where records can be updated, you'll fail this check.

Watchlist version traceability: "We screened against the watchlist" is not sufficient. "We screened against OFAC SDN List version 20260415-1423, which was active from 2026-04-15 14:23 UTC" is sufficient.

Override documentation: When analysts override an automated decision, clearing a flagged transaction or escalating an auto-cleared one, the rationale must be documented in the compliance record. Systems that allow override without documentation create audit exposure.

Compliance Is One Layer

The architecture above handles transaction screening and AML monitoring. It's one component of a full agentic AI banking stack. For the complete architecture covering KYC automation, fraud detection, lending decisioning and portfolio risk management, the agentic AI in banking guide covers the full system design that compliance monitoring plugs into.

Compliance is just one banking use case. For the complete architecture guide covering lending, KYC, fraud detection and portfolio management, we published the complete agentic AI in banking guide. The compliance layer described here is designed to integrate cleanly with each of those use cases.

Published by Dextra Labs | AI Consulting & Enterprise Agent Development