Originally published at twarx.com - read the full interactive version there.
Last Updated: July 21, 2026
Most AI technology deployments in banking are solving the wrong problem entirely. When Goldman Sachs put its AI banking agents into wider production this year, the headline was model capability — but the real work was coordination between systems that were never designed to hand off to each other. The intelligence was never the bottleneck.
This piece is about a decision every financial services operator now faces: deploy a custom small language model (SLM) or an off-the-shelf LLM like GPT-4o, Claude, or Gemini — orchestrated through LangGraph, CrewAI, or n8n. It matters right now because the enterprise AI agent market is exploding and a 2026 Boomi study of 300+ IT and business leaders (fielded Q4 2025, published January 2026) found that integration — not smarter models — drives agent trust.
After this, you'll know exactly which to deploy, what it costs, and how to close the gap that quietly wrecks most rollouts.
The custom SLM vs off-the-shelf LLM decision is rarely about raw intelligence — it's about where the AI Coordination Gap opens between models and core banking systems. Source
AI Technology in Banking: Why the Model Choice Is the Least Important Decision You'll Make
Financial services leaders discover something inconvenient after they've already signed the vendor contract: the language model is maybe 20% of the value and 80% of the marketing. The other 80% — where nearly every project stalls — lives in the coordination between the model, your core banking platform, your KYC systems, your fraud engines, and your compliance logging. That is the part nobody demos.
A six-step approval pipeline where each step is 97% reliable is only 83% reliable end-to-end. In a mortgage origination or trade-settlement workflow, that 17% failure rate isn't a rounding error — it's a regulatory incident. Teams discover this after they've shipped, when a customer complaint escalates and nobody can explain which handoff dropped the data. I've watched this play out more than once. It never gets less painful.
The choice between a custom SLM and an off-the-shelf LLM is real and consequential. But it's downstream of a bigger question: have you designed the coordination layer that connects the model to the systems that actually run your business? That single question separates the firms shipping durable AI technology from those endlessly re-benchmarking models.
The banks winning with AI agents are not the ones with the most GPUs. They're the ones who solved coordination between systems no one originally designed to talk to each other.
Coined Framework — Extract This
The AI Coordination Gap
Definition: The AI Coordination Gap is the reliability and trust deficit that opens between an AI model's raw capability and the fragmented enterprise systems it must orchestrate to complete a real task. It is a systemic failure of handoffs — schema mismatches, dropped fields, non-idempotent writes, missing audit trails — not a failure of intelligence. A smarter model cannot close it; only disciplined coordination across grounding, integration, orchestration, and governance can.
In financial services this gap gets amplified by three forces: strict auditability requirements, latency-sensitive transactions, and data locked in siloed systems of record — core banking, CRM, fraud, custody. A general-purpose LLM is spectacular at reasoning over language. By default it's terrible at knowing that your loan-origination system uses a proprietary status code, or that a settlement instruction must be idempotent. That knowledge lives in the coordination layer. That's what you're actually buying, building, or neglecting.
Gartner analysts have been blunt about where the risk concentrates. As Anushree Verma, Senior Director Analyst at Gartner, put it in the firm's 2025 agentic AI guidance: 'Most agentic AI projects right now are early stage experiments or proof of concepts that are mostly driven by hype and are often misapplied.' The pattern she describes is the coordination gap by another name — impressive models bolted onto integrations that were never designed to carry them.
This article breaks the decision into a five-layer framework, shows how each layer works in practice, walks through named deployments including Goldman Sachs and Morgan Stanley, and answers the seven questions operators ask most. The goal isn't to tell you SLM or LLM is 'better.' It's to give you a model for deciding — and a blueprint for closing the coordination gap regardless of which you pick.
15x
cost delta: a self-hosted fine-tuned SLM runs ~$4K/mo vs $60K+/mo for GPT-4o at 2M docs/month
[Modeled from published API + inference pricing, 2026](https://azure.microsoft.com/en-us/products/phi)
28%
of enterprises cite integration — not model quality — as the top blocker to agent trust
[Boomi Agent Study, 2026 (n=300+, fielded Q4 2025)](https://boomi.com/)
83%
end-to-end reliability of a 6-step pipeline at 97% per-step accuracy (0.97^6)
[Compound reliability math, arXiv 2024](https://arxiv.org/)
What Is a Custom SLM and Why Do Financial Firms Suddenly Want One?
A small language model (SLM) is a model in the roughly 1B–15B parameter range — think Microsoft's Phi-3, Mistral 7B, Llama 3.1 8B, or Google's Gemma — that you can fine-tune on your own data and run on your own infrastructure. An off-the-shelf LLM is a frontier model (GPT-4o, Claude Opus, Gemini 2.5) accessed via API, where you rent intelligence by the token.
The naive framing is 'big model = smart, small model = cheap.' That framing is wrong, and it costs banks millions. The real axes are control, latency, cost at scale, and data residency — the exact four things regulated finance cares about most. Get those four right and the parameter count barely matters.
A fine-tuned 7B model that knows your loan codes will beat a frontier model that doesn't — on your task, every time. Domain fit beats raw IQ inside the enterprise.
Now the claim most operators resist: for narrow, high-volume, latency-sensitive financial tasks, a fine-tuned SLM often outperforms a frontier LLM — not on general reasoning, but on the specific task, at roughly 1/20th the inference cost. Andrej Karpathy, former Director of AI at Tesla and a founding member of OpenAI, has repeatedly argued that most production tasks are narrow enough that a smaller, specialized model is the correct engineering choice. The frontier model is a Swiss Army knife. Your fraud-triage classifier needs a scalpel.
A custom SLM lives inside your VPC and knows your domain; an off-the-shelf LLM rents general intelligence by the token. The right choice depends on volume, latency, and data residency — not benchmark scores. Source
DimensionCustom SLM (fine-tuned)Off-the-Shelf LLM (API)
Inference cost at scaleVery low — self-hosted, ~$0.0001/1K tokens amortizedHigh at volume — $0.003–$0.015/1K tokens
Latency20–80ms on-prem, predictable300ms–2s, network-dependent
Data residencyFull control, stays in your VPCLeaves your perimeter (unless private endpoint)
General reasoningWeaker outside training domainBest-in-class, broad
Time to first value6–12 weeks (data + fine-tune)Days
Best fitHigh-volume narrow tasks: triage, classification, extractionComplex reasoning, low volume, prototyping
AuditabilityFull — you own weights and logsPartial — vendor-dependent
A fine-tuned Llama 3.1 8B model serving a document-extraction task at 2M documents/month can cost under $4,000/month self-hosted — versus $60,000+/month routing the same volume through GPT-4o. That's a 15x delta. But the number is meaningless if the extraction output isn't correctly coordinated into your downstream loan system.
What Is the AI Coordination Gap? The Five-Layer Framework
Whether you deploy an SLM or an LLM, the value only materializes when five layers work together. Skip any one and the gap opens. Every layer below is numbered so you can reference it directly in a design review — this is The Five-Layer Coordination Stack.
The Five-Layer Coordination Stack: Deployment Architecture for Financial Services AI Technology
1
**Model Layer — SLM or LLM selection**
Fine-tuned Mistral/Llama SLM for narrow high-volume tasks, or GPT-4o/Claude for complex reasoning. Decision driven by volume, latency SLA, and data residency — not benchmarks. Latency budget set here.
↓
2
**Grounding Layer — RAG + vector DB**
Retrieval-Augmented Generation over your policies, rate sheets, and compliance docs using Pinecone or pgvector. Prevents hallucination on facts the model was never trained on. Sub-100ms retrieval target.
↓
3
**Tool/Integration Layer — MCP + connectors**
Model Context Protocol servers expose core banking, CRM, and fraud APIs as typed tools. This is where the coordination gap most often opens. Idempotency and typed schemas mandatory.
↓
4
**Orchestration Layer — LangGraph / CrewAI / n8n**
State machine that sequences steps, handles retries, routes between agents, and enforces human-in-the-loop checkpoints. Deterministic control flow around probabilistic models.
↓
5
**Governance Layer — audit, eval, guardrails**
Full trace logging, output evaluation, PII redaction, and rollback. In regulated finance this is not optional — it's what makes the whole system deployable at all.
The sequence matters because failure compounds downstream — a great model (Layer 1) grounded on bad data (Layer 2) will confidently corrupt your core banking system (Layer 3).
Layer 1: Model — The Decision Everyone Overweights
Choose the SLM path when your task is narrow, high-volume, and latency-sensitive: transaction categorization, document extraction, fraud pre-screening, chat intent classification. Choose the LLM path for complex multi-step reasoning at lower volumes: drafting credit memos, summarizing earnings calls, answering nuanced advisor queries. Many mature deployments run both — an SLM for the 90% of routine traffic and an LLM fallback for the hard 10%. This cascade pattern works in production. Use it. If you're weighing which foundation models to build on, enterprise AI teams increasingly favor a portfolio over a single-vendor bet.
Layer 2: Grounding — Where RAG Beats Fine-Tuning
Fine-tuning teaches a model behavior and format. RAG gives it current facts. Your rate sheets change weekly; you don't re-fine-tune weekly. Instead you retrieve the current sheet from a vector database at query time. Pinecone and Postgres pgvector are both production-ready here. This is the single most cost-effective way to cut hallucination in a financial context — and it's the first thing I'd wire up before touching anything else.
Layer 3: Tool/Integration — The Actual Coordination Gap
Coined Framework
The AI Coordination Gap
The gap is widest at the integration layer: the model produces a perfect answer, but the handoff to your core banking API drops a field, mismatches a schema, or fires twice. The intelligence was never the problem — the plumbing was.
This is where Anthropic's Model Context Protocol (MCP) has become the standard. MCP lets you expose your internal systems as typed, discoverable tools the model can call safely. Instead of brittle prompt-glued API calls, you define a contract. For financial systems, every write operation must be idempotent — a retried settlement instruction that executes twice isn't a bug report, it's a compliance incident.
Layer 4: Orchestration — Deterministic Control Around Probabilistic Models
LangGraph (production-ready), CrewAI (maturing fast), and n8n (production-ready for lower-code teams) sit here. The orchestration layer is a state machine: it decides what happens next, retries failed steps, and — critically in finance — inserts human-in-the-loop checkpoints before any irreversible action. You can learn more about multi-agent systems and how they coordinate.
Layer 5: Governance — Non-Negotiable in Regulated Finance
Every model call, retrieval, and tool invocation must be logged for audit. Output evaluation runs continuously. PII is redacted before it hits any external API. The NIST AI Risk Management Framework is a useful reference here. I would not ship a financial AI system without this layer in place — and no compliance officer worth their salary will let you either.
The orchestration layer wraps probabilistic models in deterministic control flow — retries, routing, and human-in-the-loop checkpoints that make AI deployable in regulated finance. Source
How Do You Deploy AI Technology in Banking? A Practical Build Sequence
Don't start with the model. Start with the workflow you want to automate and map its handoffs. Every time I've seen a team open with model selection, they end up retrofitting the integration layer under a ticking deadline. A few years back I watched a mid-size regional lender — roughly a $40M annual tech budget — burn most of two quarters fine-tuning a model for a loan-status workflow, only to discover the real blocker was a core banking API that returned undocumented status codes. The model was flawless. The plumbing wasn't. Here's the build order that avoids that.
First, instrument the existing manual process and measure baseline: minutes per task, error rate, where handoffs fail. Second, build the governance and tool layer before the fancy model — stand up MCP connectors and audit logging with a cheap model. Third, add RAG grounding. Fourth, decide SLM vs LLM based on the volume and latency you actually measured. Fifth, wrap it in LangGraph orchestration with a human checkpoint on every irreversible action. That sequence isn't glamorous. It works.
For teams evaluating ready-made components, you can explore our AI agent library for pre-built financial workflow agents, and see how workflow automation patterns apply to back-office operations.
Python — LangGraph SLM/LLM cascade with human checkpoint
Route routine traffic to a cheap fine-tuned SLM,
escalate hard cases to a frontier LLM, and require
human approval before any irreversible banking action.
from langgraph.graph import StateGraph, END
def classify(state):
# Fine-tuned SLM handles ~90% of intent classification
confidence = slm_classify(state['query'])
state['confidence'] = confidence
return state
def route(state):
# Low confidence -> escalate to frontier LLM, else execute
if state['confidence']
The single highest-ROI move in most financial AI projects is not upgrading the model — it's adding idempotency keys to write operations at the tool layer. It eliminates the double-execution failure mode that causes the most severe production incidents.
What Do Most Companies Get Wrong About SLM vs LLM?
Firms treat this as a procurement decision — pick a vendor, sign a contract, done. It's an architecture decision. Teams benchmark models against MMLU and pick the highest scorer, when their actual task is 4-way intent classification that a fine-tuned 3B model nails at 40x lower cost. Then they build the model layer first and bolt on integration last. That sequencing guarantees the coordination gap opens exactly where it hurts most. I've seen this pattern eat two to three quarters of runway before anyone admits the model was never the problem.
Real Deployments: Goldman Sachs, Morgan Stanley, and JPMorgan
Goldman Sachs rolled out its GS AI Assistant to over 10,000 employees and has been advancing autonomous coding and banking agents. In a June 2025 interview, Marco Argenti, Goldman's Chief Information Officer, described the firm bringing on an autonomous software-engineering agent (Devin) as part of a 'hybrid workforce' — with heavy emphasis on control and governance rather than raw model capability. The intelligence was never in doubt. The deployment work was coordination and controls.
Goldman's own CIO frames AI adoption as building a 'hybrid workforce' of humans and agents — not chasing a smarter model. The bottleneck was always coordination and control, never IQ.
Morgan Stanley built its wealth-management assistant on OpenAI models, grounded via RAG over its vast internal research library — a textbook Layer 2 grounding play. The value came not from the raw model but from retrieval over 100,000+ proprietary documents no off-the-shelf model had ever seen. Jeff McMillan, who led the firm's AI effort, has been direct: the retrieval and evaluation harness was the hard part. The model was almost incidental.
JPMorgan deployed its LLM Suite to tens of thousands of employees and continues to invest heavily in in-house model capability — a hybrid stance where sensitive, high-volume tasks move toward controlled internal models while frontier LLMs handle exploratory reasoning. This mirrors the cascade pattern exactly. You can see parallels in how enterprise AI deployments favor hybrid model portfolios over single-vendor bets.
[
▶
Watch on YouTube
How Goldman Sachs Is Deploying AI Agents in Banking
Financial services AI agent deployment
](https://www.youtube.com/results?search_query=goldman+sachs+ai+agents+financial+services)
Common Mistakes That Open the Coordination Gap
❌
Mistake: Picking the model by benchmark score
Choosing GPT-4o because it tops MMLU, when your task is 4-way intent classification at 3M requests/month. You pay 40x for reasoning you'll never use, and inference latency blows your SLA.
✅
Fix: Fine-tune a Mistral 7B or Llama 3.1 8B on 5,000 labeled examples of YOUR task. Benchmark on your data, not public leaderboards.
❌
Mistake: Non-idempotent tool calls
The orchestrator retries a failed step and the payment or settlement instruction executes twice. This is the most common severe incident in production financial AI — and it has nothing to do with the model.
✅
Fix: Attach idempotency keys to every write operation at the MCP tool layer. Make retries safe by design.
❌
Mistake: Fine-tuning for facts that change
Teams fine-tune a model on current rate sheets, then re-train every time rates change. It's expensive, slow, and the model hallucinates stale numbers between retrains.
✅
Fix: Use RAG with Pinecone or pgvector for anything that changes. Fine-tune only for behavior, format, and domain tone.
❌
Mistake: No human checkpoint on irreversible actions
Letting an agent autonomously execute trades, approve loans, or move funds. Regulators won't accept it, and one hallucinated action becomes a headline.
✅
Fix: Use LangGraph's interrupt-before pattern to force human approval on every irreversible node. Log the approval for audit.
What Comes Next: Predictions for Financial Services AI
2026 H2
**MCP becomes the default integration standard in banking**
With Anthropic's Model Context Protocol adoption accelerating across major vendors, expect core banking platforms to ship native MCP servers — closing the integration layer of the coordination gap.
2027 H1
**SLM cascades become the cost-standard architecture**
As fine-tuning tooling matures and inference costs of frontier models stay high, the SLM-first-with-LLM-fallback cascade becomes the default for high-volume financial workflows.
2027 H2
**Regulators formalize AI agent audit requirements**
Following the enterprise agent adoption surge, expect explicit governance-layer mandates — full trace logging and human-checkpoint requirements written into financial supervisory guidance, echoing the EU AI Act.
2028
**Owned model weights become a competitive moat**
Banks that built custom SLMs on proprietary data will hold a durable advantage over those renting general intelligence — the data-plus-weights moat compounds.
The trajectory for financial services AI: from renting frontier LLM intelligence toward owned custom SLM cascades governed by MCP-standard integration — the durable close of the AI Coordination Gap. Source
Coined Framework
The AI Coordination Gap
Closing the gap is a five-layer discipline, not a model upgrade. The firms that treat coordination — grounding, integration, orchestration, governance — as the product will out-execute those chasing benchmark scores.
Go deeper on the orchestration options in our guides to LangGraph, AutoGen, and building resilient AI agents and orchestration patterns. If you'd rather deploy proven components than build from scratch, our library of production-ready AI agents covers the most common financial workflows out of the box.
Coined Framework
The AI Coordination Gap
Every dollar spent on a smarter model while the integration and governance layers remain unbuilt widens the gap instead of closing it. Spend on coordination first.
Frequently Asked Questions
What is agentic AI?
Agentic AI is a system where a language model plans, calls tools, and takes multi-step actions toward a goal with some autonomy — not just answering a single prompt. In practice the model can query an API, evaluate the result, decide the next step, and loop until done. Frameworks like LangGraph, CrewAI, and AutoGen provide the orchestration. In financial services, agentic systems handle reconciliation, document processing, and customer triage — but always with human-in-the-loop checkpoints before irreversible actions. The key distinction from a chatbot is that an agent has access to tools via MCP connectors and can change the state of real systems. Production deployments cap autonomy tightly and log every action for audit.
How does multi-agent orchestration work?
Multi-agent orchestration coordinates several specialized agents through a controller that routes work between them, with a shared state object carrying context. LangGraph models this as a state machine: nodes are agents or functions, edges define transitions. For example, a classifier agent (a cheap SLM) routes a request, a reasoning agent (an LLM) handles complex cases, and a validator agent checks output before a tool executes. The orchestrator handles retries, timeouts, and human checkpoints. The critical design principle is deterministic control flow around probabilistic models — you don't let the model choose the whole path freely; you constrain it. This is exactly what closes the coordination gap in high-stakes financial workflows.
What companies are using AI agents in banking?
Goldman Sachs, Morgan Stanley, and JPMorgan are the leading examples — all won on coordination, not model choice. Goldman deployed its GS AI Assistant to over 10,000 employees and is advancing autonomous coding agents. Morgan Stanley built a wealth-management assistant on OpenAI models grounded via RAG over its internal research library. JPMorgan rolled out its LLM Suite to tens of thousands of staff and invests heavily in in-house models. Beyond finance, Klarna, Salesforce (Agentforce), and Intercom run customer-service agents at scale. The common thread among successful deployments is investment in the integration, grounding, and governance layers. The Boomi 2026 study found integration drives agent trust more than model capability — which matches what these firms report.
What is the difference between RAG and fine-tuning?
Fine-tuning changes how a model responds (behavior, format, tone); RAG changes what facts it knows at query time by retrieving them from a vector database. Use fine-tuning for stable patterns: your compliance writing style, your loan-code taxonomy, your classification task. Use RAG for anything that changes: rate sheets, policies, customer records, retrieved from Pinecone or pgvector. The most common mistake is fine-tuning on facts that change, which forces expensive retrains and causes stale hallucinations. Most production financial systems use both — a fine-tuned model for behavior, RAG for current facts. They're complementary, not competing.
How do I get started with LangGraph?
Install with pip install langgraph langchain, define a shared state schema, add nodes (functions or agents), and connect them with edges — start with a single linear graph before adding conditional routing. The core concepts are the StateGraph, nodes, edges, and conditional edges. For financial use cases, learn the interrupt-before pattern immediately, which pauses execution for human approval on sensitive nodes — this is what makes the system deployable in regulated environments. Use LangSmith for tracing so you can debug and audit every step. Begin with a low-stakes internal workflow like document summarization before touching anything that writes to core systems. The official LangChain docs have runnable quickstarts, and our LangGraph guide walks through a full financial-services example.
What are the biggest AI failures to learn from?
The most instructive AI failures share one pattern: the model worked, but the coordination failed. Air Canada's chatbot gave a customer wrong policy information and a tribunal held the airline liable — a grounding-layer failure where the bot wasn't connected to authoritative policy. Several banks have had agents double-execute transactions due to non-idempotent tool calls during retries — an integration-layer failure. Chatbots have been manipulated into agreeing to absurd terms because there was no output validation — a governance-layer failure. Upgrading the model would not have prevented any of these. The fix is grounding with RAG, idempotent tool design, output guardrails, and human checkpoints on irreversible actions.
What is MCP in AI?
MCP (Model Context Protocol) is an open standard from Anthropic that lets AI models connect to external tools, data, and systems through a consistent, typed interface. Instead of writing brittle custom glue code for each API, you run an MCP server that exposes your systems — core banking, CRM, fraud engines — as discoverable tools with defined schemas. The model calls these tools safely, and you control exactly what it can access. MCP is rapidly becoming the default integration layer for enterprise AI because it directly addresses the coordination gap: it standardizes the handoff between the model and your systems. For financial services, MCP servers let you enforce typed contracts, idempotency, and permission scoping — turning fragile prompt-glued integrations into auditable, production-grade connections.
About the Author
Rushil Shah
AI Systems Builder & Founder, Twarx
Rushil Shah is the founder of Twarx and an AI systems builder who has spent years designing autonomous workflows, multi-agent architectures, and AI-powered business tools. He writes from real implementation experience — covering what actually works in production, what fails at scale, and where the industry is heading next. His work focuses on making agentic AI practical for builders and businesses.
LinkedIn · Full Profile
This article was originally published on Twarx. Follow for daily deep dives on AI agents and automation.
Top comments (0)