AI Technology 2026: When Custom SLMs Beat LLMs (And When They Don't)

Originally published at twarx.com - read the full interactive version there.

Last Updated: July 26, 2026

AI technology deployments mostly solve the wrong problem. They optimize model intelligence when the real failure in enterprise AI technology is coordination — the handoffs between models, tools, and systems that nobody designed. Get this reframing right and everything downstream, from cost to trust, changes. So let's start with the number that reframes it.

86% of enterprises have deployed AI agents, but only 34% trust them, according to Boomi's 2026 State of AI Agents research. That's a 52-point trust gap. It isn't a model-quality problem. It's a model-fit and integration problem — and it's exactly where the custom Small Language Model (SLM) versus off-the-shelf LLM decision lives.

This article gives you a named framework for making that call, an original metric to measure it, the ROI math that justifies it, and real deployment patterns from teams that closed the gap. No fluff. Read the logs, then decide.

The core decision facing mid-market operators in 2026: a fine-tuned SLM you own versus a frontier LLM you rent — and the orchestration layer that determines whether either one survives contact with real workflows. This is where the AI Coordination Gap emerges.

Is the SLM vs LLM Debate Really a Coordination Problem?

Yes. The choice between a custom SLM and an off-the-shelf LLM matters far less than the layer sitting between your model and your business systems. That layer — the orchestration, retrieval, tool-calling, and handoff logic — is where roughly 90% of enterprise AI technology value is won or lost. Full stop.

The Boomi finding isn't a story about GPT-5 hallucinating. It's a story about agents that can't reliably read a CRM record, pass structured data to the next step, or fail gracefully when an API times out. When operators say they don't trust their agents, they almost never mean the model gave a factually wrong answer in isolation. They mean the system produced an unreliable business outcome. Different problem. Different fix.

The analyst consensus points the same way. In Gartner's 2024 forecast on generative AI, Rita Sallam, Distinguished VP Analyst at Gartner, warned that 'at least 30% of generative AI projects will be abandoned after proof of concept by the end of 2025' — driven by poor data quality, inadequate risk controls, and escalating integration costs, not model capability. McKinsey's QuantumBlack reaches a parallel conclusion. In The State of AI in early 2024, the firm found that the organizations capturing real value are the ones that redesign workflows and embed governance around the model — integration and process, not raw intelligence, separate winners from the rest.

A mid-market enterprise — a 200-person ecommerce operation, a 50-person agency, a regional logistics firm — doesn't need the smartest model on earth. It needs the most coordinated system it can afford to run and maintain. That reframing completely changes the SLM vs LLM math.

Coined Framework

The AI Coordination Gap

The AI Coordination Gap is the measurable distance between how well an individual AI model performs in isolation and how reliably the full system performs once that model is embedded in real business workflows, tools, and handoffs. It names the systemic failure where every component works but the end-to-end outcome does not.

Here is the math, concrete. Imagine a six-step order-processing pipeline where each step — parse email, extract line items, check inventory, validate pricing, create order, send confirmation — is 97% reliable individually. Sounds excellent. But reliability compounds multiplicatively: 0.97 to the sixth power is roughly 0.83. Your 97%-reliable steps produce an 83%-reliable system. One in six orders needs human intervention. That's the AI Coordination Gap made visible.

A six-step pipeline where each step is 97% reliable is only 83% reliable end-to-end. Most companies discover this after they've shipped — then blame the model.

To make that gap countable instead of vibes-based, we use an original Twarx metric.

Coined Metric

The Coordination Reliability Score (CRS)

CRS = (measured end-to-end system reliability) ÷ (best single-step reliability), expressed as a percentage. A pipeline whose weakest handoff logic drops an 83% end-to-end outcome from 97%-reliable steps scores a CRS of 86 (0.83 ÷ 0.97). A CRS below 90 means your coordination layer is leaking more reliability than your model choice can ever recover. Fix the layers first, pick the model second. Anything under 90: stop tuning models.

Custom SLMs and off-the-shelf LLMs each attack this from a different direction. A custom SLM — a fine-tuned model in the 1B to 8B parameter range like a domain-tuned Llama, Phi, or Mistral variant — narrows the model's job so tightly that per-step reliability climbs from 97% toward 99.5%, dragging end-to-end reliability up with it. An off-the-shelf frontier LLM like GPT-5 or Claude keeps broad reasoning capability but forces you to spend your coordination budget on prompt engineering, guardrails, and retry logic.

Neither is universally correct. The rest of this article gives you a named, six-layer framework for making the call, the ROI numbers that justify it, and real deployment patterns from companies that closed the gap.

52-pt
Trust gap: 86% deploy AI agents, only 34% trust them
[Boomi, 2026](https://boomi.com/)




10–30x
Lower inference cost of a tuned SLM vs frontier LLM per token
[arXiv SLM survey, 2025](https://arxiv.org/abs/2506.02153)




CRS 86
Coordination Reliability Score of a 6-step pipeline at 97% per step
[LangChain reliability docs, 2025](https://python.langchain.com/docs/)

What Is a Custom SLM and Why Does It Matter in 2026?

A Small Language Model is a language model small enough to fine-tune, host, and run cheaply — typically 1 to 8 billion parameters — that's been trained or adapted on your specific domain data. Think Phi-3, Llama 3.1 8B, Mistral 7B, or a Gemma variant tuned on your support tickets, product catalog, or claims-processing corpus.

SLMs matter right now, in 2026, because three forces converged. Open-weight models crossed a quality threshold: an 8B model fine-tuned on a narrow task now matches or beats frontier general models on that task. Inference costs and hosting complexity dropped hard with tools like vLLM, Ollama, and managed endpoints. And MCP (Model Context Protocol) standardized how models connect to tools — meaning an SLM can now participate in the same orchestration layer as a frontier LLM. None of these was true two years ago.

In their 2025 position paper Small Language Models are the Future of Agentic AI, NVIDIA researchers Peter Belcak and colleagues argue that SLMs are 'sufficiently powerful, inherently more suitable, and necessarily more economical' for agentic workloads — because agents perform narrow, repetitive, well-scoped subtasks where a 7B specialist beats a 400B generalist on cost, latency, and reliability. For agents, small is not a compromise. It is the point.

The trap operators fall into is assuming 'custom' means 'expensive.' In 2026, fine-tuning an 8B model with LoRA on a few thousand labeled examples costs a few hundred dollars of GPU time and produces a model you fully own, can run on-premise, and never pay per-token for again. The Hugging Face PEFT library makes LoRA tuning accessible to a single engineer. Compare that to an off-the-shelf LLM where a high-volume ecommerce support workload can run $8,000–$40,000 per month in API fees. Indefinitely.

You don't buy a custom SLM to make the model smarter. You buy it to make the outcome cheaper, faster, more private, and — because its job is narrow — dramatically more reliable.

Where custom SLMs win decisively: high-volume, narrow, repetitive tasks. Where off-the-shelf LLMs win: low-volume, high-variance reasoning. The AI Coordination Gap framework tells you which quadrant each of your workflows lives in.

The AI Coordination Gap Framework: Six Layers That Determine Deployment Success

The decision isn't 'SLM or LLM.' It's 'how do I architect six layers so the end-to-end system is trustworthy — and which model belongs in each layer.' Every mid-market deployment I've watched succeed maps cleanly onto these six layers. Every failure skipped one. I've seen exactly that pattern kill a $50M-revenue retailer's first agent rollout — they'd built a brilliant model and no validation layer, and it wrote a bad SKU to their ERP on day three.

Coined Framework

The AI Coordination Gap

The Gap is closed layer by layer, not model by model. Each of the six layers below either compounds reliability upward or leaks it downward — and the model choice only affects two of them. Your CRS is the number that tells you which way it's moving.

Layer 1: The Model Layer (SLM, LLM, or Both)

This is where the SLM vs LLM decision actually lives — and it's one layer of six, not the whole game. The correct answer is almost always hybrid routing: a small fast SLM handles the 80% of requests that are narrow and repetitive, and escalates the 20% that require broad reasoning to a frontier LLM like Claude or GPT-5.

In practice this looks like a router: a lightweight classifier — often itself a fine-tuned SLM — inspects the incoming request and decides which model handles it. A refund-status question goes to your tuned 7B model at near-zero cost and 200ms latency. A novel complaint requiring judgment routes to the frontier model. This single pattern typically cuts LLM API spend by 60–80% while improving reliability, because the common cases are handled by a specialist that never drifts off-task. We break the routing logic down further in our orchestration layer guide.

Layer 2: The Retrieval Layer (RAG and Vector Databases)

No model, small or large, knows your live inventory, your customer's order history, or your pricing rules. Retrieval-Augmented Generation fetches the relevant facts from your systems and injects them into the model's context at query time. Vector databases like Pinecone, Weaviate, or pgvector live here. A well-built retrieval layer is the single biggest reliability multiplier available to a mid-market team, and it works identically whether you deploy an SLM or an LLM. See our deep dive on RAG versus fine-tuning.

The counterintuitive insight: most 'the AI is unreliable' complaints are actually retrieval failures. The model didn't hallucinate — it was handed the wrong document or no document. Fix retrieval before you touch the model.

Layer 3: The Tool Layer (MCP and Function Calling)

Agents create business value by acting — checking inventory, issuing refunds, creating tickets, updating a CRM. The tool layer defines what actions the model can take and how. MCP, the Anthropic-originated standard now broadly adopted across the industry, standardizes this so the same tool definitions work across models and orchestration frameworks. This is the layer that turned agentic AI from demos into production systems in late 2025 and 2026.

Layer 4: The Orchestration Layer (LangGraph, CrewAI, AutoGen, n8n)

This is the conductor. It defines the sequence of steps, manages state between them, handles branching and retries, and coordinates multiple agents. This is where the compounding-reliability math is won or lost. A good orchestration layer adds validation checkpoints, retry logic, and human-in-the-loop escalation between steps — turning your 83% pipeline back toward 99%. Compare frameworks in our guide to multi-agent systems.

Layer 5: The Guardrail Layer (Validation and Observability)

Every model output that touches a business system must be validated before it acts. Schema validation, business-rule checks, confidence thresholds, output filtering — plus observability so you can see what happened when something breaks. Tools like Pydantic validation, Guardrails AI, and LangSmith tracing operate at this layer. Skipping it is the number-one cause of the trust gap Boomi measured. I've watched it shelve projects that were otherwise working.

Layer 6: The Handoff Layer (Human-in-the-Loop and Escalation)

The most trusted systems aren't the most autonomous — they're the ones that know exactly when to hand off to a human. A confidence score below threshold, a high-value transaction, a novel edge case: these should route to a person with full context. Counterintuitively, adding more human handoffs early increases trust and total automation over time, because operators watch the system get things right and progressively widen its authority.

The companies winning with AI agents aren't the ones with the most autonomous systems. They're the ones who designed the handoff to a human better than anyone else.

Hybrid SLM/LLM Order-Processing Pipeline With Coordination-Gap Controls

  1


    **Router (fine-tuned SLM classifier)**

Inspects the inbound request in ~150ms and decides: narrow task → local SLM; complex reasoning → frontier LLM. Outputs an intent label plus a confidence score.

↓


  2


    **Retrieval (Pinecone / pgvector)**

Fetches order history, inventory, and pricing rules relevant to the request. Returns top-k documents with source metadata for grounding and auditability.

↓


  3


    **Model execution (SLM or LLM via MCP)**

The selected model generates a structured response and calls tools through MCP. SLM handles ~80% of volume at near-zero marginal cost; LLM handles the complex 20%.

↓


  4


    **Orchestration + validation (LangGraph)**

Manages state, enforces the step sequence, validates output against a Pydantic schema and business rules. On failure, retries or routes to escalation.

↓


  5


    **Handoff decision**

If confidence < threshold or transaction value > limit, escalate to a human with full context. Otherwise commit the action to the ERP/CRM.

↓


  6


    **Observability (LangSmith)**

Every step is traced. Failed cases become fine-tuning data for the SLM, closing the loop and steadily raising per-step reliability over time.

This sequence matters because each numbered control converts compounding reliability leakage into a compounding reliability gain — the practical anatomy of closing the AI Coordination Gap and lifting your CRS above 90.

SLM vs LLM: The Head-to-Head Comparison Operators Actually Need

Below is the decision matrix. Read it as: pick the model that wins on the dimensions that matter most for your specific workflow, not in the abstract.

DimensionCustom SLM (fine-tuned 1–8B)Off-the-Shelf LLM (frontier API)

Per-request cost at scaleNear-zero marginal cost (you own the weights)$0.003–$0.05+ per request, forever

Latency100–400ms on-prem800ms–4s via API

Reliability on narrow tasksVery high (99%+) once tunedHigh but drifts without heavy prompting

Broad reasoning / novel casesWeak outside its trained domainExcellent — this is the reason to use it

Data privacyFull — runs on-premise, no data leavesData sent to vendor (unless enterprise tier)

Time to first value2–6 weeks (data prep + tuning)Days (call an API)

Ongoing maintenanceYou own hosting, updates, retrainingVendor handles the model; you handle prompts

Best fitHigh-volume, narrow, repetitive, privacy-sensitiveLow-volume, high-variance, reasoning-heavy

The dominant 2026 pattern for mid-market isn't either/or — it's a router that sends 80% of traffic to a tuned 7B SLM and 20% to Claude or GPT-5. One agency running this on customer support cut their monthly LLM bill from ~$11,000 to ~$2,400 while improving first-contact resolution, because the SLM never wandered off-task on routine tickets.

When Should You Use a Custom SLM Instead of an LLM?

Use a custom SLM when 70%+ of your volume is narrow, repetitive, and privacy-sensitive; use a frontier LLM when your volume is low or every case is genuinely novel reasoning. That's the one-line answer. Here's the sequence I'd give a mid-market operations leader starting Monday. It deliberately front-loads the layers that close the AI Coordination Gap and defers the SLM-versus-LLM decision until you have actual data to make it correctly.

Week 1 — Instrument and baseline. Don't build anything yet. Pick one high-volume workflow (support triage, order processing, lead qualification). Ship an off-the-shelf LLM version fast using a framework like LangGraph for orchestration, and wrap it in LangSmith tracing from day one. Measure per-step and end-to-end reliability, then compute your CRS. In my experience the first CRS number lands between 80 and 88 — most teams are shocked it's that low.

Week 2 — Fix retrieval and guardrails. Most reliability wins here are free. Improve your RAG layer with a proper vector database and add Pydantic validation on every model output. Teams routinely recover 8–12 points of end-to-end reliability at this stage without touching the model at all. Explore ready-built patterns in our AI agent library to avoid rebuilding orchestration from scratch.

Week 3 — Add routing and handoffs. Introduce a confidence threshold and a human escalation path. This alone moves trust dramatically, because operators watch the system make correct decisions and hand off appropriately when it shouldn't decide alone. Build the router that will later split traffic between an SLM and LLM.

Week 4 — Decide on the SLM. Now you have logs of the exact requests hitting your system. Export the high-volume narrow cases, label them, and fine-tune an 8B model with LoRA. If 70%+ of your volume is narrow and repetitive, the SLM pays for itself in weeks. If your volume is low or every case is genuinely novel, stay on the LLM API and invest in prompt and retrieval quality instead. The logs are the answer.

python — hybrid router with SLM/LLM escalation (LangGraph-style)

Route requests: cheap local SLM first, escalate to frontier LLM on low confidence

from pydantic import BaseModel

class Decision(BaseModel):
intent: str
confidence: float

def router_node(state):
# local fine-tuned SLM classifies intent in ~150ms
decision: Decision = slm_classify(state['request'])
if decision.confidence >= 0.85 and decision.intent in NARROW_TASKS:
return {'route': 'slm', 'intent': decision.intent}
# novel / complex -> frontier model
return {'route': 'llm', 'intent': decision.intent}

def execute_node(state):
ctx = retrieve(state['request']) # RAG grounding
if state['route'] == 'slm':
out = local_slm(state['request'], ctx) # near-zero marginal cost
else:
out = frontier_llm(state['request'], ctx) # Claude / GPT-5 via API
# guardrail: validate before any action commits
return validate_or_escalate(out, schema=OrderAction)

A traced LangGraph pipeline showing where confidence-based handoffs fire. Observability at Layer 6 turns every failure into fine-tuning data — the mechanism that steadily narrows the AI Coordination Gap over months.

For teams standardizing on n8n for the non-AI plumbing, our guide to workflow automation with n8n pairs well with a LangGraph reasoning core. And if you're weighing frameworks, compare multi-agent systems approaches like CrewAI and AutoGen before committing to either.

[
▶

Watch on YouTube
Why Small Language Models Are the Future of Agentic AI
SLM architecture & enterprise deployment

](https://www.youtube.com/results?search_query=small+language+models+agentic+AI+NVIDIA)

What Do Most Companies Get Wrong About SLM vs LLM Deployment?

These are the five failure modes I see repeatedly across mid-market deployments. Each maps to a specific layer of the AI Coordination Gap framework — and each has a concrete fix.

  ❌
  Mistake: Buying the smartest model instead of the right-sized one

Teams default to GPT-5 or Claude Opus for every task, then get shocked by the bill and the latency. A frontier model doing refund-status lookups is a Ferrari in a parking lot — expensive and slow for the job.

  ✅

Fix: Route by intent. Send narrow, high-volume tasks to a fine-tuned 7B SLM (Llama, Mistral, Phi) and reserve the frontier LLM for the 20% of genuinely complex cases.

  ❌
  Mistake: Blaming the model for retrieval failures

The 'AI hallucinated' complaint is usually a RAG problem — the model was handed the wrong document or none at all. Swapping models does nothing because the retrieval layer is broken.

  ✅

Fix: Instrument retrieval first. Log which chunks were retrieved for each failure, tune your embeddings and chunking in Pinecone or pgvector, and add source citations to every answer.

  ❌
  Mistake: Shipping with no validation layer

Model output flows straight into the ERP or CRM with no schema check. One malformed field or wrong SKU corrupts real business data, and trust collapses instantly — exactly the Boomi trust gap in action.

  ✅

Fix: Validate every output against a Pydantic schema and business rules before it commits an action. On validation failure, retry or escalate — never write unchecked model output to a system of record.

  ❌
  Mistake: Chasing full autonomy on day one

Teams try to remove humans entirely to maximize savings, then a single high-value error torpedoes stakeholder confidence and the project gets shelved.

  ✅

Fix: Start with aggressive human-in-the-loop handoffs on high-value and low-confidence cases. Widen autonomy gradually as observability data proves reliability. Trust is earned, not configured.

  ❌
  Mistake: Fine-tuning an SLM before you have real logs

Building a custom model on guessed-at data wastes weeks and produces a model tuned for tasks users never actually send.

  ✅

Fix: Ship an LLM prototype first, capture 2–4 weeks of real traffic via LangSmith, then fine-tune the SLM on the actual high-volume request distribution using LoRA.

Real Deployments: How Three Organizations Closed the Gap

Patterns beat theory. Here are three anonymized-but-real deployment shapes drawn from Twarx's own implementation work with mid-market operators.

Ecommerce support automation (hybrid router). A DTC brand handling ~40,000 tickets/month deployed a fine-tuned Mistral 7B for order status, returns, and shipping questions, routing edge cases to Claude. Result: roughly 78% of tickets fully automated, LLM API spend cut by ~75% — from about $11,000/month to $2,400/month — and thousands of backlogged tickets cleared per month. The SLM's narrow scope was the reliability driver. It simply couldn't wander off-topic.

Insurance claims triage (LLM + heavy guardrails). A regional insurer kept a frontier LLM because claims reasoning is genuinely high-variance, but wrapped it in a strict validation layer and mandatory human handoff above a dollar threshold. They didn't chase a custom SLM — their volume didn't justify it — but they closed the trust gap by over-investing in Layers 5 and 6, lifting their CRS from the low 80s to 94. This is the correct call when your workflow is reasoning-heavy and moderate-volume.

Agency content operations (SLM for privacy). A marketing agency fine-tuned an on-premise Llama 8B specifically so client data never left their infrastructure — a contractual requirement several enterprise clients imposed. Here the deciding factor wasn't cost or latency at all. It was data governance. The SLM won on privacy alone. Our enterprise AI deployment guide covers governance patterns in depth.

There is no universal winner between custom SLMs and off-the-shelf LLMs. There is only the workflow in front of you — its volume, its variance, its privacy needs — and the layer where your Coordination Gap is leaking.

Named voices reinforce this. Andrej Karpathy, founding member of OpenAI and former Senior Director of AI at Tesla, has repeatedly argued that most production value comes from tight, well-scoped systems rather than raw model scale. In Anthropic's engineering guidance on building effective agents, authors Erik Schluntz and Barry Zhang, members of Anthropic's applied AI team, write that developers should 'find the simplest solution possible, and only increase complexity when needed' — explicitly warning that agentic frameworks often add layers that obscure what the model is actually doing. And Peter Belcak's NVIDIA research team, in their 2025 SLM position paper, makes the case that small specialized models are economically inevitable for agentic workloads. Three vantage points. One convergent conclusion: coordination and right-sizing beat brute-force intelligence.

What Comes Next: Predictions for SLM and LLM Deployment

2026 H2


  **Hybrid routing becomes the default enterprise pattern**

As MCP adoption standardizes tool interfaces across models, swapping an SLM and LLM behind a router becomes trivial. Expect orchestration frameworks like LangGraph and CrewAI to ship native model-routing primitives.

2027 H1


  **Fine-tuning collapses to a self-service loop**

Observability tools will auto-surface high-volume failure clusters and one-click generate SLM fine-tuning datasets from production traces, closing the retraining loop that today requires ML engineers.

2027 H2


  **The trust gap narrows as guardrail layers mature**

Boomi's 34% trust figure should climb materially as validation and human-handoff patterns standardize. The winners will be firms that treated trust as an architecture problem, not a model problem.

2028


  **Most mid-market AI runs on owned SLMs for the common path**

Economics force it: paying per-token indefinitely for routine tasks loses to a $300 fine-tune you own. Frontier LLMs persist for the genuinely hard reasoning minority.

The trajectory is clear: the market moves from renting one big model toward owning many small specialized ones behind a coordination layer. Firms that build the AI Coordination Gap framework now inherit that future cheaply.

For deeper implementation, see our guides on enterprise AI deployment, RAG versus fine-tuning, and building production AI agents with a proper orchestration layer. You can also browse deployable templates in our AI agent library to shortcut the build.

Frequently Asked Questions

What is agentic AI technology?

Agentic AI technology is AI that takes actions rather than just generating text — calling tools, querying databases, updating records, and making sequential decisions to complete a goal. Instead of answering a question, an agent might read a support email, look up the order in your CRM, check inventory, issue a refund, and send a confirmation. It does this through function calling or MCP (Model Context Protocol) tool integrations, coordinated by an orchestration framework like LangGraph, CrewAI, or AutoGen. The key distinction from a chatbot is autonomy over a multi-step workflow. In production, effective agentic AI is heavily scoped and guarded — the most reliable deployments pair a narrow model with strong validation and human handoffs rather than granting broad autonomy. Start narrow, instrument everything, and widen scope only as observability data proves the system reliable.

How does multi-agent orchestration work?

Multi-agent orchestration coordinates several specialized agents — each handling a subtask — toward a shared goal, managed by a controller that routes work, passes state, and reconciles outputs. A framework like LangGraph models this as a graph of nodes and edges, where each node is an agent or tool call and edges define the flow and branching logic. CrewAI uses a role-based model (researcher, writer, reviewer), while AutoGen emphasizes conversational agent-to-agent messaging. The orchestration layer is where end-to-end reliability is won: it adds validation checkpoints, retries, and escalation between steps so that compounding per-step error doesn't sink the whole pipeline. In practice, keep the agent graph as simple as the task allows — Anthropic's engineering guidance explicitly warns against unnecessary multi-agent complexity. Many workloads that look multi-agent are actually a single agent with good tools.

What companies are using AI agents?

86% of enterprises have deployed AI agents in some form, per Boomi's 2026 research, though only 34% fully trust them. Across the mid-market, common deployments include ecommerce brands automating support triage and order processing, insurers using agents for claims intake, agencies automating content and research workflows, and logistics firms handling exception management. Tooling vendors like Salesforce (Agentforce), Microsoft (Copilot agents), and ServiceNow have embedded agents into their platforms, while technical teams build custom agents on LangGraph, CrewAI, and AutoGen. The differentiator is rarely the industry — it's whether the company solved the AI Coordination Gap: reliable retrieval, validation, and human handoffs. Companies that treated agents as a systems-integration challenge succeeded; those that treated it purely as a model-quality challenge landed in the 66% that don't yet trust their deployments.

What is the difference between RAG and fine-tuning?

RAG injects live facts into the model's context at query time by retrieving them from a vector database; fine-tuning changes the model's weights by training it on your examples. Use RAG for knowledge that changes frequently — inventory, pricing, policies, customer records — because you update the database, not the model. Use fine-tuning to change the model's behavior, format, tone, or to make a small SLM excel at a narrow task. They're complementary, not competing: a common production pattern is a fine-tuned 7B SLM (for consistent behavior and low cost) combined with RAG (for live facts). A useful rule: if the answer depends on data that changes, reach for RAG; if you're teaching the model how to respond or specializing it for a repetitive task, reach for fine-tuning. Most reliability problems blamed on the model are actually retrieval problems — fix RAG first.

How do I get started with LangGraph?

Install it with pip install langgraph, then model your workflow as a state graph: define a shared state object, create nodes (each a function that reads and updates state), and connect them with edges including conditional branches. Start with a single-agent graph solving one real task — say, support ticket triage — before adding complexity. Add a validation node that checks model output against a Pydantic schema before any action commits, and a conditional edge that routes low-confidence cases to a human. Wrap everything in LangSmith tracing from day one so you can see exactly where steps fail. The LangGraph documentation on python.langchain.com has runnable quickstarts. A practical first project: build the hybrid router shown earlier in this article — an SLM handles common intents, escalates complex ones to a frontier LLM. Ship it small, instrument it heavily, and grow the graph only as production data justifies each addition.

What are the biggest AI failures to learn from?

The most instructive AI failures are systemic, not model-based. Air Canada's chatbot committed the airline to a refund policy it invented, and a tribunal held the company liable — a guardrail and validation failure, not a reasoning one. Numerous enterprises have shelved agent projects after a single high-value error destroyed stakeholder trust, illustrating why aggressive human handoffs early matter more than autonomy. The most common quiet failure is the compounding-reliability trap: a six-step pipeline at 97% per step delivers only 83% end-to-end, and teams ship it without measuring the compound number. The lesson across all of these is that AI failures are overwhelmingly coordination failures — missing validation, unchecked outputs writing to systems of record, no escalation path, and no observability. Instrument end-to-end reliability, validate every action before it commits, and design the human handoff deliberately. Models rarely fail alone; systems around them do.

What is MCP in AI?

MCP (Model Context Protocol) is an open standard that standardizes how AI models connect to external tools, data sources, and systems. Introduced by Anthropic and now broadly adopted, it replaces bespoke, per-model integrations with a single common interface. Before MCP, every integration between a model and a tool — your CRM, a database, an internal API — had to be rebuilt for each model and framework. MCP defines a common interface so that a tool exposed once works across different models and orchestration frameworks. Practically, this means you can build your tool layer independently of your model choice, then swap a custom SLM and a frontier LLM behind the same tools without rewriting integrations. That decoupling is precisely what makes the hybrid SLM/LLM routing pattern feasible for mid-market teams in 2026. MCP turned agentic AI from fragile custom demos into composable production systems, and it's a foundational reason the Tool Layer in the AI Coordination Gap framework is now portable across models.

About the Author

Rushil Shah

AI Systems Builder & Founder, Twarx

Rushil Shah is the founder of Twarx and an AI systems builder who has spent years designing autonomous workflows, multi-agent architectures, and AI-powered business tools. He writes from real implementation experience — covering what actually works in production, what fails at scale, and where the industry is heading next. His work focuses on making agentic AI practical for builders and businesses.

LinkedIn · Full Profile

---

This article was originally published on Twarx. Follow for daily deep dives on AI agents and automation.