DEV Community: Hardik Mehta

Why JSON Schema Isn't Enough for Production AI

Hardik Mehta — Mon, 06 Jul 2026 10:09:07 +0000

Your LLM Can Return Perfect JSON and Still Be Completely Wrong

Most developers (including me) assumed one thing when Structured Outputs became available:

If my JSON validates against the schema, my extraction is correct.

Unfortunately, that's not true.

After spending the last few months building ShapeCraft, a structured output library for Node.js that works across OpenAI, Claude, Ollama, and Groq, I realized something important:

JSON Schema guarantees structure—not truth.

Once you understand that distinction, you'll design much more reliable AI systems.

Let's look at an example

Suppose you're extracting data from an invoice.

The invoice says:

Invoice Total: $900

Your LLM returns:

{
  "invoiceTotal": 1200
}

Now let's validate it.

Valid JSON? Yes
Matches JSON Schema? Yes
Correct data type? Yes
Required field present? Yes
Correct answer? No

The response is structurally perfect but factually wrong.

That's the difference between structural correctness and semantic correctness.

JSON Schema only solves one problem

JSON Schema is excellent at validating structure.

It ensures things like:

Valid JSON
Required fields
Correct data types
Arrays and objects
Enum values
Consistent response shape

That's incredibly useful.

But JSON Schema cannot answer questions like:

Did the model extract the correct value?
Is this value actually present in the source document?
Did the model hallucinate?
Can I trust this extraction?

Those are semantic questions.

Structural Validation vs Semantic Validation

Structural Validation	Semantic Validation
Valid JSON	Correct value
Required fields	Grounded in source
Correct data types	No hallucination
Schema compliance	Evidence exists
Safe to deserialize	Safe to trust

Most AI SDKs solve the first column.

Production AI requires both.

Not all structured output works the same way

While building ShapeCraft, I discovered something else that surprised me.

Every provider advertises structured output, but they don't enforce it the same way.

Provider	Implementation	Guarantee
OpenAI	Server-side schema enforcement	Native
Groq	Server-side schema enforcement	Native
Ollama	Grammar-constrained decoding (GBNF)	Constrained
Anthropic	Prompt + parse + retry	Best-effort

They all produce structured output.

But they provide different guarantees.

That's an important distinction that's often hidden behind a single API.

Structure ≠ Truth

Even with the strongest schema enforcement available today, a model can still return:

{
  "customerName": "John Smith",
  "invoiceTotal": 1200,
  "invoiceDate": "2026-07-01"
}

The JSON is valid.

The schema passes.

The data types are correct.

The answer can still be wrong.

No JSON Schema validator can detect that.

What production AI actually needs

I now think of AI validation as two independent layers.

Layer 1 - Structural Validation

Questions like:

Is this valid JSON?
Does it match my schema?
Can my application deserialize it safely?

This is where JSON Schema shines.

Layer 2 - Semantic Validation

Questions like:

Is the extracted value correct?
Can this answer be traced back to the original source?
Does evidence exist?
Should I trust this response?

This layer is significantly harder.

And it's where I believe the next generation of AI tooling needs to evolve.

Building ShapeCraft

While exploring these differences, I built ShapeCraft.

ShapeCraft is an open-source TypeScript library that provides a unified structured output API across multiple providers while exposing the actual guarantee level instead of pretending every backend behaves the same.

import { generate, openai } from "@aviasole/shapecraft";

const result = await generate(
  openai({ model: "gpt-4o-mini" }),
  schema,
  prompt
);

console.log(result.guaranteeLevel);

// "native"
// "constrained"
// "best-effort"

Current providers:

OpenAI
Groq
Ollama (GBNF constrained decoding)
Anthropic

Supported schema types:

Zod
JSON Schema
XML
Regular Expressions
Custom Validators

Streaming, retries, and multi-turn structured conversations are also supported through the same API.

GitHub:

https://github.com/aviasoletechnologies/shapecraft

npm:

npm install @aviasole/shapecraft

Where I think AI tooling is heading

Structured output solved one of the biggest problems in LLM development.

Before it, we spent countless hours:

Fixing malformed JSON
Cleaning invalid responses
Writing regex parsers
Handling missing fields

That problem is largely solved.

The next challenge isn't better JSON.

It's better confidence.

Imagine every response also including:

Evidence for every extracted field
Field-level confidence scores
Source attribution
Semantic verification
Reliability metrics

That's where I believe production AI is heading.

Final thoughts

Structured output is one of the biggest improvements in modern LLM APIs.

But it's important to understand what it actually guarantees.

It guarantees structure.

It does not guarantee correctness.

Once I separated those two ideas, it completely changed how I think about designing reliable AI systems.

How are you validating AI output in production today?

Schema validation only?
Human review?
Semantic verification?
Something else?

I'd love to hear how others are solving this problem.

Stop Guessing: Guaranteed Structured Output from LLMs in Node.js

Hardik Mehta — Wed, 01 Jul 2026 06:02:31 +0000

Stop Parsing LLM JSON. The Backend Already Knows Better.

If you've built anything with LLMs, you've probably written code like this:

const raw = await openai.chat.completions.create({ ... });

let parsed;

try {
  parsed = JSON.parse(raw.choices[0].message.content);
} catch {
  // retry?
  // strip markdown?
  // log?
  // cry?
}

Then you added retry logic.

Then the model started wrapping JSON in Markdown code fences.

Then it added a trailing comma.

Then you added another regex.

Eventually your structured output pipeline became more complicated than the feature you were trying to build.

The interesting part is:

This isn't actually an LLM problem.

It's a backend capability problem.

Not all structured output is equal

Most libraries expose a single "structured output" API regardless of which model you're using.

Under the hood, those providers work very differently.

OpenAI

OpenAI performs server-side schema enforcement.

If the model doesn't satisfy the schema, the API won't return malformed JSON.

Guarantee: Native.

Groq

Groq provides a similar native structured output mechanism.

Again, validation happens before the response reaches your application.

Guarantee: Native.

Ollama

This is where things get interesting.

Ollama supports GBNF grammar constraints.

Instead of validating JSON after generation, the grammar is applied during token sampling.

That means the model literally cannot emit a token that violates your schema.

There is no malformed JSON to parse.

There is no retry loop because of formatting mistakes.

There is no JSON.parse() risk.

For local models, this is probably the strongest structured output guarantee currently available.

Anthropic

Anthropic currently has no equivalent enforcement mechanism.

The common approach is:

Prompt carefully
Parse the response
Validate
Retry if necessary

It works well, but it's still best effort, not guaranteed.

Why this matters

Many libraries abstract all providers behind exactly the same API.

That sounds convenient.

But it also hides an important architectural difference.

Treating every provider the same means you're either:

trusting outputs more than you should, or
ignoring guarantees your backend already provides.

Knowing the guarantee level lets you design better systems.

For example:

Fewer retries
Simpler parsing
Better observability
More predictable production behavior
Cleaner error handling

A practical example

Here's a structured output example using Ollama with Zod.

import { generate, ollama } from "@aviasole/shapecraft";
import { z } from "zod";

const result = await generate(
  ollama({ model: "llama3.2" }),
  z.object({
    name: z.string(),
    score: z.number(),
  }),
  "Rate this essay."
);

console.log(result.data);

The returned object is fully typed, and with Ollama the schema is enforced through GBNF grammar during generation.

Different backends, different guarantees

Thinking about structured output in terms of guarantee levels makes provider behavior much easier to reason about.

Backend	Guarantee	Enforcement
OpenAI	Native	Server-side schema validation
Groq	Native	Server-side schema validation
Ollama	Constrained	Token-level GBNF grammar constraints
Anthropic	Best effort	Prompt + validation + retries

The API surface might look identical.

The reliability characteristics are not.

Building around guarantee levels

I've been experimenting with making these differences explicit rather than hiding them.

In the library I maintain, every generation returns the guarantee level alongside the parsed result.

const result = await generate(model, schema, prompt);

result.guaranteeLevel;
// "native"
// "constrained"
// "best-effort"

Instead of pretending every backend behaves the same, your application can make informed decisions based on the actual guarantees available.

Final thoughts

Structured output isn't just about getting valid JSON.

It's about understanding how that JSON became valid.

There's a big difference between:

"the model happened to follow the prompt,"
"the server enforced the schema," and
"the model physically could not generate an invalid token."

Those differences matter once you're building production systems.

Disclosure: I'm one of the maintainers of ShapeCraft, an open-source library that exposes structured output guarantee levels across OpenAI, Groq, Ollama, and Anthropic instead of treating every backend as identical.

I'd love to hear how you're handling structured output reliability in production.

Why We Read the Legacy System Before Touching It: A Healthcare Modernization Story

Hardik Mehta — Tue, 23 Jun 2026 07:25:24 +0000

The Call That Started Everything

The client's CTO opened the call with a concrete problem: their prescription processing system was failing on roughly 40% of real-world inputs, routed to a manual review queue that had become load-bearing infrastructure. The developer who had built the original system had left 18 months ago. A mobile app integration was due in 90 days.

We asked to see the codebase before saying anything else.

What we found was a system built on a reasonable 2019 commercial OCR library, surrounded by years of conditional logic that had accumulated without any of it being documented. Each addition had made sense at the time - a fix for handwritten abbreviations, a workaround for a specific clinic's prescription format, a retry loop for low-confidence scans. Taken together, they had created something no current team member could reason about in full.

The 40% failure rate was not a sign the original system was poorly designed. It was a sign the system had drifted far enough from its original design assumptions that it needed to be replaced.

The business impact was clear: one operations staff member was doing nothing but processing the manual queue. New features were blocked because any change to the conditional logic was too risky without documentation. The system was costing more to maintain than to have originally built it differently.

Why Two Prior Attempts Had Failed

McKinsey research consistently finds that companies spend between 60 and 80 percent of their IT budgets maintaining existing systems rather than building new ones. The client had tried to break out of that cycle twice.

The first attempt was a rewrite from scratch. A contractor built a new OCR system over four months. It was technically cleaner - but it didn't match the behavior of the legacy system on the inputs the business actually received. Medicine names the legacy system had resolved were missed. Patient identity edge cases that the legacy system had handled silently broke. The contractor had no behavioral baseline to compare against, so they had no way to know what was missing until users reported failures. The rewrite was abandoned.

The second attempt was a vendor replacement - an off-the-shelf healthcare OCR API. Accuracy on clean inputs was better than the legacy system, but the API returned raw extracted text, not structured data. Integrating its output into the existing database schema would have required rebuilding most of the downstream application. The integration cost exceeded the modernization budget.

Both attempts shared the same root cause: the team tried to replace the system before they fully understood what it was doing.

Gartner estimates that 70 percent of modernization projects fail for this reason - not because the replacement technology was wrong, but because the team didn't have a reliable map of the system's actual behavior before they started replacing it.

Step One: A Behavioral Baseline, Not a Code Analysis

The first thing we did was add logging to the production system.

Over two weeks, we captured approximately 2,000 real prescription submissions - the input image, every conditional branch triggered, every medicine name resolved or failed, the patient identity outcome, and the final database write or queue assignment. This gave us a behavioral baseline: a record of what the system actually did on real production inputs, across the full range of formats it encountered.

This golden set became the behavioral contract for the entire migration. Every replacement component we built had to match or improve on the legacy system's outputs on this set before handling production traffic.

We then used LLM-assisted analysis to read and map the codebase - every conditional branch, every fallback path, every hardcoded value. The output was a plain-language map of what each component did.

Here is where we want to be precise about what that analysis produced, because this is where projects go wrong when they over-rely on AI tooling:

AI can describe what code does. It cannot explain why a rule was added.

Consider an example:

if customer_id == 1234:
    skip_verification()

An LLM analysis correctly identifies this as a verification bypass for a specific customer ID. It cannot tell you whether that customer is a test account, a long-standing enterprise client with a contractual exception, a developer shortcut that never got removed, or a compliance workaround for a specific regulatory situation. That answer lives in someone's memory, a Slack thread, or a contract - not in the code.

We found several rules like this in the client's system. For each one, we had to go to the remaining team members and, in some cases, dig through email history to understand the original intent. Some rules were still valid. Two had been superseded by changes elsewhere in the system and could be safely removed. One reflected a client-specific exception that needed to be replicated exactly in the replacement.

The LLM analysis compressed the discovery process from weeks to days. The stakeholder validation work - confirming why each rule existed and whether it was still needed - could not be compressed. That part required humans with institutional knowledge, and no tooling changes that.

Three findings from the combined analysis shaped the migration plan:

Finding 1: The 40% queue rate was not random. It clustered around three input types - handwritten prescriptions, regional brand-name drugs not in the hardcoded medicine list, and poor-lighting photographs. The rest of the system handled its inputs correctly. Most of it did not need to be replaced.

Finding 2: The medicine name lookup was the highest-leverage failure point. The static list had been manually maintained and had coverage gaps for regional generics and newer branded molecules. Every failure that reached the manual queue traced to a name the list didn't recognize.

Finding 3: The patient identity resolution logic was correct. It handled multi-patient households, ambiguous name matches, and confirmation flows accurately. Replacing it would introduce risk with no corresponding benefit.

The Replacement Strategy: What Moved, What Stayed, What Changed

We need to be direct about something that often gets glossed over in modernization case studies: you don't eliminate complexity, you move it.

The old system handled edge cases through conditional logic in the application code. The new system handles those same edge cases - they have to go somewhere - through Vision LLM prompt design, RAG pipeline threshold tuning, and agent orchestration logic. The complexity still exists. What changed is where it lives and how maintainable it is.

In the old system, complexity lived in undocumented conditional branches. When something broke, the only way to diagnose it was to read the code and guess which branch had triggered. When a new edge case appeared, a developer added another branch. There was no systematic way to test whether a change had broken anything else.

In the new system, complexity lives in components that can be independently versioned, tested, and monitored. A prompt change is reviewable. A RAG threshold change can be validated against the golden set before deployment. An agent tool-call failure produces a structured log entry that can be replayed. The diagnostic process changes from reading undocumented code to querying structured logs.

That's the actual value proposition - not "less complexity" but "complexity in a form you can work with."

The migration itself had four phases, each independently testable and each with its own rollback path:

Phase 1: Replace the OCR engine with a Vision LLM.

The commercial OCR library output raw character strings with no understanding of document structure. A Vision LLM (we used Google Gemini) extracts structured fields - medicine name, patient name, dosage, frequency, clinic - in a single pass, handling handwritten text, abbreviations, and variable image quality.

What this moved: the conditional logic that existed to normalize raw OCR output - handling abbreviations, splitting combined entries, cleaning whitespace - became part of the extraction prompt and the validation layer. The branching is still there; it's in the prompt design and the structured output schema instead of the application code.

Before deploying to production, we ran the Vision LLM against the full 2,000-item golden set and compared outputs against what the legacy OCR had produced. Zero regressions on the inputs the legacy system had processed correctly. Significant improvement on the inputs it had failed.

Rollback path: the legacy OCR engine ran in parallel for four weeks. Any prescription returning a Vision LLM confidence score below the calibrated threshold was automatically rerouted to the legacy OCR path.

Phase 2: Replace the static medicine list with a four-stage RAG pipeline.

A static list cannot expand automatically. The new pipeline has four sequential stages: in-memory cache for high-frequency drugs, exact database match, trigram fuzzy search for OCR noise and spelling variants, and vector semantic search for phonetic variants and partial matches. Medicines that pass no stage are queued for automated scraping and added to the database.

What this moved: the coverage decisions that had been made implicitly by choosing which drugs to add to the static list became explicit in the RAG pipeline - threshold tuning, confidence cutoffs, the definition of "close enough" for fuzzy matching. These decisions are now documented in code and tunable from data rather than locked in a spreadsheet someone last updated nine months ago.

Threshold tuning required significant calibration against the golden set. Too permissive on fuzzy matching produces false positives - a medicine name matched to the wrong drug. Too strict increases the queue rate. We iterated through several threshold combinations until the new pipeline matched the legacy list's resolution rate on clean inputs while outperforming it on the noisy inputs that had previously gone to the manual queue.

Rollback path: the legacy medicine list remained as a fallback for any name the new pipeline returned below the confidence threshold.

Phase 3: Wrap the patient identity logic, don't replace it.

The behavioral analysis confirmed this component was correct. We extracted it, wrote a behavioral test suite documenting its outputs for every input class in the golden set, and wrapped it behind a clean API interface. The new system called it identically to how the old system had.

This was the single highest-leverage decision in the project. Replacing a working component to make it cleaner is one of the most common sources of regression in modernization projects. If the behavior is correct and the blast radius of a change is high, wrap it and move on.

Phase 4: Replace the accumulated conditional logic with a ReAct agent.

The legacy system processed prescriptions through a linear pipeline with hardcoded fallbacks and no structured logging. Debugging a production failure required reading the code and tracing which branch had triggered - which was difficult because the branches interacted in ways that weren't documented anywhere.

The ReAct agent orchestrates the extraction and matching steps with complete tool-call logging. Every tool call is recorded with its input and output. Any production failure can be replayed exactly from those logs. The complexity of the orchestration is still present - the agent has to make decisions about which tools to call and in what order - but those decisions are visible and testable.

The Traffic Migration: Six Weeks of Parallel Operation

We did not cut over to the new system on a date. We migrated traffic incrementally over six weeks, with automatic fallback at every stage.

Weeks 1–2: 5% of traffic. The old system handled the rest. Output discrepancies between the two systems were logged and reviewed daily. The most common discrepancy was medicine name formatting - the new system returned canonical names where the legacy system had returned raw OCR output. These weren't errors, but they required downstream validation to confirm they were improvements rather than regressions.

Weeks 3–4: 25% of traffic. We calculated per-stage hit rates for the medicine RAG pipeline on live inputs - not just the golden set. The vector search stage was handling 11% of resolutions, within the expected 8–15% range, indicating the pipeline was behaving as designed on real production inputs.

Week 5: 50% of traffic. The manual queue rate dropped from 40% to under 8% of total submissions. The operations staff member assigned to the queue was transitioned to other work during this week.

Week 6: 100% of traffic. The legacy system was kept warm for two additional weeks before decommissioning.

The data migration - moving years of patient records, prescription scans, and medicine match histories from the old schema to the new one - ran as a separate workstream after the application layer was stable. We used a read-from-old, write-to-new pattern: new records written to both systems in parallel during a two-week overlap period, with nightly reconciliation checks comparing row counts and spot-checking field values against known records. No cutover happened until reconciliation passed on three consecutive nights.

Observability: What We Tracked and Why

AI systems require different monitoring than deterministic systems. You're not just watching for errors - you're watching for distribution shifts that indicate something upstream has changed.

We tracked four things continuously after go-live:

Per-stage hit rates in the medicine pipeline. If the in-memory cache dropped from 35% to 2%, either the cache wasn't warming correctly or the input distribution had shifted. If the vector search stage climbed above 20%, the exact and fuzzy stages were missing inputs they shouldn't be missing. Alerts fired on sustained deviation from established baselines, not on individual anomalies.

Vision LLM extraction confidence scores. We tracked the distribution of per-field confidence scores over time, with alerts on a shift in the low-confidence tail. A gradual drift here is a leading indicator that input quality is changing - new prescription formats, different image types - before it shows up as an increased queue rate.

Queue rate as a system health proxy. The rate of prescriptions entering the manual confirmation queue was the most reliable single indicator. A baseline of under 8% was established from the golden set. Sustained queue rate above 15% triggered an incident review.

Agent tool-call latency by stage. The ReAct agent logs every tool call with timing. When latency spikes, we can see exactly which stage caused it - something that was completely invisible in the legacy system, which had no structured instrumentation at all.

Every production failure was replayable: the agent's full tool-call trace, the exact Vision LLM output, and the input image were all logged. Diagnosing a production issue typically took minutes rather than the hours required with the original system.

Results

By end of week six:

Manual queue dropped from 40% to under 8%. The remaining 8% are genuinely ambiguous cases - new prescription formats, heavily degraded images, medicines with no prior occurrence - that legitimately require human judgment. The original 40% was the system failing on inputs it should have handled automatically.

Medicine identification accuracy reached approximately 95 percent. The four-stage pipeline resolved brand names, generics, abbreviations, and OCR noise that the static list had been failing on for two years. Coverage improves continuously as the auto-scraper adds new medicines.

Zero OCR data lost during identity confirmation. The original system discarded prescription data silently if identity resolution failed. The new system holds unresolved prescriptions in a confirmation queue while returning medicine data immediately - no input is lost.

The mobile app launched within 90 days. The modernization produced a documented API as a byproduct. The previous system had no API - every integration had coupled directly to the database. The new API allowed mobile frontend development to run in parallel with the final weeks of traffic migration.

Full technical architecture is in the OCR prescription processing case study.

What We'd Do Differently

Build the golden set before the scoping conversation. We added logging to the legacy system at the start of engagement and waited two weeks before beginning analysis. That was the right sequence, but the two-week wait could have started before we were formally engaged - earlier data means more edge cases in the baseline.

Budget more time for stakeholder rule validation. The rules that required human explanation to understand - the exceptions that existed for business or political reasons, not technical ones - took longer to validate than we had initially estimated. In a system with years of accumulated logic, assume several of the rules have non-obvious histories that a developer interview won't immediately surface.

Negotiate a longer parallel operation window for higher-stakes systems. Six weeks was sufficient here. For a system with higher transaction volume, stronger regulatory requirements, or more complex failure modes, eight to ten weeks would have been more comfortable.

What This Pattern Looks Like in Other Contexts

The sequence - behavioral baseline first, component-by-component replacement with rollback paths, incremental traffic migration, continuous observability - applies wherever critical systems have accumulated undocumented complexity.

In banking, IBM estimates 95 billion lines of COBOL are still in active production, processing approximately $3 trillion in daily transactions. The challenge isn't the technology - it's that golden-set creation at scale requires instrumentation before any replacement begins, and the stakeholder validation of extracted business rules is more time-consuming as systems get older and institutional memory degrades.

In insurance, mainframe policy calculation engines with pricing rules from the 1990s present the same stakeholder challenge in more acute form. Some of those rules exist because of regulatory requirements that have since changed, or client exceptions that were negotiated long before anyone currently at the company started. AI can surface the rules. Humans have to determine which ones are still valid.

McKinsey Digital documents that successful legacy modernizations typically see 20 to 30 percent reductions in IT operating costs. The technical approach matters, but the bigger variable is whether the team invested in a behavioral baseline and stakeholder validation before writing replacement code. The teams that skip those steps are the ones rebuilding for a third time.

If Your Organization Is in This Situation

The indicators we look for: a system that's been in production for more than two years, patched by more than one developer, with no behavioral test suite, producing operational cost through manual workarounds or blocking new features because changes feel too risky.

If that describes something you're carrying, the starting point isn't planning a replacement. It's building the behavioral baseline - logging production inputs and outputs for two to four weeks, so you have a ground-truth record of what the system actually does before you decide what to replace.

Talk to the Aviasole engineering team about what that looks like for your system, or review the OCR prescription processing case study for the full technical breakdown.

AI Automation Gives You Different Answers Than Your Old Software - Here's Why That Matters

Hardik Mehta — Thu, 18 Jun 2026 05:54:15 +0000

A logistics manager at a mid-size distributor spent three weeks trying to understand why their accounts payable process had become inconsistent. Their ERP - in place for eight years - approved a vendor invoice in under two seconds. Their new AI-powered automation platform flagged the same invoice as a potential duplicate. Both systems had access to the same data. Both vendors claimed high accuracy. The manager couldn't reconcile the two verdicts, so she escalated to her CFO, who spent half a day investigating an invoice that should have taken two seconds.

The systems weren't malfunctioning. They were working exactly as designed. The problem was that nobody had explained to the manager - or her CFO - what kind of software each one actually was.

What Rule-Based Software Actually Does

Legacy software - ERPs, CRMs, compliance checkers, billing systems - is built on explicit rules written by developers. Every decision traces back to a human-authored condition: if this, then that.

An accounts payable rule might look like: if the vendor is on the approved list AND the invoice amount is under the purchase order cap AND no matching invoice number exists in the last 90 days → approve. If any condition fails → flag for review.

This is deterministic software. The same input always produces the same output. No exceptions. No variance. Run it a thousand times on a Tuesday and a thousand times on a Friday - you get identical results. The logic is transparent, auditable, and explainable to anyone: a developer, an auditor, a regulator, a judge.

That predictability is not a limitation. For most business-critical decisions, it is exactly what you want.

What AI Automation Actually Does Under the Hood

AI automation works differently at a fundamental level. Instead of rules written by a developer, it uses patterns learned from training data - thousands or millions of past examples - to estimate what the correct answer probably is.

The output is not a verdict. It is a probability. The model says: based on everything I learned, there is a 91% chance this invoice is legitimate. The system then converts that score into an action - approve if above threshold, flag if below.

This is probabilistic software. The same input submitted twice may not score identically. The model's confidence depends on how closely the current input resembles patterns it saw during training. Shift the input slightly - different formatting, a vendor name it hasn't seen often, an unusual amount for that category - and the score shifts. Not because the facts changed, but because the model's certainty changed.

Where the Outcomes Diverge - and Why It Costs You

The divergence becomes a real business problem in three scenarios.

When you need auditability. A regulator asks why a transaction was approved. With rule-based software, you can print the exact conditions that passed and when. With AI, the honest answer is "the model scored it above threshold." That answer does not satisfy a financial regulator, a HIPAA auditor, or a contract dispute. Some industries - financial services, healthcare, insurance - have explicit requirements for explainable automated decisions. Probabilistic systems, unless purpose-built with explainability layers, do not meet that bar.

When the cost of a wrong answer is high. Rule-based systems fail in predictable ways: if you write a wrong rule, it fails the same way every time, which makes it easy to catch and fix. AI systems fail in unpredictable ways: the model may work correctly 98% of the time and fail on a specific edge case that your testing never surfaced. In loan approvals, drug dosage calculations, or compliance checks, that 2% is not an acceptable error margin.

When consistency across teams matters. Two employees using the same rule-based system get the same answer. Two employees running the same request through an AI system at different times may get different answers - leading to confusion, inconsistency, and the exact kind of operational chaos the logistics manager ran into.

Where AI Automation Wins Anyway

None of this means legacy software is always better. Rule-based systems have a hard ceiling: they require a developer to anticipate every possible condition and write a rule for it. That works when decisions are well-defined and the input is structured. It breaks down quickly in three situations.

Unstructured input. Rules need structured data. If your workflow involves processing free-text customer complaints, classifying images, extracting information from scanned documents, or analyzing support tickets - rules cannot handle this. AI can.

Scale beyond human rule-writing capacity. Fraud detection across 10 million daily transactions involves patterns no developer could enumerate. Recommendation engines across a 500,000-SKU catalog cannot be rule-driven. When the decision space is too large to hand-code, AI's ability to learn from data is not optional - it's the only viable approach.

Decisions where human judgment was already inconsistent. If two senior reviewers in your company would disagree about the right answer 30% of the time anyway, a rule-based system gives you the illusion of consistency without the reality. AI trained on enough labeled data can sometimes outperform the inconsistent baseline.

How to Decide Which System Your Business Actually Needs

Before replacing any existing system with AI - or keeping legacy software that's genuinely holding you back - run this four-question audit on each decision point in your workflow:

1. Can you write down all the conditions needed to make this decision correctly?
If yes, a rule-based system is simpler and more reliable. If not - because the decision involves pattern recognition, unstructured data, or too many variables - AI is the right tool.

2. What is the cost of a wrong answer?
High cost (financial, legal, compliance) → keep deterministic rules, or add hard rules as constraints on top of any AI layer. Low cost at high volume (content tagging, routing, recommendations) → AI is worth the variance.

3. Does the decision need to be fully explainable to an external party?
Regulator, auditor, customer, court → deterministic rules or AI with built-in explainability. Internal optimization → AI is fine.

4. Does the input involve unstructured data?
Documents, images, free text, audio → AI. Clean structured database records → rules handle this cleanly.

Most well-designed production systems use both. Deterministic rules enforce hard constraints - you must be on the approved vendor list, the amount must be under the authorized limit. AI handles pattern recognition within those constraints - within the approved vendors, which invoices look statistically unusual compared to historical patterns?

Do You Actually Need to Rebuild, or Just Layer?

The most common mistake we see product founders make is treating this as a binary choice. They look at their legacy system, decide it's outdated, and replace it wholesale with an AI automation platform - then spend months dealing with inconsistency, explainability gaps, and edge cases the AI handles differently than the old system.

The better question is usually: which specific decisions in this workflow genuinely benefit from probabilistic intelligence, and which ones should stay deterministic?

At Aviasole, we help companies answer that question before a single line of automation code gets written. Our AI development services include workflow audits that map your decision points, identify where AI creates value versus risk, and design systems that use each approach where it actually belongs - not where it's fashionable.

Most of the time, the right architecture is not a replacement. It's a layer.

Ready to Audit Your Automation Decisions?

If you're evaluating whether to automate a workflow - or trying to understand why an existing AI system is producing inconsistent results - the first step is a decision-point audit, not a technology selection.

Talk to our team about mapping your workflow. We'll tell you honestly where AI helps, where it introduces risk, and where your legacy rules are doing exactly what they should.

MCP 2.0: The Protocol That Finally Gives AI Agents a Universal Power Outlet

Hardik Mehta — Tue, 16 Jun 2026 13:49:11 +0000

A team at a mid-size SaaS company spent six weeks building a custom integration layer so their AI agent could talk to Salesforce, Jira, Confluence, and their internal data warehouse. Four tools. Six weeks. The agent still couldn't handle OAuth token refresh without manual intervention every 48 hours.

Their competitor shipped the same capability in four days.

That gap - six weeks versus four days - is what Model Context Protocol 2.0 actually closes.

What MCP Is (and Why It Took This Long to Need It)

Model Context Protocol is an open standard released by Anthropic in November 2024 that defines a universal communication layer between AI applications and external tools, data sources, and services. Think of it as USB-C for AI agents: before USB-C, every device needed a different cable. Before MCP, every AI integration needed custom connector code.

The original spec established a clean client/server architecture: an AI application (the MCP client) discovers and calls tools exposed by MCP servers, without needing to know each tool's implementation details. Within months, official MCP servers shipped for GitHub, Slack, Notion, PostgreSQL, Google Drive, and dozens more.

But the first version had real production gaps: no standardized authorization flow, a transport layer that struggled with long-running operations, and no protocol-level way for agents to know whether a tool call was safe to retry. For demos and internal prototypes, these were manageable. For enterprise production deployments - the kind that touch customer data, need compliance audit trails, and run 24/7 - they were blockers.

The March 2025 spec update - what the community calls MCP 2.0 - fixed all three.

What Actually Changed in MCP 2.0

Three changes. Each one fixes a specific production failure mode.

OAuth 2.0 Authorization

The original MCP spec left auth up to implementers. In practice, that meant agents holding long-lived admin credentials - the path of least resistance, and a compliance nightmare.

MCP 2.0 makes OAuth 2.0 a first-class protocol element. An MCP server declares its authorization requirements; the client satisfies them before tool calls are permitted. A healthcare agent connecting to an EHR system gets scoped, time-limited tokens - not standing admin access. Compliance teams audit agent permissions through existing OAuth infrastructure. Least-privilege access becomes enforceable at the protocol level, not just a policy aspiration that gets ignored under delivery pressure.

Streamable HTTP Transport

The original transport was Server-Sent Events: useful for real-time updates, but one-directional and stateless in ways that made long-running operations awkward. If an agent triggered a 20-minute data pipeline and needed live progress updates, SSE was a painful compromise.

MCP 2.0 replaces SSE with Streamable HTTP: bidirectional, resumable, built for operations that outlast a single HTTP request. A dev tooling agent monitoring a CI/CD pipeline gets live build events without polling. An agentic workflow processing a 500-document corpus reports progress in real time without timing out.

Tool Annotations

Before MCP 2.0, an agent had no protocol-level signal that delete_customer_record was irreversible, or that get_account_balance was safe to call ten times without side effects.

Tool annotations fix this. Server authors declare whether a tool is readOnly, destructive, or idempotent. An orchestrating agent can run read-only tools in parallel and gate destructive ones behind confirmation steps - without custom guardrail logic per tool. The annotations are hints, not enforced constraints, but they give agents the information they need to behave sensibly by default.

4 Use Cases Where MCP 2.0 Changes the Equation

1. Enterprise Data Agents

An agent querying Salesforce, pulling from a data warehouse, and cross-referencing a SharePoint knowledge base used to require three separate integrations - different auth patterns, different response schemas, different error handling for each.

With MCP, the agent talks to three MCP servers using one protocol. Add a fourth data source? Stand up another MCP server - no changes to the agent. The integration surface stays flat as the tool count grows. For clients running AI agents across 5–15 internal systems, this is the difference between a maintainable architecture and a brittle custom integration that breaks every quarter.

2. Healthcare Agents with Auth-Gated Records

Healthcare is where MCP 2.0's OAuth layer matters most. An AI agent pulling patient records, lab results, and prescription history from different systems can't hold a single admin key - HIPAA requires scoped access with audit trails.

MCP 2.0's OAuth 2.0 framework delivers exactly that: per-system tokens with scoped permissions, revocable at any time, auditable through standard OAuth infrastructure. The agent gets access to what it needs. Nothing else. And every access is logged in a format compliance teams already know how to read.

3. Multi-Agent Orchestration

As AI systems move from single agents to networks of specialized agents, the communication layer between them matters as much as the agents themselves.

MCP is increasingly the protocol layer in multi-agent architectures: an orchestrating agent dispatches tasks to specialized sub-agents via MCP tool calls, collects results, and synthesizes them. The orchestrator doesn't need to know how the research agent retrieves information or how the code agent runs tests - it calls their MCP-exposed tools. This is the pattern behind the multi-agent frameworks that are actually working in production environments, not just in research papers.

4. Developer Tooling Agents

A dev agent connecting GitHub, a CI/CD system, and application monitoring used to face a painful choice: poll for updates (slow, expensive) or build custom webhooks into each tool (complex, brittle).

Streamable HTTP transport in MCP 2.0 eliminates both options. The agent subscribes to live events from each tool via its MCP server. A failed build triggers the CI/CD server to push an event; the agent receives it, queries the GitHub MCP server for the relevant diff, checks the monitoring MCP server for correlated errors, and surfaces a diagnosis - all from one real-time signal. No polling infrastructure. No custom webhook code.

Where Aviasole Fits

We build AI agents for production - not demos, not proof-of-concepts that stall at month two. MCP 2.0 is a core part of how we architect integrations for our agentic AI engagements because it solves the problems that actually kill agent projects in enterprise environments: auth that satisfies compliance teams, integrations that don't fragment when you add a new tool, and enough protocol-level safety information for agents to behave sensibly without custom guardrails per connector.

For clients working on LLM application development, MCP also changes how we think about context retrieval - moving away from bespoke retrieval connectors toward protocol-native tool servers that any agent in the stack can reuse without duplication.

The practical signal for when MCP is worth the adoption cost: if your agent calls more than two or three external systems, you're already paying the price of custom integration complexity. MCP 2.0 is now mature enough - with OAuth, streaming, and a growing ecosystem of official servers - to be the right answer for most enterprise-grade agent architectures.

Ready to Build MCP-Native Agents?

If you're evaluating MCP 2.0 for your agent architecture - or you've already started and hit friction with auth, transport, or integration sprawl - talk to our team. We'll work through the specifics of your stack and tell you honestly whether MCP solves your actual problem or whether a simpler approach fits better.