DEV Community: MargIq

Why One Model per AI Workflow Is the Wrong Abstraction

MargIq — Mon, 27 Jul 2026 19:22:36 +0000

Most AI applications make model selection a configuration decision:

const response = await client.chat.completions.create({
  model: "openai/gpt-5",
  messages,
});

A team chooses a capable model during development, ships it, and eventually every
request in that code path flows through the same model.

This is understandable.

The stronger model works. Replacing it could create subtle quality regressions.
Generic model benchmarks rarely represent the application's actual business
requirements, and evaluating every cheaper alternative takes time.

But as traffic grows, that safe decision becomes an infrastructure tax.

I have been building MargIQ around a simple thesis:

Model selection should be based on what a business workflow requires—not only
generic benchmarks, token count, or the model a developer originally chose.

The model that is right for one workflow may be wasteful for another. And even
inside one workflow, the model that is right for a routine request may be unsafe
for the next one.

"Use a cheaper model" is not a solution

Consider four common AI workflows:

Classifying an incoming support request
Extracting structured fields from an invoice
Summarizing a long internal policy
Analysing a possible security incident

They may all use the same API and provider, but they do not have the same
requirements.

Before changing the model, we need to understand the business task:

Must the output satisfy a strict JSON schema?
Does the model need to call tools with valid arguments?
How costly is a wrong answer?
Does the output influence a financial, legal, privacy, or security decision?
Does tone matter?
Is the context unusually long?
Which fields or behaviours must always remain correct?
When must the workflow escalate to a person?

A model can perform well on a public benchmark and still be wrong for a particular
workflow.

The reverse is also true. A smaller model may be fully capable of performing a
bounded classification or extraction task. Sending that work to a frontier model
does not necessarily improve the business outcome—it may only increase the bill.

So the question is not:

Which model is best?

It is:

Which available model can satisfy the requirements of this workflow, and what
evidence do we have?

The workflow is the first useful unit of model selection

A recurring workflow contains more information than a generic prompt category.

Its system instructions, tools, response format, provider, requested model, and
message structure describe the job the application is trying to complete.

By observing that stable structure over time, a system can begin to distinguish:

Routine classification from sensitive decision support
Free-form writing from schema-bound extraction
Tool selection from direct answering
Long-context summarization from short intent routing
Low-risk operational work from requests requiring stronger judgement

That is the first layer of MargIQ.

It wraps an existing OpenAI-compatible client, identifies recurring workflows,
analyses their business and technical requirements, and evaluates models from
the application's own allowed list.

The application still requests the model it already trusts. That requested model
remains the quality anchor and fallback.

MargIQ can begin in report-only mode. It can show where a lower-cost model
appears suitable without changing production execution. Routing only becomes an
option after a workflow has earned enough evidence.

But one workflow still should not mean one model

Workflow-level model selection is better than choosing one model for an entire
application, but it is still not enough.

Individual requests inside the same workflow can have very different complexity,
risk, and consequences.

Consider an ecommerce support workflow with tools for:

Looking up an order
Creating a refund case
Escalating an issue to a human

One request says:

Can you check why my order has not shipped?

This is bounded operational work. The model needs to identify the order and call
the appropriate lookup tool.

Another request entering the same workflow reports that a damaged product caused
an injury, demands an immediate refund, and threatens public escalation.

Same application feature. Same system instructions. Same tool definitions.
Completely different consequences if the model gets it wrong.

Routing both requests to the cheapest model would be irresponsible.

Routing both to the strongest model is safe but potentially wasteful.

The workflow defines the business task. The individual request determines how
much capability and safety that particular execution requires.

That is why MargIQ's policies can contain multiple request paths inside one
workflow. The live request context can select an approved lower-cost path, retain
the requested model, or collect more evidence when the request does not match a
known path.

How the decision flow works

The important behaviour is not the happy path. It is the fallback.

If the workflow is unknown, the policy is not ready, a live request does not
match an approved path, or the request is risky or ambiguous, the application
uses the requested model.

What I found in a controlled benchmark

The current MargIQ sandbox defines 15 workflow families, including:

Intent and support classification
Invoice and CRM extraction
Policy and meeting summarization
Customer-support writing
Review moderation
Lead qualification
PII handling
Code-review triage
Multi-turn customer chat
Tool-based ecommerce support
Dynamic request-shape and security-neighbour tests

It also contains 21 explicit evaluation scenarios covering paraphrases,
multilingual inputs, noisy OCR, long context, security neighbours, credential
leaks, and authorization vulnerabilities.

In one retained 58-request controlled benchmark snapshot, MargIQ detected seven
recurring workflows. Two policies became active at 0.96 confidence.

Two transaction-level routing decisions contained complete savings evidence:

Workflow path	Requested model	Selected model	Measured model-cost reduction
Low-risk intent classification	GPT-5	GPT-4o mini	90.7%
Routine order lookup with tools	GPT-5	GPT-4.1 mini	74.8%

The higher-risk refund and safety request in the same ecommerce workflow remained
on GPT-5.

Another support-classification workflow was blocked from optimization because
candidate outputs disagreed on decision-bearing fields and the benchmark did not
contain an authoritative quality definition for resolving the difference.

That blocked workflow matters as much as the savings examples. A routing system
should not convert uncertainty into a cost-reduction claim.

What these numbers do—and do not—prove

This was controlled sandbox traffic, not customer production traffic.

Only two requests in the retained snapshot provide complete transaction-level
savings evidence. The 90.7% and 74.8% figures describe those verified paths, not
an equivalent reduction across every request.

Across all 58 requests, the measured saving was only 0.1594%. Most of that traffic
was gathering learning evidence, running report-only, retaining the requested
model, or exercising workflows that had not earned an active policy.

That aggregate result should not be hidden. It shows the cold-start cost of an
evidence-based system.

If the same approved paths process more requests with similar token profiles,
dollar savings should scale with their eligible volume. But production savings
will always depend on:

The proportion of spend belonging to eligible workflows
Input and output token distribution
The mix of routine and risky requests
Available candidate models
Provider pricing
The quality threshold the business requires
How often workflows change

The benchmark demonstrates the decision mechanism. It is not yet a production
case study.

The routing is not the hardest part

Calling a cheaper model is trivial.

Proving that it can perform a particular business workflow safely is much harder.

This is why I increasingly believe:

There is no dependable AI system without an evaluation harness.

For each workflow, MargIQ derives a quality contract describing the requirements
candidate outputs must satisfy.

Depending on the workflow, that can include:

Output and schema validity
Required fields
Tool names and argument structure
Task-correctness constraints
Safety requirements
Business-critical decisions
Escalation behaviour
Output fidelity

Candidate models are evaluated against that contract before routing becomes
eligible.

Deterministic checks can verify machine-readable requirements such as JSON,
schemas, and tool arguments. Evaluator models can assess requirements that cannot
be reduced to syntax alone. The candidate is assessed against the workflow
contract rather than treated as correct merely because it resembles the
requested-model output.

If evidence is insufficient, an evaluation is ambiguous, a critical constraint
fails, or the workflow cannot be evaluated reliably, MargIQ retains the requested
model.

The harness is not MargIQ's primary value proposition.

The primary value is intelligent, workflow-aware model selection—including
different model decisions for different requests inside the same workflow.

The harness is what gives the system permission to make those decisions.

Without it, model optimization can easily become quality degradation with better
accounting.

What I am still trying to learn

MargIQ is early. The problems I am thinking about now include:

How stable should workflow identity remain as prompts evolve?
How much evidence is enough before a routing path activates?
How should a system detect semantic drift after activation?
Which quality requirements can be evaluated automatically?
When should a human define the authoritative business rule?
How should cost, latency, and quality trade-offs be presented without creating false certainty?

I would especially value criticism from developers operating multi-model systems
in production. I am interested in where this abstraction breaks, which failure
modes are missing, and what evidence you would require before trusting automatic
model selection.

Project context: MargIQ

Editorial disclosure: I used an AI assistant to help structure and edit this
article. The architecture, implementation details, benchmark results, and
conclusions are based on my own work and were checked against retained project
evidence before publication.

I built MargIQ to learn which AI workflows actually need expensive models

MargIq — Mon, 13 Jul 2026 19:21:29 +0000

Stop treating every AI workflow the same: optimize models where it actually matters

Most AI applications make one model decision for the entire product.

A support ticket classifier, an invoice extractor, a refund decision, and a security response may all be sent through the same powerful model.

That feels safe, but it creates two problems:

Routine work becomes unnecessarily expensive.
Blindly switching everything to a cheaper model can reduce quality where it actually matters.

The real question isn't:

Which model is cheapest?

It's:

Which model is appropriate for this specific workflow, given its complexity, risk, and observed behaviour?

What I built

I built MargIQ to help answer that question using evidence from actual application traffic.

MargIQ identifies recurring AI workflows and evaluates them against the models already available in your application.

For each workflow, it can show:

Where a lower-cost model may be appropriate
Where the requested model should remain
Which routing paths have enough evidence
Where quality requirements are too ambiguous to recommend a change
The estimated or realized cost impact

The customer-facing unit is the workflow, rather than an individual prompt or a global model setting.

Quality protection matters

Reducing model cost is only useful if the application remains reliable.

When MargIQ does not have sufficient evidence, it keeps the requested model.

It also protects workflows where multiple outputs may all be defensible because the application has not clearly defined an important taxonomy, priority rule, or expected response structure.

Instead of repeatedly testing models or silently choosing a cheaper option, MargIQ explains what needs clarification before making a recommendation.

How it integrates

MargIQ is designed for server-side AI applications using compatible model-provider clients.

Getting started is as simple as:

npm install margiq

You keep your existing provider credentials and model configuration.

MargIQ works with the models your application already uses rather than requiring a specific provider.

The free plan starts in Report-only mode. It observes recurring workflows and reports potential savings without changing production routing.

When you're ready, workflow controls let you choose how optimization is applied:

Automatic — Applies supported routing paths automatically.
Report only — Continues analysis without changing requests.
Disabled — Excludes the workflow entirely.

What I'm looking for

MargIQ is now live, and I'd love feedback from founders and engineers running recurring AI workflows in production.

In particular, I'm interested in:

What evidence would you need before trusting a workflow-level model recommendation?
Does the workflow report explain the quality vs. cost trade-off clearly?
Is the server-side integration straightforward for your existing stack?

You can check it out here:

https://getmargiq.com

I built MargIQ because I believe AI cost optimization should be based on workflow evidence and business risk, not a blanket instruction to "use a smaller model."

I'd genuinely appreciate any feedback or suggestions from people building production AI systems.