Why One Model per AI Workflow Is the Wrong Abstraction

Most AI applications make model selection a configuration decision:

const response = await client.chat.completions.create({
  model: "openai/gpt-5",
  messages,
});

A team chooses a capable model during development, ships it, and eventually every
request in that code path flows through the same model.

This is understandable.

The stronger model works. Replacing it could create subtle quality regressions.
Generic model benchmarks rarely represent the application's actual business
requirements, and evaluating every cheaper alternative takes time.

But as traffic grows, that safe decision becomes an infrastructure tax.

I have been building MargIQ around a simple thesis:

Model selection should be based on what a business workflow requires—not only
generic benchmarks, token count, or the model a developer originally chose.

The model that is right for one workflow may be wasteful for another. And even
inside one workflow, the model that is right for a routine request may be unsafe
for the next one.

"Use a cheaper model" is not a solution

Consider four common AI workflows:

• Classifying an incoming support request
• Extracting structured fields from an invoice
• Summarizing a long internal policy
• Analysing a possible security incident

They may all use the same API and provider, but they do not have the same
requirements.

Before changing the model, we need to understand the business task:

• Must the output satisfy a strict JSON schema?
• Does the model need to call tools with valid arguments?
• How costly is a wrong answer?
• Does the output influence a financial, legal, privacy, or security decision?
• Does tone matter?
• Is the context unusually long?
• Which fields or behaviours must always remain correct?
• When must the workflow escalate to a person?

A model can perform well on a public benchmark and still be wrong for a particular
workflow.

The reverse is also true. A smaller model may be fully capable of performing a
bounded classification or extraction task. Sending that work to a frontier model
does not necessarily improve the business outcome—it may only increase the bill.

So the question is not:

Which model is best?

It is:

Which available model can satisfy the requirements of this workflow, and what
evidence do we have?

The workflow is the first useful unit of model selection

A recurring workflow contains more information than a generic prompt category.

Its system instructions, tools, response format, provider, requested model, and
message structure describe the job the application is trying to complete.

By observing that stable structure over time, a system can begin to distinguish:

• Routine classification from sensitive decision support
• Free-form writing from schema-bound extraction
• Tool selection from direct answering
• Long-context summarization from short intent routing
• Low-risk operational work from requests requiring stronger judgement

That is the first layer of MargIQ.

It wraps an existing OpenAI-compatible client, identifies recurring workflows,
analyses their business and technical requirements, and evaluates models from
the application's own allowed list.

The application still requests the model it already trusts. That requested model
remains the quality anchor and fallback.

MargIQ can begin in report-only mode. It can show where a lower-cost model
appears suitable without changing production execution. Routing only becomes an
option after a workflow has earned enough evidence.

But one workflow still should not mean one model

Workflow-level model selection is better than choosing one model for an entire
application, but it is still not enough.

Individual requests inside the same workflow can have very different complexity,
risk, and consequences.

Consider an ecommerce support workflow with tools for:

• Looking up an order
• Creating a refund case
• Escalating an issue to a human

One request says:

Can you check why my order has not shipped?

This is bounded operational work. The model needs to identify the order and call
the appropriate lookup tool.

Another request entering the same workflow reports that a damaged product caused
an injury, demands an immediate refund, and threatens public escalation.

Same application feature. Same system instructions. Same tool definitions.
Completely different consequences if the model gets it wrong.

Routing both requests to the cheapest model would be irresponsible.

Routing both to the strongest model is safe but potentially wasteful.

The workflow defines the business task. The individual request determines how
much capability and safety that particular execution requires.

That is why MargIQ's policies can contain multiple request paths inside one
workflow. The live request context can select an approved lower-cost path, retain
the requested model, or collect more evidence when the request does not match a
known path.

How the decision flow works

The important behaviour is not the happy path. It is the fallback.

If the workflow is unknown, the policy is not ready, a live request does not
match an approved path, or the request is risky or ambiguous, the application
uses the requested model.

What I found in a controlled benchmark

The current MargIQ sandbox defines 15 workflow families, including:

• Intent and support classification
• Invoice and CRM extraction
• Policy and meeting summarization
• Customer-support writing
• Review moderation
• Lead qualification
• PII handling
• Code-review triage
• Multi-turn customer chat
• Tool-based ecommerce support
• Dynamic request-shape and security-neighbour tests

It also contains 21 explicit evaluation scenarios covering paraphrases,
multilingual inputs, noisy OCR, long context, security neighbours, credential
leaks, and authorization vulnerabilities.

In one retained 58-request controlled benchmark snapshot, MargIQ detected seven
recurring workflows. Two policies became active at 0.96 confidence.

Two transaction-level routing decisions contained complete savings evidence:

Workflow path	Requested model	Selected model	Measured model-cost reduction
Low-risk intent classification	GPT-5	GPT-4o mini	90.7%
Routine order lookup with tools	GPT-5	GPT-4.1 mini	74.8%

The higher-risk refund and safety request in the same ecommerce workflow remained
on GPT-5.

Another support-classification workflow was blocked from optimization because
candidate outputs disagreed on decision-bearing fields and the benchmark did not
contain an authoritative quality definition for resolving the difference.

That blocked workflow matters as much as the savings examples. A routing system
should not convert uncertainty into a cost-reduction claim.

What these numbers do—and do not—prove

This was controlled sandbox traffic, not customer production traffic.

Only two requests in the retained snapshot provide complete transaction-level
savings evidence. The 90.7% and 74.8% figures describe those verified paths, not
an equivalent reduction across every request.

Across all 58 requests, the measured saving was only 0.1594%. Most of that traffic
was gathering learning evidence, running report-only, retaining the requested
model, or exercising workflows that had not earned an active policy.

That aggregate result should not be hidden. It shows the cold-start cost of an
evidence-based system.

If the same approved paths process more requests with similar token profiles,
dollar savings should scale with their eligible volume. But production savings
will always depend on:

• The proportion of spend belonging to eligible workflows
• Input and output token distribution
• The mix of routine and risky requests
• Available candidate models
• Provider pricing
• The quality threshold the business requires
• How often workflows change

The benchmark demonstrates the decision mechanism. It is not yet a production
case study.

The routing is not the hardest part

Calling a cheaper model is trivial.

Proving that it can perform a particular business workflow safely is much harder.

This is why I increasingly believe:

There is no dependable AI system without an evaluation harness.

For each workflow, MargIQ derives a quality contract describing the requirements
candidate outputs must satisfy.

Depending on the workflow, that can include:

• Output and schema validity
• Required fields
• Tool names and argument structure
• Task-correctness constraints
• Safety requirements
• Business-critical decisions
• Escalation behaviour
• Output fidelity

Candidate models are evaluated against that contract before routing becomes
eligible.

Deterministic checks can verify machine-readable requirements such as JSON,
schemas, and tool arguments. Evaluator models can assess requirements that cannot
be reduced to syntax alone. The candidate is assessed against the workflow
contract rather than treated as correct merely because it resembles the
requested-model output.

If evidence is insufficient, an evaluation is ambiguous, a critical constraint
fails, or the workflow cannot be evaluated reliably, MargIQ retains the requested
model.

The harness is not MargIQ's primary value proposition.

The primary value is intelligent, workflow-aware model selection—including
different model decisions for different requests inside the same workflow.

The harness is what gives the system permission to make those decisions.

Without it, model optimization can easily become quality degradation with better
accounting.

What I am still trying to learn

MargIQ is early. The problems I am thinking about now include:

• How stable should workflow identity remain as prompts evolve?
• How much evidence is enough before a routing path activates?
• How should a system detect semantic drift after activation?
• Which quality requirements can be evaluated automatically?
• When should a human define the authoritative business rule?
• How should cost, latency, and quality trade-offs be presented without creating false certainty?

I would especially value criticism from developers operating multi-model systems
in production. I am interested in where this abstraction breaks, which failure
modes are missing, and what evidence you would require before trusting automatic
model selection.

Project context: MargIQ

---

Editorial disclosure: I used an AI assistant to help structure and edit this
article. The architecture, implementation details, benchmark results, and
conclusions are based on my own work and were checked against retained project
evidence before publication.