DEV Community

Cover image for Model selection isn't just an AI decision. It's a cost decision.
Assili Salim
Assili Salim

Posted on

Model selection isn't just an AI decision. It's a cost decision.

#ai #typescript #agents #node

A lot of AI teams optimize prompts, workflows, and infrastructure.

Far fewer treat model selection as a runtime cost-control problem.

That's becoming a mistake.

A recent example is Cursor. Switching to a more capable model can dramatically increase API usage costs, even though the only thing a developer changed was a dropdown in their editor.

Nothing else changed.

No deployment.

No architecture change.

Just a different model.

The real problem

Most agent systems treat model choice as a preference:

const model = process.env.DEFAULT_MODEL;
Enter fullscreen mode Exit fullscreen mode

In reality, it should be treated as a decision constrained by:

  • task complexity,
  • remaining session budget,
  • latency requirements,
  • and expected ROI.

Instead of asking "What's the best model?", production systems should ask:

"What's the cheapest model that can successfully complete this task?"

1. Route models by task

Not every request needs your most capable—and most expensive—model.

A simple router is often enough:

function selectModel(task: Task) {
  switch (task.complexity) {
    case "low":
      return "cheap";

    case "medium":
      return "standard";

    case "high":
      return "frontier";
  }
}
Enter fullscreen mode Exit fullscreen mode

Classification, formatting, and summarization rarely need frontier models.

Save those for the problems that actually benefit from them.

2. Make budget part of routing

Routing shouldn't depend only on the task.

It should also depend on how much budget remains.

if (remainingBudget < threshold) {
    return "cheap";
}
Enter fullscreen mode Exit fullscreen mode

That allows an agent to degrade gracefully instead of exhausting its budget on the last few requests.

3. Validate before the call

Every provider request should pass through a budget check.

That check should answer questions like:

  • Is this model registered?
  • Do we know its pricing?
  • Can this session afford another request?

If the answer is no, the request shouldn't leave your process.

Failing early is much cheaper than discovering the problem in next month's invoice.

4. Don't let context grow forever

Long-running agents become expensive because every request carries more history.

A sliding context window plus pinned instructions often reduces token usage without affecting output quality.

Keeping all history is usually the easiest implementation—not the cheapest one.

The bigger shift

The industry talks a lot about observability.

Observability tells you what happened.

Cost governance decides what is allowed to happen.

That's an important distinction.

As models become more numerous—and pricing changes more frequently—the most effective cost optimization won't come from better dashboards.

It will come from making model selection a guarded runtime decision instead of a static configuration value.

That's the architectural direction I'm exploring with AI CostGuard, but the principle applies regardless of the framework or provider you're using.
https://github.com/salimassili62-afk/ai-costguard

Top comments (0)