AI apps often become expensive for a simple reason: every request uses the same powerful model.
That approach is easy to build, but it is rarely efficient.
A customer asking for a one-line classification does not need the same model used for complex reasoning. A background summary does not always require the fastest available model. Treating every request equally is one of the easiest ways to waste money.
The expensive default
A basic implementation may look like this:
from openai import OpenAI
client = OpenAI(api_key="YOUR_API_KEY")
def generate_response(messages):
response = client.chat.completions.create(
model="gpt-5.4",
messages=messages,
)
return response.choices[0].message.content
This works, but every task goes through the same model:
- Simple classification
- FAQ responses
- Long-form writing
- Data extraction
- Complex reasoning
- Background processing
The result is predictable: higher costs, slower responses, and no clear reason for using an expensive model.
Match the model to the task
A better approach is to define models by workload:
MODELS = {
"simple": "gpt-5.4-mini",
"reasoning": "claude-sonnet-4.6",
"background": "deepseek-chat",
}
Then route requests based on the task:
def generate_response(task, messages):
model = MODELS[task]
response = client.chat.completions.create(
model=model,
messages=messages,
)
return response.choices[0].message.content
A simple request can use a faster, lower-cost model:
answer = generate_response(
"simple",
[
{
"role": "user",
"content": "Classify this ticket as billing, technical, or account-related."
}
],
)
A complex request can use a stronger model:
answer = generate_response(
"reasoning",
[
{
"role": "user",
"content": "Analyze the architectural risks in this system."
}
],
)
The application logic stays the same. Only the model changes.
Use an OpenAI-compatible endpoint
If every provider requires a separate SDK, reducing costs can create more engineering work.
You may need to manage:
- Multiple API keys
- Different request formats
- Different response structures
- Separate usage dashboards
- Provider-specific error handling
An OpenAI-compatible gateway simplifies the integration.
I work on the TokenBay team, so the example below uses TokenBay:
from openai import OpenAI
client = OpenAI(
base_url="https://api.tokenbay.com/v1",
api_key="YOUR_TOKENBAY_API_KEY",
)
Your code can keep the same request structure while testing different models:
response = client.chat.completions.create(
model="gpt-5.4-mini",
messages=[
{
"role": "user",
"content": "Extract the customer name and order number."
}
],
)
For another workload:
response = client.chat.completions.create(
model="claude-sonnet-4.6",
messages=[
{
"role": "user",
"content": "Review this technical proposal and identify its major risks."
}
],
)
You do not need to rewrite the integration every time you test a model.
Control the input, not just the model
Model choice is only one part of cost control.
Large prompts can be expensive even when the output is short. Before sending a request, consider:
- Removing irrelevant conversation history
- Limiting retrieved documents
- Summarizing old messages
- Avoiding repeated system instructions
- Setting a reasonable output limit
For example:
response = client.chat.completions.create(
model="gpt-5.4-mini",
messages=messages,
max_tokens=300,
)
More context is not automatically better. Unnecessary context can increase cost, latency, and confusion.
Cache predictable requests
Some AI requests are repeated frequently.
Examples include:
- Product descriptions
- FAQ answers
- Document classifications
- Common translations
- Standard onboarding messages
If the input and expected output are stable, caching can reduce repeated API calls:
cache = {}
def generate_cached(prompt):
if prompt in cache:
return cache[prompt]
response = client.chat.completions.create(
model="gpt-5.4-mini",
messages=[
{
"role": "user",
"content": prompt,
}
],
)
result = response.choices[0].message.content
cache[prompt] = result
return result
In production, use a proper shared cache instead of an in-memory dictionary. The principle remains the same: do not pay for the same answer repeatedly.
Measure before making decisions
Do not assume the cheapest model is automatically the best option.
Track each model’s:
- Cost per request
- Average latency
- Error rate
- Output quality
- Token usage
- Retry frequency
A cheaper model that requires many retries may cost more than a slightly more expensive model that succeeds consistently.
TokenBay can help centralize model access and provide usage visibility across different model providers. This makes it easier to compare cost and performance instead of relying on guesses.
Try Tokenbay：https://www.tokenbay.com/?utm_source=devto&utm_medium=community_content&utm_campaign=week1_free_content
Cost control is not quality reduction
The goal is not to use the cheapest model everywhere.
The goal is to spend more only where better performance creates real value.
A practical setup might look like this:
- Fast model for classification
- Low-cost model for background jobs
- Strong model for complex reasoning
- Fallback model during outages
- Human review for high-impact decisions
That is more efficient than sending every request to the most powerful model available.
Final thought
AI cost problems usually do not come from one unusually expensive request.
They come from thousands of ordinary requests being routed inefficiently.
Use the right model for the right task, reduce unnecessary context, cache repeated work, and monitor real usage.
The best AI cost strategy is not buying cheaper tokens.
It is avoiding tokens you never needed to spend.
Top comments (0)