DEV Community

gentleforge
gentleforge

Posted on

Quick Tip: Save a Ton on AI API Costs in Just 10 Minutes

#ai #webdev #programming #api

Quick Tip: Save a Ton on AI API Costs in Just 10 Minutes

So I just graduated from a coding bootcamp a few months ago, and I've been building little projects nonstop. One of them is a chatbot that helps people with homework (don't ask me why, my friend asked, I said sure). Anyway, when I got my first API bill, I actually laughed out loud because I thought there was a typo. There wasn't.

That moment sent me down a rabbit hole. I started reading everything I could about AI API costs, and I was shocked at how much money I was leaving on the table. Like, genuinely embarrassed. So I figured I'd write this up for anyone else who's out there accidentally lighting their wallet on fire.

Here's what I learned, and what I wish someone had told me before I started.

The Moment I Realized I Was Being an Idiot

When I first started, I just used whatever model the tutorial recommended. You know the one. It was GPT-4o, it worked great, and I never thought twice about it. Then I saw a number in a spreadsheet somewhere that said the output cost was $10/M tokens, and I had no idea what that even meant.

Once I actually did the math, my jaw hit the floor. For every million tokens that came out of the model, I was paying ten bucks. That's not nothing, especially when you're a fresh bootcamp grad with a $0 marketing budget and a dream.

The crazy thing is, the fix isn't even complicated. It's not like you need a PhD in machine learning or anything. You just need to be a little intentional about which model you're calling and when.

Let me walk you through everything I figured out.

The Model Swap That Changed Everything

This is the big one, and honestly it blew my mind when I first saw the numbers side by side.

The idea is simple: not every task needs the fanciest, most expensive model. A lot of the stuff I was doing could be handled by smaller, cheaper models without anyone noticing the difference. Nobody cares if your chatbot uses a super smart model to answer "what's the capital of France." It doesn't need to think hard about that.

Here's a comparison table I made for myself (and yes, I'm sharing my nerdy homework with you):

  • For plain old chatting, GPT-4o runs $10/M tokens. DeepSeek V4 Flash costs $0.25/M. That's 97.5% cheaper. Ninety-seven point five.
  • Sorting stuff into categories? GPT-4o-mini is $0.60/M. Qwen3-8B is $0.01/M. You save 98.3%.
  • Writing code? GPT-4o at $10/M versus DeepSeek Coder at $0.25/M. Another 97.5% savings.
  • Summarizing long documents? GPT-4o at $10/M, or Qwen3-32B at $0.28/M. About 97.2% off.
  • Translating text? GPT-4o at $10/M, or Qwen-MT-Turbo at $0.30/M. Ninety-seven percent savings.

I genuinely could not believe these numbers the first time I saw them. Like, why is anyone paying ten dollars when they could pay a quarter?

Here's how I set this up in my own code. I keep a little dictionary that maps task types to models, and I pick the right one based on what I'm trying to do:

import requests

MODEL_MAP = {
    "chat": "deepseek-v4-flash",        # $0.25/M
    "code": "deepseek-coder",           # $0.25/M
    "simple": "Qwen/Qwen3-8B",         # $0.01/M
    "reasoning": "deepseek-reasoner",  # $2.50/M
}

def call_llm(task_type, user_input):
    model = MODEL_MAP[task_type]
    response = requests.post(
        "https://global-apis.com/v1/chat/completions",
        headers={"Authorization": "Bearer YOUR_API_KEY"},
        json={
            "model": model,
            "messages": [{"role": "user", "content": user_input}]
        }
    )
    return response.json()

# Example usage
result = call_llm("simple", "What's 2 + 2?")
print(result)
Enter fullscreen mode Exit fullscreen mode

Just swapping the model based on the task took care of most of my savings right off the bat. Like, the vast majority. I was paying probably 5% of what I was paying before, just by being smarter about which model I called.

The Tiered Approach (Like Escalating to a Manager)

Okay so the model swap was huge, but I found another trick that pushed things even further. It's called tiered routing, and the concept is this: try the cheapest option first, and only "escalate" to something fancier if the cheap option doesn't cut it.

Think of it like this. Imagine you walk into a coffee shop and ask a simple question. Do you need to talk to the manager, or can the barista handle it? Probably the barista. Same idea here.

I built a little function that tries the cheap model first, checks if the answer is good enough, and only moves on to something more expensive if it has to:

import requests

API_URL = "https://global-apis.com/v1/chat/completions"
HEADERS = {"Authorization": "Bearer YOUR_API_KEY"}

def call_model(model, prompt):
    resp = requests.post(
        API_URL,
        headers=HEADERS,
        json={
            "model": model,
            "messages": [{"role": "user", "content": prompt}]
        }
    )
    return resp.json()

def quality_check(response):
    """Pretend we have a way to score the response quality"""
    # In real life, you'd check things like:
    # - Is the response empty?
    # - Did the model say "I don't know"?
    # - Is the response way too short?
    text = response.get("choices", [{}])[0].get("message", {}).get("content", "")
    if len(text) < 5:
        return 0.0
    if "i don't know" in text.lower():
        return 0.3
    return 0.85  # assume decent

def smart_generate(prompt):
    # Tier 1: ultra-cheap at $0.01/M
    resp = call_model("Qwen/Qwen3-8B", prompt)
    if quality_check(resp) >= 0.8:
        return resp  # about 80% of requests land here

    # Tier 2: standard at $0.25/M
    resp = call_model("deepseek-v4-flash", prompt)
    if quality_check(resp) >= 0.9:
        return resp  # another 15% or so

    # Tier 3: the big guns at $2.50/M
    return call_model("deepseek-reasoner", prompt)  # maybe 5%
Enter fullscreen mode Exit fullscreen mode

When I ran this for my customer support chatbot, the numbers were wild. I went from spending $420 a month down to $28 a month. The cheap model handled 85% of the questions just fine, and only the tricky stuff got bumped up.

I had no idea that was possible. I thought "good AI" meant "expensive AI." Turns out that's just not true.

Caching: Why Are You Asking the Same Question Twice?

This next one made me feel dumb, but in a good way. If someone asks your chatbot "What are your business hours?" do you really need to pay the API to answer that every single time? Of course not. Just remember the answer and reuse it.

I built a little cache using a basic Python dictionary. The idea is to hash the request (model + messages), and if I've seen that exact combination recently, just return what I already got:

import hashlib
import json
import time
import requests

API_URL = "https://global-apis.com/v1/chat/completions"
HEADERS = {"Authorization": "Bearer YOUR_API_KEY"}

cache = {}

def cached_chat(model, messages, ttl=3600):
    """Cache responses for ttl seconds (default 1 hour)"""
    key = hashlib.md5(
        json.dumps({"model": model, "messages": messages}).encode()
    ).hexdigest()

    if key in cache:
        entry = cache[key]
        if time.time() - entry["time"] < ttl:
            return entry["response"]  # cache hit, costs nothing extra

    response = requests.post(
        API_URL,
        headers=HEADERS,
        json={"model": model, "messages": messages}
    ).json()

    cache[key] = {"response": response, "time": time.time()}
    return response
Enter fullscreen mode Exit fullscreen mode

For my homework helper, the cache hit rate on common questions was somewhere between 50% and 80%. That's a huge chunk of requests that I'm not paying for at all anymore. The FAQ stuff, the documentation lookups, all of that just gets served from memory.

This one's free money. If you're not caching, start yesterday.

Compressing Prompts So You Pay Less Per Call

Okay this one is sneaky. So tokens are how the API measures what you send in and what you get back. More tokens means more money. So if your prompt is huge, you're paying more, even if the actual question is small.

I had a system prompt in one of my projects that was around 2,000 tokens. That's a lot! It was mostly background info and instructions. I wrote a function that uses the cheap Qwen3-8B model (which costs basically nothing) to summarize that big prompt down to something smaller before sending the real request:

def compress_prompt(text, target_ratio=0.5):
    """Compress long prompts using a cheap model"""
    if len(text) < 500:
        return text  # already short enough

    summary = requests.post(
        "https://global-apis.com/v1/chat/completions",
        headers={"Authorization": "Bearer YOUR_API_KEY"},
        json={
            "model": "Qwen/Qwen3-8B",
            "messages": [{
                "role": "user",
                "content": f"Summarize this in about {int(len(text)*target_ratio)} characters: {text}"
            }]
        }
    ).json()

    return summary["choices"][0]["message"]["content"]
Enter fullscreen mode Exit fullscreen mode

So in my case, that 2,000-token system prompt got compressed to about 400 tokens. That saved $0.024 per request on DeepSeek V4 Flash. Sounds small, right? But my bot was getting hit around 10,000 times a day. That's $240 a day. Over a year? $87,600.

Let me say that again. Eighty-seven thousand dollars a year. From compressing one prompt.

I was shook.

Batching Requests Like a Grocery Run

Last thing I want to talk about is batching. Instead of making ten separate API calls, just make one big call with all ten questions stuffed inside. Same total work, but you save on overhead and the input tokens you have to pay for are shared.

Before, my code looked like this (don't laugh, I actually did this):

questions = ["What is Python?", "What is JavaScript?", "What is Ruby?"]

# Bad: 3 separate API calls, paying for context setup each time
for question in questions:
    response = requests.post(
        "https://global-apis.com/v1/chat/completions",
        headers={"Authorization": "Bearer YOUR_API_KEY"},
        json={
            "model": "deepseek-v4-flash",
            "messages": [{"role": "user", "content": question}]
        }
    )
Enter fullscreen mode Exit fullscreen mode

After:

questions = ["What is Python?", "What is JavaScript?", "What is Ruby?"]

# Good: 1 API call, share the context across all questions
combined_prompt = "Answer each question briefly with a numbered list:\n"
for i, q in enumerate(questions, 1):
    combined_prompt += f"{i}. {q}\n"

response = requests.post(
    "https://global-apis.com/v1/chat/completions",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "model": "deepseek-v4-flash",
        "messages": [{"role": "user", "content": combined_prompt}]
    }
)
Enter fullscreen mode Exit fullscreen mode

That's typically a 10-20% savings on whatever you were already spending. Easy win.

Putting It All Together

So if you stack all of these together, here's what the math looks like for me:

  • Pick the right model for the task: saves about 90%
  • Route cheap first, expensive only when needed: pushes savings to 95%
  • Cache the obvious stuff: another 20-50% on top
  • Compress big prompts: another 15-30% per request
  • Batch when you can: another 10-20%

Combined, I went from spending hundreds a month to spending basically nothing. My homework chatbot went from "this is going to bankrupt me" to "I might actually make some money off this thing." And the best part is, the answers still look just as good. My users have no idea I'm using cheaper models, and honestly neither would I

Top comments (0)