DEV Community

fiercedash
fiercedash

Posted on

How I Cut My AI API Bill by 95% — An Indie Hacker's Guide

#api #deepseek #machinelearning #python

Look, how I Cut My AI API Bill by 95% — An Indie Hacker's Guide

honestly, I wanna start this post with a confession. For the first three months of building my SaaS, I was bleeding money on AI APIs. Like, EMBARRASSING amounts. I didnt even realize it because the bills just showed up on my card and I was too lazy to dig into the line items.

Then one Tuesday I sat down with a coffee and actually read the invoice. I nearly spit it out. I was spending roughly 5-10x what I should have been. Same product, same users, just... dumb choices.

So I spent the next few weeks ripping my stack apart and rebuilding it with cost in mind. Heres what I learned, and what you can steal. Pretty much every trick here is stuff I wish someone had told me BEFORE I shipped.

Let me get into it.

The Awful Moment I Realized I Was Getting Ripped Off

So picture this. Im running a customer support chatbot for a niche SaaS. Its pulling in around 5,000 conversations a month. Each one was going through GPT-4o because, honestly, I just defaulted to "the good one" without thinking. Output cost: $10/M tokens.

My monthly bill? $420.

Now, $420 isnt gonna bankrupt anyone. But when I tell you the same chatbot, after I refactored it, costs me $28/month... yeah. Thats a 93% reduction. Same quality. Same UX. Just smarter routing.

I gotta say, that gap between "what I was paying" and "what I should have been paying" is pretty much the entire reason Im writing this post. I cant be the only one doing this wrong.

Move #1: Stop Using the Expensive Model for Everything

This one is so obvious in hindsight it kinda hurts. But when youre heads-down shipping, you just reach for GPT-4o for everything. Chat? GPT-4o. Classification? GPT-4o. "Hey can you summarize this?" GPT-4o.

Gotta stop doing that.

Heres the mental model I use now: match the MODEL to the TASK. Not every job needs a Ferrari. Sometimes a bicycle gets you there.

Let me show you what I mean. These are the swaps that did the heavy lifting:

What Im Doing What I Used To Use What I Use Now Savings
Basic chat GPT-4o ($10/M) DeepSeek V4 Flash ($0.25/M) 97.5%
Classification GPT-4o-mini ($0.60/M) Qwen3-8B ($0.01/M) 98.3%
Code generation GPT-4o ($10/M) DeepSeek Coder ($0.25/M) 97.5%
Summarization GPT-4o ($10/M) Qwen3-32B ($0.28/M) 97.2%
Translation GPT-4o ($10/M) Qwen-MT-Turbo ($0.30/M) 97%

Just swapping models gave me 90% savings ALONE. Before I touched anything else. Wild.

Heres the actual code I run in production:

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_GLOBAL_API_KEY",
    base_url="https://global-apis.com/v1"
)

MODEL_MAP = {
    "chat": "deepseek-v4-flash",         # $0.25/M
    "code": "deepseek-coder",            # $0.25/M
    "simple": "Qwen/Qwen3-8B",           # $0.01/M (yes, ONE CENT)
    "reasoning": "deepseek-reasoner",    # $2.50/M
}

def pick_model(user_input):
    # dumb classifier - could be smarter, but works
    if "explain" in user_input.lower() or "why" in user_input.lower():
        return MODEL_MAP["reasoning"]
    if "function" in user_input or "code" in user_input:
        return MODEL_MAP["code"]
    if len(user_input) < 50:
        return MODEL_MAP["simple"]
    return MODEL_MAP["chat"]

def generate(user_input):
    model = pick_model(user_input)
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": user_input}]
    )
    return response.choices[0].message.content
Enter fullscreen mode Exit fullscreen mode

See that base_url? Thats Global API - I literally point every model call at the same endpoint and pick the model in the request. No juggling five different provider accounts. Huge quality of life win.

Move #2: Tiered Routing — Cheap First, Escalate Only When Needed

Okay this is where the real magic happens. Instead of picking ONE model for a request, I run the request through cheap models FIRST and only escalate to expensive ones if the cheap one flunked.

Honestly, this is the move that took me from $420 to $28.

Heres the playbook:

  • 85% of requests get handled by Qwen3-8B ($0.01/M). Yes. ONE CENT per million tokens. Its absurd.
  • 10-15% bump up to DeepSeek V4 Flash ($0.25/M) when the cheap one isnt confident enough.
  • The remaining ~5% go to deepseek-reasoner ($2.50/M) for the hard stuff. 
def smart_generate(prompt, max_budget=0.50):
    """Try cheap first, escalate if quality insufficient"""

    # Tier 1: ultra-budget
    resp = call_model("Qwen/Qwen3-8B", prompt)
    if quality_check(resp) >= 0.8:
        return resp  # most requests stop here

    # Tier 2: standard
    resp = call_model("deepseek-v4-flash", prompt)
    if quality_check(resp) >= 0.9:
        return resp

    # Tier 3: premium
    return call_model("deepseek-reasoner", prompt)
Enter fullscreen mode Exit fullscreen mode

The trick is having a quality_check function. Mine is dead simple - it does a second pass with a cheap model asking "did this answer the question?" and scores confidence. Costs almost nothing because the checker model is cheap. Saves a fortune because youre not calling deepseek-reasoner on every dang request.

This is the change that turned my chatbot from a $420/month line item into a $28/month line item. I cannot stress this enough. If you only do ONE thing from this post, do this.

Move #3: Cache Everything You Possibly Can

Heres a fun fact - roughly 20-30% of API calls in most apps are duplicates or near-duplicates. Users ask the same questions. Bots rephrase the same prompts. Your cache layer is just sitting there, unused.

I built a tiny caching wrapper around my API client. Took maybe 20 minutes. Saved me thousands.

import hashlib
import json
import time

cache = {}

def cached_chat(model, messages, ttl=3600):
    key = hashlib.md5(
        json.dumps({"model": model, "messages": messages}).encode()
    ).hexdigest()

    if key in cache:
        entry = cache[key]
        if time.time() - entry["time"] < ttl:
            return entry["response"]  # cache hit - $0 cost

    response = client.chat.completions.create(
        model=model,
        messages=messages
    )
    cache[key] = {"response": response, "time": time.time()}
    return response
Enter fullscreen mode Exit fullscreen mode

For FAQ-style traffic you can hit 50-80% cache rates. That means HALF your bill just disappears. Combined with semantic caching (caching similar-but-not-identical prompts) you can push that even higher.

Honestly I feel like caching is the most underrated optimization in AI apps. Everyone skips it. Dont be everyone.

Move #4: Compress Your Prompts Before Sending

This one stings because I was doing it wrong for MONTHS. I had these massive system prompts. Like, 2,000 tokens of "you are a helpful assistant that..." boilerplate. Every single request carried that weight.

Then I realized - wait, I can just SUMMARIZE that prompt with a cheap model first.

So now my flow is: cheap model reads the long prompt, outputs a compressed version, then the real model uses the compressed version to answer. Two calls instead of one, but the second call is way cheaper because it has way fewer input tokens.

def compress_prompt(text, target_ratio=0.5):
    if len(text) < 500:
        return text  # already short enough

    summary = call_model("Qwen/Qwen3-8B",
        f"Summarize this in {int(len(text)*target_ratio)} chars: {text}"
    )
    return summary
Enter fullscreen mode Exit fullscreen mode

The math on this is wild. Compressing a 2,000-token prompt down to 400 tokens saves $0.024 per request on DeepSeek V4 Flash. Sounds tiny. But I was running 10,000 requests a day at one point. Do the math:

  • $0.024 x 10,000 = $240/day
  • $240 x 365 = $87,600/year

Yeah. EIGHTY-SEVEN THOUSAND DOLLARS A YEAR. From compressing prompts. I almost didnt write this post because I felt dumb for not catching it sooner.

Move #5: Batch Your Requests

Last one for today. If youre sending a bunch of related requests, batch them into one API call. The pricing on most models charges you per token, not per request, so youre basically paying for the same input overhead over and over.

Before:

for question in questions:
    response = client.chat.completions.create(
        model="deepseek-v4-flash",
        messages=[{"role": "user", "content": question}]
    )
Enter fullscreen mode Exit fullscreen mode

After:

combined = "\n".join([f"{i+1}. {q}" for i, q in enumerate(questions)])
response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{
        "role": "user", 
        "content": f"Answer each numbered question:\n{combined}\n\nFormat: 1. answer\n2. answer\n..."
    }]
)
Enter fullscreen mode Exit fullscreen mode

You shave 10-20% off just by removing the redundant input tokens. Easy win.

Putting It All Together

Quick recap of where the savings came from in my setup:

  1. Smart model selection: 90% reduction just from not defaulting to GPT-4o
  2. Tiered routing: Another chunk because most requests never touch the expensive model
  3. Caching: 20-50% additional savings on top of that
  4. Prompt compression: 15-30% per request on whatever does go through
  5. Batching: 10-20% extra

Stacked together? My bill went from $420 to $28. Thats a 93% drop. Honestly I could probably squeeze out more with semantic caching and aggressive TTL tuning, but Im happy where Im at.

Some Real Talk About Implementation

Look, Im not gonna pretend this is a one-afternoon project. The tiered routing especially - you need a real quality check function, and building one that actually works takes iteration. I probably spent two weeks dialing mine in.

But the model selection swap? Thats an afternoon. The caching? An hour. The prompt compression? A few hours. You can start stacking these tomorrow and see savings on your NEXT bill.

Pretty much the worst thing you can do is nothing. Every day you run on GPT-4o for everything is a day youre lighting money on fire.

Where Global API Fits In

One thing that made all of this WAY easier was using Global API as my single endpoint. Instead of juggling OpenAI keys, DeepSeek keys, Qwen keys, etc, I just point everything at https://global-apis.com/v1 and pick the model in the request. One bill. One dashboard. One place to set rate limits.

If youre juggling multiple model providers, its worth checking out. global-apis.com - I dont work for them or anything, just genuinely made my life easier.

Wrapping Up

If youre an indie hacker shipping AI stuff, the TLDR is this: you ARE overspending. Probably by a lot. And the fix isnt complicated - its just work nobody warned you about.

Start with the model swap. Thats the 90% win right there. Then add caching. Then add routing. Then layer on compression and batching. Watch your bill crater.

Good luck out there. Ship the thing. But ship it cheap.

Top comments (0)