How Much Does an LLM Really Cost? A Developer's Guide to Estimating API Spend

#ai #webdev #productivity #llm

If you're adding an AI feature to a product, the first question from your PM or finance team is almost always: "How much will this cost us per month?" The good news: for most teams the answer is far more predictable than it looks. The bad news: you have to do a little token math first.

This post walks through a developer-friendly way to estimate and control LLM API costs, inspired by a recent pricing breakdown from the team at Journeyhorizon.

Training cost vs. inference cost: only one matters to you

Headlines about LLMs costing "hundreds of millions of dollars" refer to training frontier models — GPU clusters, curated datasets, RLHF, and huge research teams. Unless you're an AI lab, that's not your bill.

Your bill is inference via API: you pay per token you send and per token you receive. It scales with usage, which means you can model it like any other metered cloud service.

How API pricing actually works

A few rules of thumb that apply to almost every provider:

You pay per token, not per request. A token is roughly ¾ of an English word.
Input and output are priced separately. Output tokens are usually 2–5x more expensive than input tokens, because generation is more compute-heavy than reading.
Context size matters. Very long context windows can carry higher per-token rates.
Prompt caching is your friend. If you resend the same system prompt, catalogue, or style guide on every call, caching can cut that part of the cost dramatically.

Prices range from a few cents per million tokens for small "micro" models up to hundreds of dollars per million tokens for premium reasoning models. Always check the provider's current pricing page — these numbers move fast, and mostly downward.

A tiny cost estimator

Here's the formula I use before writing a single line of feature code:

def monthly_llm_cost(requests_per_month, in_tokens, out_tokens,
                     price_in_per_m, price_out_per_m):
    """Estimate monthly LLM spend in USD."""
    total_in = requests_per_month * in_tokens
    total_out = requests_per_month * out_tokens
    return (total_in / 1_000_000) * price_in_per_m + \
           (total_out / 1_000_000) * price_out_per_m

# Example: natural-language search
# 100k searches/month, ~200 input + ~100 output tokens each,
# on a small model priced at $1 in / $5 out per million tokens
print(monthly_llm_cost(100_000, 200, 100, 1, 5))  # -> 70.0

Swap in your own volumes and your provider's current prices. Get real token counts by running a handful of representative prompts and reading the usage field in the API response.

Two quick scenarios

Auto-tagging listings. 10,000 items/day, ~100 input tokens and ~50 output tokens each. That's ~30M input and ~15M output tokens per month. On a small, cheap model this lands somewhere between ~$15 and ~$100 per month depending on the tier — trivial compared to the manual effort it replaces.

Support chatbot. Multi-turn conversations resend history on every turn, so input tokens grow quickly. This is where caching, history summarisation, and a cheaper default model make the biggest difference.

The lesson: cost scales linearly with usage, and the model choice is the biggest multiplier. The same feature can cost 5–10x more just by defaulting to a flagship model.

Practical ways to cut your LLM bill

Right-size the model. Classification, routing, and extraction rarely need a frontier model. Benchmark your task on 2–3 tiers and pick the cheapest one that meets your quality bar.
Trim your prompts. Shorter, clearer system prompts with one or two good examples often outperform long ones — and cost less.
Cache static context. Product catalogues, policies, and style guides are perfect caching candidates.
Batch non-urgent work. If results aren't needed in real time, process them in batches (many providers offer discounted batch APIs).
Use a cascade. Let a small model handle the easy 80%, and escalate only low-confidence cases to a bigger model — or to a human.
Cap output length. Set max_tokens sensibly; output is the expensive side.
Monitor per feature. Tag requests by feature so you know exactly which part of your product is driving spend.

If you'd rather not build all of this tuning in-house, Journeyhorizon offers a dedicated AI cost optimization service that helps teams pick the right models, architect efficient integrations, and keep spend under control as usage grows.

Self-hosting: is open source "free"?

Open-weight models like Llama or Mistral remove per-token fees, but you trade them for GPU infrastructure, ops work, and engineering time. Self-hosting usually only pays off at very high, steady volume or when data residency requirements rule out third-party APIs.

Wrapping up

The real question isn't "Can we afford an LLM?" — for most products, you can. It's "Does this feature earn back its token cost through conversion, retention, or saved operational time?" Do the token math up front, choose the smallest model that works, and measure spend per feature from day one.

For the original pricing breakdown and more on building AI-powered products and marketplaces, check out Journeyhorizon's blog.

What's your go-to trick for keeping LLM costs down? Share it in the comments!

DEV Community