rolltok

Posted on Oct 12

Understanding API Pricing: Why Token-Based Billing Confuses Everyone (And How to Make Sense of It)

#llm #ai #api #programming

Last Week, I was debugging a customer's bill. They'd been charged $47 for what they thought would be a $10/month subscription. The frustration was palpable: "I just wanted to test a few prompts. How did this happen?"

If you've ever stared at an API pricing page and felt confused, you're not alone. Token-based billing is the industry standard, but almost nobody explains it well. Let me fix that.

The Problem with "Per Token" Pricing

Most API providers show you something like this:

GPT-4o-mini: $0.15 per 1M input tokens, $0.60 per 1M output tokens

Sounds cheap, right? Until you realize:

You don't know how many tokens your prompt uses until you send it
Input and output have different prices (output is usually 3-4x more expensive)
System prompts count as input tokens (and they can be huge)
Context accumulates (every message in the conversation adds up)

So when you see "$0.15 per 1M tokens," your brain thinks "pennies." But your actual bill? Could be dollars.

What Actually Happens When You Send a Request

Let's trace a real example. You send this to an API:

messages = [
    {"role": "system", "content": "You are a helpful assistant..."},  # ~50 tokens
    {"role": "user", "content": "Explain quantum computing in 3 sentences."}  # ~15 tokens
]

What the API sees:

Input: 65 tokens (system prompt + your message)
Output: ~80 tokens (the response)

What you're charged:

Input cost: 65 × $0.15 / 1,000,000 = $0.00000975
Output cost: 80 × $0.60 / 1,000,000 = $0.000048
Total: $0.00005775

That's less than a cent. So where does the $47 come from?

The Conversation Multiplier Effect

Here's the trap: every message in the conversation history gets re-sent with each request.

Turn 1:

Input: 65 tokens → $0.00000975
Output: 80 tokens → $0.000048

Turn 2 (you ask a follow-up):

Input: 65 + 80 + 20 (new message) = 165 tokens → $0.00002475
Output: 100 tokens → $0.00006

Turn 3:

Input: 165 + 100 + 25 = 290 tokens → $0.0000435
Output: 120 tokens → $0.000072

By turn 10, you're paying for 10x the context. By turn 50, you're paying for 50x.

This is why a "simple test" can cost $47. It's not the per-token price—it's the accumulation.

How to Predict Your Actual Cost

Here's a formula I use with customers:

Estimated monthly cost = (avg_tokens_per_request × requests_per_day × 30) × (input_price + output_price) / 2

Example:

You send 20 requests per day
Each request averages 500 input tokens + 300 output tokens
You use GPT-4o-mini ($0.15 input, $0.60 output)

Daily input: 500 × 20 = 10,000 tokens
Daily output: 300 × 20 = 6,000 tokens
Monthly input: 10,000 × 30 = 300,000 tokens
Monthly output: 6,000 × 30 = 180,000 tokens

Monthly cost:
- Input: 300,000 × $0.15 / 1,000,000 = $0.045
- Output: 180,000 × $0.60 / 1,000,000 = $0.108
- **Total: $0.153**

Wait, that's only 15 cents? That can't be right.

The missing piece: conversation history.

If each request includes 5 previous messages (average 200 tokens each), your actual input per request is:

500 (new) + 5 × 200 (history) = 1,500 tokens

Recalculate:

Monthly input: 1,500 × 20 × 30 = 900,000 tokens
Monthly cost:
- Input: 900,000 × $0.15 / 1,000,000 = $0.135
- Output: 180,000 × $0.60 / 1,000,000 = $0.108
- **Total: $0.243**

Still cheap. But now imagine:

100 requests per day (not 20)
20 messages in history (not 5)
Using GPT-4 ($2.50 input, $10 output) instead of GPT-4o-mini

Monthly input: (500 + 20 × 200) × 100 × 30 = 13,500,000 tokens
Monthly output: 300 × 100 × 30 = 900,000 tokens

Monthly cost:
- Input: 13,500,000 × $2.50 / 1,000,000 = $33.75
- Output: 900,000 × $10 / 1,000,000 = $9
- **Total: $42.75**

That's how you get to $47.

The Subscription Model Alternative

This is why some providers (including RollTok) offer monthly subscriptions with token quotas. Instead of pay-per-token, you get:

Predictable billing: $10/month for X tokens
No surprise bills: You know your max cost upfront
Simpler mental model: "I have a budget" vs "I'm metered per token"

The tradeoff? Unused tokens don't roll over. If you buy 1M tokens and use 200K, the rest expire at month-end.

Is that fair? It depends on your usage pattern:

Consistent daily use: Subscription is usually cheaper
Burst usage: Pay-per-token might be better
Testing/experimentation: Subscription gives you a budget cap

How to Choose the Right Model for You

Ask yourself:

How many requests per day?

< 10: Pay-per-token
10-50: Either works
50: Subscription likely cheaper

How long are your conversations?

< 5 turns: Pay-per-token
5-20 turns: Either works
20 turns: Subscription (context accumulation kills you)

Do you need cost predictability?

Yes: Subscription
No: Pay-per-token

What's your budget tolerance?

"I can't exceed $20/month": Subscription
"I'll pay for what I use": Pay-per-token

A Transparent Pricing Calculator

Here's a simple calculator I built for customers. Plug in your numbers:

def estimate_monthly_cost(
    requests_per_day,
    avg_input_tokens,
    avg_output_tokens,
    conversation_turn,
    input_price_per_million,
    output_price_per_million
):
    # Account for conversation history
    actual_input_per_request = avg_input_tokens + (conversation_turn * 200)

    monthly_input = actual_input_per_request * requests_per_day * 30
    monthly_output = avg_output_tokens * requests_per_day * 30

    input_cost = monthly_input * input_price_per_million / 1_000_000
    output_cost = monthly_output * output_price_per_million / 1_000_000

    return input_cost + output_cost

# Example: 50 requests/day, 500 input tokens, 300 output tokens, 10 turns
cost = estimate_monthly_cost(50, 500, 300, 10, 0.15, 0.60)
print(f"Estimated monthly cost: ${cost:.2f}")
# Output: Estimated monthly cost: $1.35

Use this before you commit to a provider. It'll save you from surprise bills.

What Providers Should Do Better

Honestly? Most API providers make this way too confusing. Here's what I'd like to see:

Show estimated cost before sending: "This request will cost ~$0.003"
Break down input vs output: Don't just show "total tokens"
Warn about context accumulation: "Your conversation now has 50 turns. Input costs are 10x higher."
Offer a cost simulator: "If you send 20 requests/day, you'll spend ~$X/month"

Some providers are starting to do this. Most aren't.

The Bottom Line

Token-based billing isn't inherently bad—it's flexible and granular. But it requires you to understand:

Input and output have different prices
Context accumulates
Your actual cost depends on conversation length, not just prompt length

If you're evaluating API providers, don't just look at the per-token price. Look at:

Your actual usage pattern (requests/day, conversation length)
Whether you need cost predictability (subscription vs pay-per-token)
What happens to unused tokens (rollover vs expire)

And if a provider only shows you "per 1M tokens" without explaining the conversation multiplier effect? Run.

Full disclosure: I work on RollTok, which offers monthly subscriptions with token quotas. I think that model works better for consistent users, but pay-per-token is fine for burst usage. This article is my attempt to explain both models honestly.

If you found this helpful, I'm writing a series on API pricing, authentication, and debugging. Follow me for more.

DEV Community