Token-Based LLM API with Request-Based Pricing: A Comprehensive Guide

Most LLM APIs bill by the token. You count input tokens, output tokens, and sometimes cache tokens, then multiply by rate cards that vary by model and context window. For simple chat, this is manageable. For long-context retrieval, agent loops, or multi-modal pipelines, token math becomes a tax on every iteration. This guide explains how token-based pricing works, where it breaks down, and why a request-based alternative is becoming the pragmatic choice for production workloads.

How Token-Based Pricing Works

Token-based providers split costs into at least two dimensions: input tokens and output tokens. Some add a third dimension for cached context or premium model tiers. Because context windows have grown from 4K to 1M+ tokens, a single request that passes a large code base or document corpus can consume as many input tokens as hundreds of short queries. Providers such as Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale all use variations of this scheme.

The mechanics are straightforward. You estimate prompt size, guess completion length, and multiply by the current rate card. If your application appends retrieved documents, conversation history, or image patches, the token count grows non-linearly. Discounts for prompt caching exist, but they typically require exact prefix matches and still leave you with an output token bill you cannot predict until the stream ends.

Where Token Models Break Down

Token-based billing creates friction in three common scenarios:

• Agentic workflows. Each tool call and observation adds history. A ReAct loop that runs five turns can multiply your base prompt by 5x or more.
• Long-context RAG. Injecting retrieved chunks into a prompt is standard practice. With token pricing, every extra paragraph retrieved directly increases cost.
• Multi-modal inputs. Vision models tokenize image patches at high ratios. A single high-resolution screenshot can cost more than the text response it generates.

The result is variable spend. Engineering teams end up building token guards, truncation heuristics, and billing alerts instead of focusing on product logic.

Request-Based Pricing Defined

Request-based pricing replaces the token multiplier with a flat unit: one API request equals one cost. It does not matter if your prompt is ten tokens or one hundred thousand. The price is fixed before you send the request.

Oxlo.ai uses this model. Every call to the chat/completions, embeddings, images/generations, audio/transcriptions, or audio/speech endpoints incurs one flat cost per request regardless of prompt length. For long-context and agentic workloads, this removes the penalty for passing large states, full documents, or rich multi-turn histories to the model.

Workloads That Favor Flat Requests

A flat per-request rate is not just simpler. It is often significantly cheaper when your workload exhibits any of the following traits:

• Deep reasoning and coding. Models like DeepSeek R1 671B MoE, Qwen 3 Coder 30B, and Oxlo.ai Coder Fast are most useful when you feed them entire modules or stack traces. Truncation to save tokens defeats the purpose.
• Long-context inference. DeepSeek V4 Flash supports a 1M context window. Kimi K2.6 offers 131K context with advanced reasoning and vision. With request-based pricing, you can use the full window without a linear cost spike.
• Agentic tool use. GLM 5, Minimax M2.5, and Qwen 3 32B are built for long-horizon agentic tasks. Flat pricing lets the agent maintain a full scratchpad and tool history across many turns.
• Batch document processing. Sending a full PDF as context to Llama 3.3 70B or GPT-Oss 120B costs the same as sending a one-sentence question.

Migrating to Oxlo.ai

Oxlo.ai is fully OpenAI SDK compatible. You change the base URL and API key, and existing code runs without modification. There are no cold starts on popular models, and you keep the same streaming, JSON mode, function calling, and vision features you already use.

Below is a minimal Python example that sends a long prompt to Llama 3.3 70B. The cost is the same whether the user message is one line or five hundred lines.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_OXLO_API_KEY",
    base_url="https://api.oxlo.ai/v1"
)

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "You are a precise technical assistant."},
        {"role": "user", "content": "Refactor this 500-line module into three classes."}
    ],
    stream=True
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

The same pattern works for the entire model catalog, including DeepSeek V3.2, Kimi K2.5, Kimi K2 Thinking, and DeepSeek R1 671B MoE. You can also switch to embeddings, image generation, or transcription endpoints without learning a new SDK.

Model Catalog and Features

Oxlo.ai hosts 45+ open-source and proprietary models across seven categories:

• LLMs and reasoning. Qwen 3, Llama 3/4, DeepSeek R1 and V3, Kimi K2.x, GPT-Oss, Mistral, GLM 5, Minimax.
• Code. Qwen 3 Coder 30B, DeepSeek Coder, Oxlo.ai Coder Fast.
• Vision. Gemma 3 27B, Kimi VL A3B, and vision-capable chat models.
• Image generation. Oxlo.ai Image Pro and Ultra, Flux.1, SDXL, Stable Diffusion 3.5.
• Audio. Whisper Large v3, Turbo, Medium, and Kokoro 82M text-to-speech.
• Embeddings. BGE-Large, E5-Large.
• Object detection. YOLOv9, YOLOv11.

All endpoints support streaming responses, multi-turn conversations, and tool use where the underlying model allows it.

Calculating Your Break-Even

To decide whether request-based pricing fits your stack, audit your current token spend across three variables:

1. Peak input size. Find the 95th percentile input token count in your production logs.
2. Output variance. Measure how much your completion lengths swing between requests.
3. Request volume. Count how many API calls you make per day, not how many tokens they contain.

If your workload mixes long prompts with unpredictable outputs, a flat per-request rate removes the scaling tax on input length. For exact plan details and to compare against your current provider, see the Oxlo.ai pricing page. Oxlo.ai also offers a Free tier with 60 requests per day and a 7-day full-access trial, so you can validate the model with real traffic before committing.

Conclusion

Token-based pricing made sense when context windows were small and prompts were short. Modern workloads, 1M context models, and agentic architectures have inverted that assumption. Paying by the request aligns cost with business value, not with the number of words in your prompt. If you are building agents, code assistants, or long-context RAG systems, Oxlo.ai provides a developer-first, OpenAI-compatible platform with flat per-request pricing, no cold starts, and a broad model catalog that includes the latest reasoning, coding, and vision models.