LLM Pricing Models: Token-Based vs Request-Based

#costoptimization #oxlo #ai

Choosing an LLM inference provider requires more than comparing model cards. The pricing model underneath directly determines how your costs scale as you move from prototypes to production. Today, the market is split between token-based billing, used by most major providers, and request-based billing, a flat-rate alternative that changes the economics of long-context and agentic workloads. Understanding the mechanics of each is essential for controlling spend without limiting capability.

How Token-Based Pricing Works

Token-based providers charge for every unit of text processed. That includes every token in your system prompt, your retrieved context, your conversation history, and the model's generated output. If you send a 50,000-token prompt and receive a 500-token response, your bill reflects the sum of both.

This model is the default for many platforms, including Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale. It aligns cost with raw compute, but it introduces significant variance. A short greeting costs almost nothing, while a long-context retrieval-augmented generation (RAG) query or a multi-turn agent loop can accumulate thousands of input tokens per turn. For teams running agents that maintain large conversation buffers or process extensive document contexts, costs become difficult to forecast and can scale nonlinearly with usage.

How Request-Based Pricing Works

Request-based pricing replaces the token counter with a flat cost per API call. Whether you send 500 tokens or 100,000 tokens, the price is the same. This shifts the economic incentive: you optimize for inference quality and context completeness, not for prompt compression.

Oxlo.ai is a developer-first inference platform built on this model. Instead of metering input and output tokens, Oxlo.ai charges one flat rate per request regardless of prompt length. This makes the platform significantly cheaper for long-context and agentic workloads, where token counts are inherently high. You can view exact rates on the Oxlo.ai pricing page.

Cost Predictability in Practice

Consider a support agent that retrieves fifty documents into context before answering a user question. Under token-based billing, every message in the thread incurs costs for the entire retrieved corpus plus the growing chat history. As the conversation lengthens, the input token count rises, and so does the cost per turn.

Under Oxlo.ai's request-based model, each turn is a single request. The cost does not scale with the size of the document pile or the conversation history. This predictability simplifies budgeting. You estimate costs based on user sessions or API calls, not on internal token accounting.

The difference becomes more pronounced at scale. For workloads where a single request can carry hundreds of thousands of tokens, request-based pricing can be 10-100x cheaper than token-based alternatives. The savings are not marginal; they change which architectures are economically viable.

When Request-Based Wins

Flat per-request pricing is not universally superior, but it wins in specific high-value scenarios.

Agentic workflows: Autonomous agents make multiple tool calls and maintain large state buffers. Token-based billing penalizes every extra byte of context. Oxlo.ai removes that penalty.
Long-context RAG: Pipelines that inject large document sets, codebases, or knowledge bases into the prompt naturally generate high input token counts. With Oxlo.ai, you pay per query, not per token retrieved.
Multi-turn conversations: Chat applications with extensive system prompts and history can accumulate context quickly. Request-based pricing keeps the marginal cost of each additional turn constant.
Batch processing: Large-scale jobs that process lengthy records benefit from cost stability across variable input lengths.

Oxlo.ai supports these use cases with 45+ models across seven categories, including DeepSeek R1 671B MoE for reasoning, Llama 3.3 70B for general tasks, Kimi K2.6 for agentic coding and vision, and GLM 5 for long-horizon agents. There are no cold starts on popular models, so latency remains consistent even under load.

Migrating to Oxlo.ai

Oxlo.ai is fully OpenAI SDK compatible, which means migration is usually a one-line change to your base URL.

import openai

client = openai.OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

# One flat request, regardless of prompt length
response = client.chat.completions.create(
    model="deepseek-r1-671b",
    messages=[
        {"role": "system", "content": "You are a helpful coding assistant."},
        {"role": "user", "content": long_codebase_context + user_question}
    ]
)

The platform exposes standard endpoints for chat completions, embeddings, image generation, audio transcription, and text-to-speech. Function calling, JSON mode, streaming, and vision inputs are all supported. If you are currently using a token-based provider for long-context work, switching to Oxlo.ai can reduce costs and simplify forecasting without requiring architectural changes.

Conclusion

Token-based pricing ties your bill to the volume of text processed, which makes it unpredictable for complex, context-heavy applications. Request-based pricing decouples cost from prompt length, giving teams a fixed unit of cost that aligns with business metrics like user sessions or API calls.

For developers building agents, long-context RAG, or any workload where input tokens routinely run into the tens or hundreds of thousands, Oxlo.ai offers a flat per-request alternative that is purpose-built for production scale. With broad model support, no cold starts, and full OpenAI SDK compatibility, it is a relevant option for teams that want cost control without sacrificing capability. You can explore the free tier or compare plans directly at https://oxlo.ai/pricing.

DEV Community