Token-Based vs Request-Based LLM Pricing: A Comprehensive Guide

Most AI inference platforms, including Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale, charge by the token. Input tokens, output tokens, and sometimes premium tokens for longer contexts all roll into a variable bill that is hard to forecast. Oxlo.ai takes a different approach. With request-based pricing, your invoice reflects how many times you called the model, not how many words you fed it. For teams running long-context retrieval pipelines, multi-step agents, or large-batch code reviews, the difference between these two models is not marginal. It is architectural.

How Token-Based Pricing Works

Under token-based pricing, every chunk of text sent to or received from the model is metered. A single API call might contain a 10,000-token system prompt, a 50,000-token document, and a 2,000-token completion. Each component carries its own rate. Costs scale linearly with prompt length, and output length is only known after generation finishes. This makes budgeting a statistical exercise. Teams often end up over-provisioning or aggressively truncating context windows to avoid surprises.

How Request-Based Pricing Works

Request-based pricing flattens the cost structure into a single line item per API request. Whether you send 500 tokens or 50,000 tokens, the charge is identical. This removes the incentive to strip system prompts, compress history, or split documents into tiny chunks. It also makes load testing and capacity planning trivial. If you know your peak QPS, you know your ceiling.

Where Request-Based Pricing Wins

The advantage of request-based pricing widens as context length grows. A long-context RAG query that stuffs five retrieved documents into the prompt might consume 30,000 input tokens. Under token metering, that single call is expensive. Under request-based billing, it is one unit.

Agentic workflows compound the effect. An agent that iterates through tool calls, appending results to a growing conversation history, generates dozens of requests per task. With token-based pricing, each round trip adds input and output tokens. With request-based pricing, only the number of round trips matters.

Consider a coding assistant that analyzes an entire repository context:

import openai

client = openai.OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_API_KEY"
)

# One request carrying a full file tree and diff
response = client.chat.completions.create(
    model="deepseek-r1-671b",
    messages=[{
        "role": "user",
        "content": "Review this codebase for memory leaks.\n\n" + large_context
    }],
    stream=True
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

In a token