Token-based pricing dominates the LLM API market, but it introduces a fundamental mismatch for modern workloads. As prompts grow longer and agentic workflows multiply reasoning steps, costs scale linearly with every additional token. For teams shipping retrieval-augmented generation, code review agents, or multi-turn assistants, the result is unpredictable monthly bills and constant input optimization. Request-based pricing removes that coupling entirely. With a flat cost per API call regardless of prompt length, platforms like Oxlo.ai reframe inference economics around completed tasks, not consumed characters.
The Hidden Cost of Token-Based Inference
Most providers, including Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale, bill by the token. Input tokens and output tokens are metered separately, which means a 100,000-token context window incurs a 100,000-token charge on every request. For applications that pass entire codebases, long conversation histories, or large document corpora into the prompt, this pricing model creates a direct tax on context richness.
Agentic loops compound the problem. A single user request can trigger multiple tool calls, each with its own bulky system prompt and few-shot examples. The token counter resets and accumulates on every hop, making it difficult to forecast spend from user activity alone.
How Request-Based Pricing Changes the Equation
Request-based pricing decouples cost from prompt length. One API call costs one flat unit, whether you send 500 tokens or 50,000. This model favors workloads where context is essential, not optional. Long-context reasoning, multi-step agent orchestration, and large-batch document processing become economically viable because the bill scales with actions taken, not words consumed.
Oxlo.ai uses this structure. Every request to the chat completions endpoint, from a short classification task to a 131K-context analysis with Kimi K2.6, draws from the same predictable quota. For teams operating agentic systems, that predictability turns infrastructure cost from a variable tax into a fixed operational line item. In practice, request-based pricing can be 10-100x cheaper than token-based alternatives for long-context workloads.
Where Token-Based Providers Still Fit
Token-based billing is not without merit. Short prompts, low-latency classification, or single-turn completions where input length never exceeds a few hundred tokens can be cost-effective when priced per token. If your workload is highly irregular and 90% of requests are tiny, metering by consumption may minimize waste.
The friction begins when average prompt length crosses into the thousands of tokens, or when architectural decisions start being driven by a desire to trim the token count rather than improve the product.
Oxlo.ai in Practice
Oxlo.ai is a developer-first inference platform built around this request-based model. It offers 45+ open-source and proprietary models across seven categories, including LLMs, code models, vision, image generation, audio, embeddings, and object detection. The API is fully OpenAI SDK compatible, so switching requires only a base URL change.
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key="your-oxlo.ai-api-key"
)
response = client.chat.completions.create(
model="deepseek-r1-671b",
messages=[
{"role": "system", "content": "You are a senior software architect."},
{"role": "user", "content": open("large_codebase.py").read()}
],
stream=True
)
for chunk in response:
print(chunk.choices[0].delta.content, end="")
Because Oxlo.ai charges per request, passing an entire file or lengthy conversation history does not inflate the bill. The platform also exposes streaming, function calling, JSON mode, vision inputs, and multi-turn conversations through standard endpoints. Popular models such as DeepSeek R1 671B MoE for reasoning, Llama 3.3 70B for general tasks, and Qwen 3 32B for multilingual agent workflows are available with no cold starts.
Selecting the Right Model for the Workload
Cost structure should not be the only factor. Model capability determines whether a request succeeds or fails. Oxlo.ai organizes its catalog into categories that map directly to common engineering needs. For deep reasoning and complex coding, DeepSeek R1 671B MoE or Kimi K2.6 offer advanced chain-of-thought performance. For high-volume agentic tasks, GLM 5 or Minimax M2.5 provide tool-use and long-horizon execution. For vision tasks, Gemma 3 27B or Kimi VL A3B handle image inputs.
Because the pricing is request-based, you can select the best model for the task without worrying that a larger context window will trigger a disproportionate cost penalty.
Making the Switch
If your current token bill scales unpredictably with context length, or if engineering time is being spent trimming prompts to save money rather than improve accuracy, it is worth evaluating a request-based alternative. Oxlo.ai provides a free tier with 60 requests per day and a 7-day full-access trial, which is enough to benchmark real workloads against existing token-based bills.
For long-context applications and agentic systems, the economics are straightforward. A flat per-request rate can reduce costs significantly compared to token-based metering. Detailed plan information is available on the Oxlo.ai pricing page.
Top comments (0)