Most LLM inference platforms bill by the token. You count input tokens, output tokens, and sometimes tokens embedded in tool contexts. For short chat messages, this feels predictable. For long documents, agent loops, or multi-turn reasoning, token counts explode and so does cost. Request-based pricing removes that variable entirely. You pay once per API call, regardless of how many tokens move through the context window.
This is where Oxlo.ai fits. Oxlo.ai is a developer-first AI inference platform with a flat cost per API request. Unlike token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale, your cost does not scale with prompt length. For long-context and agentic workloads, that architectural difference can make Oxlo.ai significantly cheaper without sacrificing model choice or API compatibility.
The hidden cost of tokens
Token-based providers meter every chunk of text that enters or leaves the model. A 128k context fill, a system prompt, a long RAG retrieval, or a recursive agent scratchpad all add to the bill. Because total cost equals token volume multiplied by per-token rates, workloads with large inputs or high iteration counts create unpredictable spend. Teams often resort to aggressive truncation, context compression, or model downsizing to stay inside budget.
How request-based pricing works
With request-based pricing, the unit of cost is the HTTP request. One call to the chat completions endpoint costs the same whether you send a 50-word greeting or a 50,000-word legal brief. This makes budgeting linear: if your application makes 1,000 calls per day, you know the exact ceiling. There are no surprise overages from a user pasting a novel into the prompt window.
Oxlo.ai uses this model across its entire catalog. The platform offers 45+ open-source and proprietary models across seven categories, including LLMs, code models, vision models, image generation, audio, embeddings, and object detection. Every endpoint, from chat/completions to audio/transcriptions, follows the same request-based philosophy.
Where request pricing wins
Long-context workloads are the obvious fit. If you are summarizing books, analyzing repositories, or running retrieval-augmented generation over large knowledge bases, token counts can reach six or seven figures per call. Under token-based billing, that becomes expensive fast. Under Oxlo.ai's flat per-request structure, the cost stays constant.
Agentic workflows are the second major win. Agents often loop through tool calls, memory updates, and multi-turn reasoning chains. Each iteration adds tokens. When cost is tied to tokens, every extra thought step is a tax on performance. Request-based pricing lets you give the agent more room to reason without watching the meter spin. Oxlo.ai notes that request-based pricing can be 10-100x cheaper than token-based approaches for long-context workloads.
SDK compatibility and integration
Switching pricing models should not mean rewriting your stack. Oxlo.ai exposes a fully OpenAI-compatible API with base URL https://api.oxlo.ai/v1. You can keep using the official OpenAI SDKs in Python, Node.js, or cURL. Change the base URL and API key, and existing code runs unchanged.
Example Python client setup:
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key="your-oxlo.ai-api-key"
)
response = client.chat.completions.create(
model="deepseek-r1-671b",
messages=[
{"role": "system", "content": "You are a careful reasoning assistant."},
{"role": "user", "content": "Explain the trade-offs between MoE and dense architectures. Use the full context available."}
],
stream=True
)
for chunk in response:
print(chunk.choices[0].delta.content or "", end="")
Because Oxlo.ai supports streaming, function calling, JSON mode, vision input, and multi-turn conversations, the migration path is usually a two-line diff.
Model catalog and performance
Request-based pricing is only useful if
Top comments (0)