Most LLM inference platforms bill by the token. You count input and output tokens, multiply by a per-million rate, and hope your monthly invoice stays predictable. For teams running long-context prompts or agentic workflows, token-based pricing introduces a scaling tax: every extra line of context, every tool result fed back into the prompt, and every turn in a conversation adds to the bill. Request-based pricing removes that variable. You pay a flat cost per API call, whether the payload is one sentence or a 100,000-token document.
Token-Based vs Request-Based Pricing
Token-based providers, including Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale, charge for the total number of tokens processed. This model maps cost directly to compute consumption. For short queries and low-traffic prototypes, it is straightforward. You send a brief prompt, receive a brief answer, and the charge is minimal.
Request-based pricing flips the unit of measurement from tokens to HTTP requests. Oxlo.ai uses this model. A single request costs the same regardless of prompt length or response size. This shifts the cost risk away from context length and toward call volume, which is easier to forecast for many applications.
Where Token-Based Costs Scale
Token-based costs grow linearly with input size. If you are building retrieval-augmented generation pipelines that inject dozens of document chunks into the context window, or agent systems that append tool outputs back into the prompt each round, your input tokens accumulate fast. Multi-turn conversations suffer the same effect: the full history is re-sent on every turn.
Models now support context windows of 128,000 tokens or more. Filling even a fraction of that window with token-based billing can turn a routine inference call into a significant line item. The price is not in the complexity of the task, but in the size of the prompt.
When Flat Request Pricing Wins
Flat request pricing wins when your prompts are large and your architecture is iterative. Consider a coding assistant that reads an entire repository file to suggest a refactor. Under token pricing, loading an 80,000-token context incurs a large input charge before the model generates a single completion token. Under request pricing, the call is one flat unit.
Agentic workflows amplify the difference. An agent that performs five tool calls, each result appended to context, might send the accumulated payload multiple times. With token-based billing, each loop is more expensive than the last. With request-based billing, each loop is a fixed cost.
Oxlo.ai carries several models built for this exact profile. DeepSeek V4 Flash supports a 1 million token context window and near state-of-the-art open-source reasoning. Kimi K2.6 handles advanced agentic coding and vision across 131,000 tokens. GLM 5 is a 744B parameter MoE designed for long-horizon agentic tasks. With Oxlo.ai, running these models at full context does not trigger a proportional price increase.
Oxlo.ai Request-Based Inference
Oxlo.ai is a developer-first inference platform offering 45+ open-source and proprietary models across seven categories. Beyond LLMs and chat models, the platform provides code models such as Qwen 3 Coder 30B and Oxlo.ai Coder Fast, vision models like Gemma 3 27B and Kimi VL A3B, image generation, audio transcription and speech, embeddings, and object detection. All endpoints are fully OpenAI SDK compatible, and popular models start with no cold starts.
Pricing is structured around predictable tiers. The Free plan includes 60 requests per day across 16+ models and opens with a 7-day full-access trial. The Pro plan provides 1,000 requests per day across all models. The Premium plan raises that to 5,000 requests per day with priority queue access. Enterprise plans offer custom unlimited volume on dedicated GPUs. Because the cost is per request, long-context and agentic workloads can be 10 to 100 times cheaper than token-based alternatives. See the exact tiers at https://oxlo.ai/pricing.
Migrating a project takes minutes. Change the base URL and API key, and existing Python, Node.js, or cURL code continues to work.
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key="your-oxlo.ai-api-key"
)
response = client.chat.completions.create(
model="deepseek-v4-flash",
messages=[
{"role": "system", "content": "You are a senior software engineer."},
{"role": "user", "content": open("large_codebase.py").read()}
],
stream=False
)
print(response.choices[0].message.content)This single request costs the same whether the file is 1,000 tokens or 80,000 tokens. Streaming responses, JSON mode, function calling, and vision inputs are all supported through the same interface.
Switching to Oxlo.ai
If your current token-based bill is driven by large prompts, conversation history, or agent loops, moving to a request-based model removes the primary cost driver. Oxlo.ai functions as a drop-in replacement for the OpenAI SDK, so you do not need to rewrite prompts or parsing logic.
Start by routing long-context or high-frequency agent calls to Oxlo.ai. Keep the same SDK, change the model string to one of the 45+ available options, and measure the difference in monthly spend. For teams that want dedicated capacity and guaranteed savings, the Enterprise plan offers unlimited requests on dedicated GPUs with guaranteed 30% off your current provider.
Predictable pricing should not require predicting token counts. With Oxlo.ai, you forecast API calls, not context windows.
Top comments (0)