Understanding Request-Based Pricing for LLMs: A Comprehensive Guide

#costoptimization #oxlo #ai

Most LLM platforms bill by the token. Input tokens, output tokens, and cached context all carry a cost that scales with payload size. For teams running long-context retrieval, agentic loops, or large-codebase analysis, this creates unpredictable monthly bills and forces developers to optimize for token count rather than task completion. Request-based pricing removes that variable. With this model, you pay one flat cost for each API call regardless of whether you send a 200-word prompt or a 200,000-word document. The result is simpler forecasting, fewer billing surprises, and a pricing structure that aligns cost with business actions rather than raw character counts.

The Mechanics of Token-Based Billing

Under token-based pricing, every piece of text sent to the model is converted into tokens and metered separately. That includes system prompts, conversation history, tool definitions, document context, and the model's own output. Providers such as Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale use this model. As your context window grows or your agent iterates through multi-step tool calls, the bill grows in parallel. A single long-context request can cost as much as dozens of short ones, which makes capacity planning a forecasting exercise in natural language length.

How Request-Based Pricing Works

Request-based pricing flips the unit of account from the token to the API call. One flat cost is charged per request, independent of prompt length or output size. Whether you pass a concise user question or a full codebase with embedded documentation, the price remains the same. This model treats the LLM as an operational endpoint, not a utility metered by the word. Oxlo.ai uses this approach across its entire catalog of more than 45 models, making it a relevant option for teams that want cost stability. You can see exact plan details at the Oxlo.ai pricing page.

Workloads That See the Biggest Gains

Not every workload benefits equally. Request-based pricing shines when prompts are long, context is retained across turns, or agents loop repeatedly. Examples include retrieval-augmented generation with large document chunks, software engineering tasks that feed entire modules into the context window, multi-turn customer support threads, and autonomous agent workflows that append tool outputs back into the prompt. In these scenarios, token-based bills scale linearly with context length, while request-based costs stay flat. Oxlo.ai is specifically positioned for these patterns, with models such as DeepSeek V4 Flash supporting 1M context windows and Kimi K2.6 handling advanced agentic coding at a fixed request cost.

Predictability Over Micro-Optimization

When cost is tied to requests, budgeting becomes a matter of traffic engineering, not linguistics. You can estimate monthly spend from user session counts or agent loop iterations without calculating average tokens per prompt. This removes the incentive to strip whitespace, abbreviate system prompts, or truncate history purely to save money. Oxlo.ai offers tiered plans including a Free tier with 60 requests per day, a Pro tier at $80 per month for 1,000 requests per day, and a Premium tier at $350 per month for 5,000 requests per day, with custom Enterprise options for unlimited volume. Because each request carries the same cost, your finance team can model spend directly from product metrics.

Counting Requests, Not Tokens

The easiest way to understand the model is to see it in code. The snippet below uses the OpenAI SDK pointed at Oxlo.ai. Note that the cost of the call depends on the fact that it is one request, not on the length of the codebase string.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

# One request. One flat cost. No token math required.
response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "You are a senior software architect."},
        {"role": "user", "content": large_codebase + "\n\nSuggest a refactoring plan."}
    ],
    stream=True
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Because Oxlo.ai is fully OpenAI SDK compatible, this is a drop-in replacement. You keep your existing instrumentation and retry logic. The only difference is that your bill at the end of the month reflects the number of API calls you made, not the cumulative token volume.