Building a Language Model with LLM

The llm command-line utility has become a standard fixture in developer toolchains for prototyping with large language models. It offers a clean abstraction over dozens of providers, letting you switch models with a single flag. What the interface does not solve is inference economics. If you are iterating on agentic workflows, stuffing long documents into context windows, or running batch evaluations, token-based billing means cost grows with every token added to the prompt. Oxlo.ai is an inference platform built for exactly those patterns. With request-based pricing, a fully OpenAI-compatible API, and no cold starts on popular models, it is a natural backend to pair with tools like llm.

Why the Backend Matters More Than the Interface

Tools like llm standardize the client side, but the provider still controls latency, model availability, and pricing structure. For long-context workloads, token-based providers scale cost linearly with input length. Oxlo.ai uses request-based pricing, so one flat cost per API request covers the entire prompt regardless of how many tokens you send. This can make it significantly cheaper for long-context and agentic workloads. No cold starts on popular models also means your commands return immediately without waiting for GPU spin-up.

Configuring Oxlo.ai as Your OpenAI-Compatible Backend

Oxlo.ai exposes a fully OpenAI-compatible endpoint at https://api.oxlo.ai/v1. Because the path and schema match the OpenAI specification, you can redirect any standard client by changing two variables: the API key and the base URL. If you are using the OpenAI Python SDK, the swap requires no other code changes.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_OXLO_API_KEY",
    base_url="https://api.oxlo.ai/v1"
)

For CLI tools that support custom OpenAI endpoints, set the base URL to https://api.oxlo.ai/v1 and provide your Oxlo.ai API key. The tool will treat Oxlo.ai's catalog of 45+ models as native options.

Prototyping a Long-Context Agent

Oxlo.ai supports function calling, streaming, JSON mode, and multi-turn conversations. These features let you move beyond simple chat and build agentic prototypes that maintain state and invoke tools. The following example uses the OpenAI SDK against Oxlo.ai to run a tool-assisted conversation. Substitute the model identifier with the exact name for Llama 3.3 70B, DeepSeek V3.2, or another model from the Oxlo.ai catalog.

import json
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_OXLO_API_KEY",
    base_url="https://api.oxlo.ai/v1"
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "fetch_pricing",
            "description": "Retrieve current Oxlo.ai pricing details",
            "parameters": {
                "type": "object",
                "properties": {
                    "plan": {"type": "string", "enum": ["Free", "Pro", "Premium"]}
                },
                "required": ["plan"]
            }
        }
    }
]

messages = [
    {"role": "system", "content": "You are a sales assistant. Use the fetch_pricing tool when asked about costs."},
    {"role": "user", "content": "What is included in the Pro plan?"}
]

response = client.chat.completions.create(
    model="<model-id>",
    messages=messages,
    tools=tools,
    stream=False
)

message = response.choices[0].message
if message.tool_calls:
    print(f"Tool call requested: {message.tool_calls[0].function.name}")
else:
    print(message.content)

Because Oxlo.ai does not charge by the token, you can expand the system prompt with few-shot examples or inject large context documents without incrementally raising the cost of every request.

How Request-Based Pricing Changes Iteration

When you pay per token, long-context development encourages optimization rituals that consume engineering time. You truncate documents, count tokens manually, and avoid verbose prompts. Request-based pricing removes that friction. On Oxlo.ai, a request costs the same whether you send a one-line prompt or a long document. This is particularly useful when prototyping retrieval-augmented generation pipelines or agent loops, where context naturally inflates. For current plan details, see the Oxlo.ai pricing page.

Extending to Embeddings, Vision, and Audio

A complete application usually needs more than text generation. Oxlo.ai provides endpoints for embeddings, image generation, audio transcription, and text-to-speech under the same base URL. You can embed documents with BGE-Large or E5-Large, transcribe audio with Whisper Large v3, or generate images without managing separate provider accounts. Keeping the entire pipeline on one platform simplifies key management and billing.

# Embeddings for retrieval
emb = client.embeddings.create(
    model="<embedding-model-id>",
    input="Request-based pricing simplifies long-context workloads."
)

# Image generation (different endpoint, same base URL)
image = client.images.generate(
    model="<image-model-id>",
    prompt="A developer reading API documentation.",
    n=1,
    size="1024x1024"
)

Putting It Together

The llm ecosystem and the broader OpenAI-compatible toolchain give developers portability. Oxlo.ai capitalizes on that portability by offering a drop-in backend with a fundamentally different pricing model. If your workloads involve long contexts, agentic steps, or high request volume, routing your existing tools to Oxlo.ai is a low-friction way to reduce costs and eliminate cold starts. Start by pointing your base URL to https://api.oxlo.ai/v1 and referencing the model catalog to find the right identifier for your task.