Introduction to Token-Based LLM APIs

#learnai #oxlo #ai

Most LLM providers bill by the token, so costs scale with every word in the prompt and response. In this tutorial, I will build a support ticket agent that counts tokens and drafts replies, using Oxlo.ai's OpenAI-compatible API to run the inference. If you are evaluating providers for high-volume or long-context workloads, understanding how token math accumulates is the first step toward optimizing spend.

What you'll need

Python 3.10+
The OpenAI SDK: pip install openai (we will also use tiktoken for counting)
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Configure the Oxlo.ai client

Oxlo.ai exposes a fully OpenAI-compatible endpoint, so the SDK is a direct drop-in. I start by instantiating the client and verifying connectivity with a small test call.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="deepseek-v3.2",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Confirm connection."},
    ],
)

print(response.choices[0].message.content)

Step 2: Define the system prompt

The agent acts as a tier-one support engineer. It classifies the issue, drafts a concise reply, and flags urgency when needed.

SYSTEM_PROMPT = """You are a support agent for a developer platform. Follow these rules:
1. Classify the ticket into one of: Bug, Billing, Integration, or General.
2. Draft a reply under 100 words.
3. If the issue is urgent, prepend [URGENT] to the subject line.
4. Do not ask for follow-up unless critical information is missing.

Ticket:
"""

Step 3: Add a token estimator

Token-based APIs charge for both the prompt you send and the completion you receive. I use tiktoken to count tokens locally so the agent can log usage before it makes the request.

import tiktoken

def count_tokens(text):
    # cl100k_base is a close proxy for most modern transformer tokenizers
    encoding = tiktoken.get_encoding("cl100k_base")
    return len(encoding.encode(text))

def estimate_request_size(system_prompt, user_message, expected_output_tokens=150):
    prompt_text = system_prompt + user_message
    input_tokens = count_tokens(prompt_text)
    return {
        "input_tokens": input_tokens,
        "output_tokens": expected_output_tokens,
        "total_tokens": input_tokens + expected_output_tokens
    }

Step 4: Build the agent core

This function estimates token usage, sends the ticket to Oxlo.ai, and prints the result. Because Oxlo.ai uses request-based pricing, the token estimate is useful for forecasting what a token-based provider would bill, while Oxlo.ai charges one flat cost per request regardless of length.

def process_ticket(ticket_body, model="llama-3.3-70b"):
    sizing = estimate_request_size(SYSTEM_PROMPT, ticket_body)
    print(f"Estimated tokens: {sizing['total_tokens']} "
          f"({sizing['input_tokens']} in, {sizing['output_tokens']} out)")

    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": ticket_body},
        ],
        max_tokens=200,
        temperature=0.3,
    )

    reply = response.choices[0].message.content
    actual_output_tokens = count_tokens(reply)
    print(f"Actual output tokens: {actual_output_tokens}")
    return reply

Step 5: Batch process multiple tickets

With token-based billing, a long input or a verbose reply directly increases cost. This loop demonstrates how usage accumulates across a queue. On Oxlo.ai, each ticket is still just one request, which makes long-context and agentic loops significantly more predictable.

tickets = [
    "My API key returns 401 after rotating it this morning.",
    "I was charged twice last month. Can you refund the duplicate?",
    "How do I enable JSON mode in the Python SDK? The example in the docs throws a validation error.",
]

for ticket in tickets:
    print("=" * 50)
    print(f"Ticket: {ticket[:50]}...")
    reply = process_ticket(ticket, model="qwen-3-32b")
    print(f"Reply:\n{reply}\n")

Run it

Save the full script as support_agent.py, set your key, and run it.

export OXLO_API_KEY="sk-oxlo.ai-..."
python support_agent.py

Example output:

Estimated tokens: 187 (142 in, 45 out)
Actual output tokens: 52
==================================================
Ticket: My API key returns 401 after rotating it this morning...
Reply:
Subject: [URGENT] Integration - API Key 401 Error

Your new key may take up to 60 seconds to propagate. Clear your SDK client cache and retry. If the issue persists, verify the key string has no trailing whitespace.

Estimated tokens: 195 (143 in, 52 out)
Actual output tokens: 61
==================================================
Ticket: I was charged twice last month. Can you refund the duplicate?...
Reply:
Subject: Billing - Duplicate Charge

I have located the duplicate transaction and issued a refund. You should see the credit within 3 to 5 business days.

Next steps

Swap in Oxlo.ai's long-context models such as kimi-k2.6 or deepseek-v4-flash and pass in full conversation histories or documentation pages. Because Oxlo.ai uses request-based pricing, your cost stays flat even when the token count grows. For exact plan details, see https://oxlo.ai/pricing.

If you want to extend this, wire the agent to a webhook and store actual token counts in a time-series database to compare projected token-based spend against Oxlo.ai's per-request invoices.

DEV Community