Adding a large language model to a web application can turn a static interface into an interactive assistant, but the path from API key to production feature is rarely obvious. Beginners often struggle with where to run inference, how to manage secrets, and how to stream output without freezing the UI. This guide walks through a practical, full-stack integration using standard HTTP and the OpenAI SDK pattern, with concrete code you can run today.
Architecture Overview
A typical LLM web integration has three layers. The client browser handles rendering and user input. A backend server, which you control, stores the API key and orchestrates calls. The inference provider runs the model. Never expose provider keys in frontend bundles. Instead, have your backend proxy requests so you can add retries, logging, and rate limiting.
Choosing an Inference Provider
You need a provider that supports the model you want, scales without cold starts, and fits your pricing model. Token-based billing can make long-context and agentic workloads unpredictable because costs grow with every word in the prompt. Oxlo.ai uses request-based pricing, which means one flat cost per API call regardless of input length. For applications that send large documents or multi-turn conversation history, this can dramatically simplify budgeting. Oxlo.ai offers 45+ models across chat, code, vision, and embeddings, is fully OpenAI SDK compatible, and requires no code changes beyond updating the base URL. You can start on the free tier with 60 requests per day and explore the full catalog before committing. See the latest plans at https://oxlo.ai/pricing.
Building the Backend
We will use Python with FastAPI. Install dependencies:
pip install fastapi uvicorn openai python-dotenvCreate a .env file:
OXLO_API_KEY=your_oxlo_keymain.py:
import os
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from openai import OpenAI
app = FastAPI()
app.add_middleware(
CORSMiddleware,
allow_origins=["http://localhost:3000"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
client = OpenAI(
api_key=os.getenv("OXLO_API_KEY"),
base_url="https://api.oxlo.ai/v1",
)
class ChatRequest(BaseModel):
message: str
@app.post("/chat")
async def chat(req: ChatRequest):
try:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": req.message}],
stream=False,
)
return {"reply": response.choices[0].message.content}
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))Run with uvicorn main:app --reload. The backend now proxies requests to Oxlo.ai without exposing your key.
Wiring the Frontend
A minimal HTML page:
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>LLM Chat</title>
</head>
<body>
<div id="chat"></div>
<input id="msg" type="text" placeholder="Ask something..." />
<button onclick="send()">Send</button>
<script>
async function send() {
const input = document.getElementById('msg');
const box = document.getElementById('chat');
const res = await fetch('http://localhost:8000/chat', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({message: input.value})
});
const data = await res.json();
const p = document.createElement('p');
p.textContent = data.reply || data.error;
box.appendChild(p);
input.value = '';
}
</script>
</body>
</html>Streaming Responses
Blocking UI updates feel slow. OpenAI SDK streaming works with Oxlo.ai by setting stream=True and returning an SSE stream.
Update the backend:
from fastapi.responses import StreamingResponse
@app.post("/chat/stream")
async def chat_stream(req: ChatRequest):
def event_generator():
stream = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": req.message}],
stream=True,
)
for chunk in stream:
text = chunk.choices[0].delta.content
if text:
yield f"data: {text}\n\n"
yield "data: [DONE]\n\n"
return StreamingResponse(event_generator(), media_type="text/event-stream")Update the frontend to read the stream with fetch and a ReadableStream reader:
async function sendStream() {
const input = document.getElementById('msg');
const box = document.getElementById('chat');
const p = document.createElement('p');
box.appendChild(p);
const res = await fetch('http://localhost:8000/chat/stream', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({message: input.value})
});
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const {done, value} = await reader.read();
if (done) break;
buffer += decoder.decode(value, {stream: true});
const lines = buffer.split('\n');
buffer = lines.pop();
for (const line of lines) {
if (line.startsWith('data: ')) {
const payload = line.slice(6);
if (payload === '[DONE]') return;
p.textContent += payload;
}
}
}
}Adding Tool Use
Many applications need more than text. Function calling lets the model request actions. Oxlo.ai supports tool use on compatible models such as Qwen 3 32B and Llama 3.3 70B.
Define a tool schema:
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string"}
},
"required": ["city"]
}
}
}
]
response = client.chat.completions.create(
model="qwen3-32b",
messages=[{"role": "user", "content": req.message}],
tools=tools,
tool_choice="auto",
)If the model returns a tool call, execute the function locally and send the result back in a second request. This pattern works identically on Oxlo.ai because the endpoints follow the OpenAI chat/completions schema.
Cost Control and Context Windows
When you build features that summarize PDFs, iterate over codebase files, or maintain long agentic memory, token-based bills scale with every extra sentence. Oxlo.ai’s request-based pricing removes that variable. A single request costs the same whether you send a one-line prompt or a full conversation transcript near the context limit. This makes long-context workflows, like analyzing 131K tokens with Kimi K2.6 or using DeepSeek V4 Flash with 1M context, far easier to forecast. You can compare plans at https://oxlo.ai/pricing.
For additional safety, always enforce a maximum request budget in your backend. Track daily usage in a simple counter or Redis key, and fallback to a smaller model if you approach your plan limit.
Conclusion
Integrating an LLM into a web application is fundamentally an API integration problem: secure the key on the backend, stream tokens to the frontend, and handle errors gracefully. By using an OpenAI SDK-compatible provider, you keep your stack portable and your code short. Oxlo.ai fits this workflow with a drop-in base URL change, request-based pricing that favors long-context applications, and a free tier that lets you experiment before deploying. Start with the examples above, point your client to https://api.oxlo.ai/v1, and iterate.
Top comments (0)