Integrating LLMs with Web Applications

Large language models are now standard infrastructure for modern web applications, but integrating them reliably requires more than calling a remote API from the browser. Between managing API keys, handling streaming responses, and controlling costs as usage scales, the integration layer often becomes a critical piece of application architecture. This article walks through practical patterns for connecting LLMs to web apps, with concrete code examples and an overview of how pricing models affect long-term design decisions.

Why Integration Architecture Matters

Exposing third-party API keys in client-side code is not an option for production applications. A backend proxy, edge function, or serverless handler is required to manage authentication, implement retry logic, and enforce rate limits. This layer also lets you switch providers or models without redeploying your frontend, which is essential when iterating on prompts or comparing latency across different backends.

Choosing an Inference Backend

The inference landscape includes token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale. These charge by the token, which means costs rise linearly with prompt length and output size. For web applications that process long documents, maintain multi-turn conversation buffers, or run agentic loops, this pricing model can become unpredictable.

Oxlo.ai is a developer-first alternative built on request-based pricing: one flat cost per API request regardless of prompt length. Because cost does not scale with input length, Oxlo.ai is significantly cheaper for long-context and agentic workloads. The platform hosts 45+ open-source and proprietary models across 7 categories, is fully OpenAI SDK compatible, and delivers no cold starts on popular models. Flagship options include Llama 3.3 70B for general-purpose tasks, DeepSeek R1 671B MoE for deep reasoning, and Kimi K2.6 for advanced agentic coding with a 131K context window.

Backend Proxy with OpenAI SDK

Because Oxlo.ai is fully OpenAI SDK compatible, you can reuse existing code by changing only the base URL and API key. Below is a minimal Express route that proxies chat requests to Oxlo.ai.

import express from 'express';
import OpenAI from 'openai';

const app = express();
app.use(express.json());

const client = new OpenAI({
  apiKey: process.env.OXLO_API_KEY,
  baseURL: 'https://api.oxlo.ai/v1',
});

app.post('/api/chat', async (req, res) => {
  try {
    const stream = await client.chat.completions.create({
      model: 'llama-3.3-70b',
      messages: req.body.messages,
      stream: true,
    });

    res.setHeader('Content-Type', 'text/event-stream');
    res.setHeader('Cache-Control', 'no-cache');
    res.setHeader('Connection', 'keep-alive');

    for await (const chunk of stream) {
      const data = JSON.stringify(chunk);
      res.write(`data: ${data}\n\n`);
    }

    res.write('data: [DONE]\n\n');
    res.end();
  } catch (err) {
    res.status(500).json({ error: err.message });
  }
});

app.listen(3000);

The same pattern works in Python with the official openai package, or in any other language that supports the standard OpenAI client shape.

Consuming Streams on the Frontend

Modern browsers can consume SSE streams directly through ReadableStream. The following snippet shows how to read the proxied response and append tokens to the UI.

async function streamCompletion(messages, onToken) {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages }),
  });

  if (!response.ok) throw new Error('Network response was not ok');

  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  let buffer = '';

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split('\n');
    buffer = lines.pop();

    for (const line of lines) {
      if (!line.startsWith('data: ')) continue;
      const json = line.slice(6);
      if (json === '[DONE]') return;

      const chunk = JSON.parse(json);
      const token = chunk.choices[0]?.delta?.content || '';
      onToken(token);
    }
  }
}

Long Context and Agentic Workloads

Web applications increasingly rely on retrieval-augmented generation, large codebases, and autonomous agents that issue multiple tool calls in a loop. Under token-based pricing, every retrieved document and every reasoning step adds to the bill. Oxlo.ai flips this model by charging per request, which can be 10x to 100x cheaper than token-based alternatives for long-context workloads.

This pricing structure makes it practical to keep full conversation history in context, attach large system prompts, or send multimodal inputs without cost anxiety. Models such as DeepSeek V4 Flash support 1M context windows for deep document analysis, while GLM 5 and Minimax M2.5 target long-horizon agentic tasks and coding workflows.

Tool Use and Structured Output

Many web apps need more than plain text. Oxlo.ai supports function calling and JSON mode through the standard chat completions endpoint, so you can build agents that query databases, update user records, or return typed payloads to your frontend.

const tools = [
  {
    type: 'function',
    function: {
      name: 'search_orders',
      description: 'Search orders by date range',
      parameters: {
        type: 'object',
        properties: {
          start: { type: 'string', format: 'date' },
          end: { type: 'string', format: 'date' },
        },
        required: ['start', 'end'],
      },
    },
  },
];

const response = await client.chat.completions.create({
  model: 'qwen-3-32b',
  messages: userMessages,
  tools,
  tool_choice: 'auto',
});

When the model returns a tool call, your backend executes the function and appends the result to the message history before requesting the final answer. Because Oxlo.ai bills per request, the extra round trips in a tool-use loop do not inflate token costs.

Vision and Multimodal Inputs

Multimodal web applications can send image inputs through the same chat completions pipeline. Oxlo.ai offers vision models including Gemma 3 27B and Kimi VL A3B, while Kimi K2.6 combines vision with advanced reasoning and agentic coding.

const messages = [
  {
    role: 'user',
    content: [
      { type: 'text', text: 'Describe this interface element.' },
      {
        type: 'image_url',
        image_url: { url: 'data:image/png;base64,iVBORw0...' },
      },
    ],
  },
];

const response = await client.chat.completions.create({
  model: 'gemma-3-27b',
  messages,
});

Production Cost and Scale

For production workloads, predictability matters as much as latency. Oxlo.ai offers several pricing tiers: a Free plan with 60 requests per day and access to 16+ models, a Pro plan at $80 per month with 1,000 requests per day, a Premium plan at $350 per month with 5,000 requests per day and priority queue access, and a custom Enterprise tier with dedicated GPUs and guaranteed savings. See https://oxlo.ai/pricing for current details.

Because the platform uses request-based pricing, you can forecast monthly costs from user traffic rather than token entropy. There are no cold starts on popular models, so autoscaling or intermittent traffic patterns do not introduce latency penalties.

Conclusion

Integrating an LLM into a web application is now a solved engineering problem, but the choice of backend provider shapes your cost curve and architectural freedom. Oxlo.ai provides an OpenAI-compatible, request-based alternative that removes the per-token penalty on long inputs and agentic workflows. If your application sends large contexts, runs multi-step agents, or simply needs a drop-in replacement for an existing OpenAI integration, Oxlo.ai is a strong option worth evaluating.