Integrating LLM with Web Applications: A Comprehensive Guide

Integrating large language models into web applications has moved from experimental feature to core infrastructure. Whether you are building a customer support widget, a code assistant, or an agentic dashboard, the integration pattern you choose determines latency, cost, and maintainability. This guide covers the architectural decisions, implementation details, and operational considerations for production-grade LLM integrations, with concrete examples using Oxlo.ai as the inference backend.

Architectural Patterns

Every LLM integration starts with a decision about where the API call originates. The two dominant patterns are direct client-side requests and proxied server-side requests.

Direct client-side integration is tempting because it eliminates backend complexity, but it exposes API keys and leaves you without a place to enforce business logic or audit logs. A safer pattern is to route all LLM traffic through your own backend or an edge function. This lets you sanitize inputs, manage authentication, and swap providers without touching frontend code.

Because Oxlo.ai is fully OpenAI SDK compatible, it acts as a drop-in replacement in either pattern. You can point your existing backend at https://api.oxlo.ai/v1 and change only the base URL and authorization header.

Backend Integration with Oxlo.ai

On the server, the OpenAI SDK remains the most reliable interface. Below are minimal examples in Python and Node.js that stream a response from Oxlo.ai.

Python (FastAPI-like route)

import openai
from fastapi import FastAPI
from fastapi.responses import StreamingResponse

client = openai.AsyncOpenAI(
    api_key="YOUR_OXLO_API_KEY",
    base_url="https://api.oxlo.ai/v1"
)

app = FastAPI()

@app.post("/chat")
async def chat(message: str):
    async def event_stream():
        response = await client.chat.completions.create(
            model="llama-3.3-70b",
            messages=[{"role": "user", "content": message}],
            stream=True
        )
        async for chunk in response:
            if chunk.choices[0].delta.content:
                yield f"data: {chunk.choices[0].delta.content}\n\n"
    return StreamingResponse(event_stream(), media_type="text/event-stream")

Node.js (Express route)

import OpenAI from "openai";
import express from "express";

const client = new OpenAI({
  apiKey: process.env.OXLO_API_KEY,
  baseURL: "https://api.oxlo.ai/v1",
});

const app = express();
app.use(express.json());

app.post("/chat", async (req, res) => {
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");

  const stream = await client.chat.completions.create({
    model: "deepseek-r1-671b-moe",
    messages: req.body.messages,
    stream: true,
  });

  for await (const chunk of stream) {
    const text = chunk.choices[0]?.delta?.content || "";
    res.write(`data: ${text}\n\n`);
  }
  res.end();
});

app.listen(3000);

These examples use Oxlo.ai models such as Llama 3.3 70B for general-purpose tasks or DeepSeek R1 671B MoE for deep reasoning. Because Oxlo.ai carries no cold starts on popular models, the first chunk arrives predictably after routing.

Frontend Streaming and UI Patterns

Consuming a streaming backend in the browser requires handling Server-Sent Events or a raw fetch stream. A robust React hook looks like this:

import { useState, useCallback } from "react";

export function useChat() {
  const [messages, setMessages] = useState([]);
  const [isLoading, setIsLoading] = useState(false);

  const sendMessage = useCallback(async (text) => {
    setIsLoading(true);
    const newMessages = [...messages, { role: "user", content: text }];
    setMessages(newMessages);

    const res = await fetch("/chat", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ messages: newMessages }),
    });

    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let assistantText = "";

    setMessages([...newMessages, { role: "assistant", content: "" }]);

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      const chunk = decoder.decode(value, { stream: true });
      const lines = chunk.split("\n").filter((l) => l.startsWith("data: "));
      for (const line of lines) {
        const data = line.replace("data: ", "");
        assistantText += data;
        setMessages((prev) => {
          const next = [...prev];
          next[next.length - 1].content = assistantText;
          return next;
        });
      }
    }
    setIsLoading(false);
  }, [messages]);

  return { messages, sendMessage, isLoading };
}

This pattern keeps the UI responsive and memory-efficient. Pair it with Oxlo.ai endpoints that support streaming responses, and you get low-latency rendering without buffering the entire completion in memory.

Tool Use and Function Calling

Modern web applications rarely stop at text generation. They query databases, call external APIs, or render custom UI components based on model output. Oxlo.ai supports function calling and tool use across its chat models, so you can implement agentic loops with the same OpenAI SDK schema.

A typical flow looks like this:</p