Integrating LLM into Web Applications: A Beginner's Guide

#product #oxlo #ai

Integrating large language models into web applications has become a standard pattern for modern software. Whether you are building chat interfaces, content generators, or agentic workflows, the core challenge remains the same: connecting your frontend to an inference backend that is fast, predictable, and cost-effective. This guide walks through a practical stack using the OpenAI SDK, a Node.js backend, and a vanilla JavaScript frontend, with Oxlo.ai as the inference provider.

Architecture Overview

A typical LLM web integration follows a three-tier pattern. The client communicates with your backend, which securely manages API keys and orchestrates calls to the LLM provider. This avoids exposing credentials in the browser and lets you implement middleware for caching, rate limiting, and logging.

Frontend: Browser-based UI handling user input and streaming responses.
Backend: Node.js or Python server routing requests to the LLM API.
Inference Layer: Oxlo.ai provides the models via an OpenAI-compatible endpoint.

Choosing an Inference Provider

Most providers bill by the token, which means costs scale with prompt length. For web applications that process long documents, multi-turn conversations, or agentic loops, this unpredictability complicates budgeting. Oxlo.ai uses request-based pricing: one flat cost per API call regardless of input size. For long-context workloads, this can be significantly cheaper than token-based alternatives. The platform offers 45+ models, including Llama 3.3 70B, DeepSeek R1 671B MoE, and Qwen 3 32B, with no cold starts and full OpenAI SDK compatibility. You can explore plans at https://oxlo.ai/pricing.

Backend Setup

Install the OpenAI SDK and create an endpoint that proxies chat requests. Because Oxlo.ai is fully OpenAI SDK compatible, you only need to change the base URL and API key.

// server.js
import express from 'express';
import OpenAI from 'openai';
import 'dotenv/config';

const app = express();
app.use(express.json());

const client = new OpenAI({
  apiKey: process.env.OXLO_API_KEY,
  baseURL: 'https://api.oxlo.ai/v1'
});

app.post('/api/chat', async (req, res) => {
  try {
    const stream = await client.chat.completions.create({
      model: 'llama-3.3-70b',
      messages: req.body.messages,
      stream: true
    });

    res.setHeader('Content-Type', 'text/event-stream');
    res.setHeader('Cache-Control', 'no-cache');
    res.setHeader('Connection', 'keep-alive');

    for await (const chunk of stream) {
      const content = chunk.choices[0]?.delta?.content || '';
      res.write(`data: ${JSON.stringify({ content })}\n\n`);
    }

    res.write('data: [DONE]\n\n');
    res.end();
  } catch (err) {
    res.status(500).json({ error: err.message });
  }
});

app.listen(3000, () => console.log('Server running on port 3000'));

This example uses streaming to deliver tokens to the client as they are generated, which keeps the UI responsive.

Frontend Integration

The frontend consumes the stream using the EventSource API or fetch with ReadableStream. Below is a minimal implementation using fetch.

<!-- index.html -->
<!DOCTYPE html>
<html>
<body>
  <div id="chat"></div>
  <input id="input" type="text" placeholder="Ask something..." />
  <button id="send">Send</button>

  <script>
    const chat = document.getElementById('chat');
    const input = document.getElementById('input');
    const send = document.getElementById('send');

    send.addEventListener('click', async () => {
      const userMsg = input.value;
      appendMessage('User', userMsg);
      input.value = '';

      const response = await fetch('/api/chat', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ messages: [{ role: 'user', content: userMsg }] })
      });

      const reader = response.body.getReader();
      const decoder = new TextDecoder();
      let assistantMsg = '';
      const msgDiv = appendMessage('Assistant', '');

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        const chunk = decoder.decode(value);
        const lines = chunk.split('\n').filter(line => line.startsWith('data: '));
        for (const line of lines) {
          const data = line.replace('data: ', '');
          if (data === '[DONE]') continue;
          const parsed = JSON.parse(data);
          assistantMsg += parsed.content;
          msgDiv.innerText = assistantMsg;
        }
      }
    });

    function appendMessage(role, text) {
      const div = document.createElement('div');
      div.innerHTML = `<strong>${role}:</strong> ${text}`;
      chat.appendChild(div);
      return div;
    }
  </script>
</body>
</html>

Advanced Patterns

Beyond basic chat, Oxlo.ai supports features that let you build structured applications.

JSON Mode

For dashboards, forms, or API glue code, you can constrain the model to return valid JSON by setting response_format: { type: 'json_object' }. This removes the need for fragile regex parsing on the client.

Function Calling

You can register tools in your backend and let the model decide when to invoke them. This is useful for retrieval-augmented generation, database lookups, or third-party API calls. Oxlo.ai supports function calling across its chat models, so you can build agentic workflows without switching providers.

Vision and Multimodal

If your web app processes user uploads, models like Kimi K2.6 and Gemma 3 27B accept base64-encoded images in the messages array. The OpenAI SDK format for vision is fully supported.

Cost Considerations

Token-based billing penalizes applications with large system prompts, few-shot examples, or long conversation histories. Oxlo.ai flips this model with flat per-request pricing. A request containing 1,000 tokens costs the same as one containing 100,000 tokens. For web apps that send full documents or maintain extended context windows, this predictability makes capacity planning straightforward. See https://oxlo.ai/pricing for current plan details.

Conclusion

Integrating an LLM into a web application requires little more than an OpenAI-compatible client and a thin backend proxy. Oxlo.ai removes friction by offering drop-in SDK compatibility, streaming support, and request-based pricing that stays flat as your prompts grow. Start with the free tier to validate your integration, then scale as your traffic increases.

DEV Community