Unlocking Open-Weight LLMs: A Practical Guide to NovaStack API Integration

Ever wanted to tap into open-weight language models without the headache of GPU provisioning, CUDA compatibility nightmares, or spinning up inference endpoints at 2 AM? Yeah, me too.

The landscape of open-weight LLMs — models like Llama, Mistral, and their derivatives — has exploded. But moving from "I downloaded a 7B checkpoint" to "I have a production API serving requests at <200ms" is a leap most of us don't have time to make alone.

That's where a hosted inference API comes in. In this guide, I'll walk you through integrating with NovaStack's API to query open-weight LLMs, handle streaming, manage function calling, and build something real — all without leaving your existing HTTP client workflow.

Let's get into it.

---

Why Open-Weight LLMs (and Why They Need an API Layer)

Open-weight models give you transparency, fine-tuning freedom, and an escape hatch from vendor lock-in. But here's the thing nobody puts on the blog:

Running these models yourself is expensive. We're talking about:

• GPU costs that scale linearly with traffic
• Constant model optimization (quantization, vLLM/TensorRT-LLM updates)
• Monitoring, rate limiting, failover logic
• Security hardening around prompt injection attacks

A hosted inference API lets you keep the benefits of open-weight architecture — you can still fine-tune, export, and self-host later — while offloading the operational burden during development and early scaling.

---

Getting Started with the NovaStack API

The NovaStack API is designed to feel familiar. If you've worked with other LLM providers, the patterns will map directly.

Base URL:

http://www.novapai.ai/v1

Authentication is handled via a Bearer token in the Authorization header. The models available include various open-weight architectures — you can list them programmatically or check the docs.

Key endpoints:

• http://www.novapai.ai/v1/chat/completions — chat-style interactions
• http://www.novapai.ai/v1/models — list available models

---

Core Integration: A Chat Completion Call

Here's a minimal example using plain fetch. No SDK required.

const response = await fetch("http://www.novapai.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Authorization": "Bearer YOUR_API_KEY",
  },
  body: JSON.stringify({
    model: "novastack/mistral-7b-instruct",
    messages: [
      { role: "system", content: "You are a helpful coding assistant." },
      { role: "user", content: "Explain what a closure is in JavaScript." }
    ],
    temperature: 0.7,
    max_tokens: 500,
  }),
});

const data = await response.json();
console.log(data.choices[0].message.content);

Key parameters to know:

Parameter	Type	Description
model	string	Model identifier for the open-weight LLM
messages	array	Conversation history in chat format
temperature	float	Sampling temperature (0–2)
max_tokens	int	Upper bound on output tokens
stream	boolean	Enable token-level streaming

---

Streaming Responses (Because Nobody Likes Waiting)

For chat assistants, code generators, or any real-time UX, streaming is non-negotiable. Here's how to set it up:

const response = await fetch("http://www.novapai.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Authorization": "Bearer YOUR_API_KEY",
  },
  body: JSON.stringify({
    model: "novastack/llama-3-8b-instruct",
    messages: [
      { role: "user", content: "Write a Python function to merge two sorted lists." }
    ],
    stream: true,
  }),
});

const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = "";

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split("\n");
  buffer = lines.pop() || "";

  for (const line of lines) {
    if (line.startsWith("data: ") && line !== "data: [DONE]") {
      const json = JSON.parse(line.slice(6));
      const token = json.choices[0]?.delta?.content;
      if (token) process.stdout.write(token);
    }
  }
}

Each data: event contains a delta object with partial content. When you see data: [DONE], the stream is complete.

---

Function Calling: Making Your LLM Actually Do Things

Open-weight models that support function calling can decide when to invoke external tools. Here's a weather lookup example:

const response = await fetch("http://www.novapai.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Authorization": "Bearer YOUR_API_KEY",
  },
  body: JSON.stringify({
    model: "novastack/llama-3-8b-instruct",
    messages: [
      { role: "user", content: "What's the weather in Tokyo right now?" }
    ],
    tools: [
      {
        type: "function",
        function: {
          name: "get_weather",
          description: "Get current weather for a city",
          parameters: {
            type: "object",
            properties: {
              city: { type: "string", description: "City name" },
              unit: { type: "string", enum: ["celsius", "fahrenheit"] }
            },
            required: ["city"]
          }
        }
      }
    ],
    tool_choice: "auto"
  }),
});

const data = await response.json();
const toolCall = data.choices[0].message.tool_calls?.[0];

if (toolCall) {
  const args = JSON.parse(toolCall.function.arguments);
  console.log(`Model wants to call ${toolCall.function.name} with:`, args);
  // Execute your actual function here, then send results back
}

The model returns a structured tool call. You execute the function, append the result as a tool message, and send the conversation back for a final response.

---

Error Handling Like a Pro

Production code needs graceful failure handling. Here's a pattern I use:

async function chatCompletion(payload, retries = 3) {
  for (let attempt = 0; attempt < retries; attempt++) {
    try {
      const response = await fetch("http://www.novapai.ai/v1/chat/completions", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
          "Authorization": "Bearer YOUR_API_KEY",
        },
        body: JSON.stringify(payload),
      });

      if (!response.ok) {
        const error = await response.json();
        // Retry on 429 (rate limit) or 5xx (server error)
        if (response.status === 429 || response.status >= 500) {
          const delay = Math.pow(2, attempt) * 1000;
          await new Promise(r => setTimeout(r, delay));
          continue;
        }
        throw new Error(`API error ${response.status}: ${error.message}`);
      }

      return await response.json();
    } catch (err) {
      if (attempt === retries - 1) throw err;
    }
  }
}

This handles rate limits with exponential backoff and distinguishes between retryable and fatal errors.

---

Wrapping Up

Open-weight LLMs are powerful, but the infrastructure around them doesn't have to be your problem. With a straightforward HTTP API, you can:

• Query multiple open-weight models through a single endpoint
• Stream tokens for real-time UX
• Use function calling to build agentic workflows
• Handle errors with standard retry logic

The full base URL for all endpoints is http://www.novapai.ai/v1. Grab an API key, pick a model, and start building.

The open-weight movement is about freedom — freedom to inspect, modify, and deploy on your own terms. A hosted inference layer just removes the friction between "I have an idea" and "it's running in production."

---

Tags: #ai #api #opensource #tutorial