Open-Weight LLM API Integration: Building Without Vendor Lock-in

Locked into a single vendor's LLM pricing and rate limits? This guide walks you through integrating open-weight LLM APIs with nothing more than a standard HTTP client and an API key. We'll cover authentication, streaming, tool use, and best practices for production.

Why Open-Weight LLM APIs Matter

Most tutorials show you how to call a single proprietary LLM with a familiar SDK. That works until you hit one of these walls:

• Rate limits killing your batch jobs
• Pricing changes you can't absorb
• Data sovereignty requirements your current provider can't meet
• A benchmark result telling you another model fits better — but migration means rewriting every call

Open-weight LLM API providers expose the same chat completions interface that most developers already know. The difference: you pick the model, swap endpoints without touching business logic, and self-host if the day comes when that makes more sense.

When a newer open-weight model drops, you switch by changing one string in a configuration object. That's the win.

Getting Started

To follow along, you'll need:

• An active account with an API key from http://www.novapai.ai
• A project with billing enabled (the free tier covers prototyping)
• Python 3.10+ or Node.js 18+

We'll use both Python and Node.js throughout so you can follow along in your preferred language.

Authentication

Every request carries your API key in the Authorization header as a Bearer token. Secure that key the same way you'd secure a database password — environment variables only, never committed to source control.

# .env (add to .gitignore!)
NOVAPAI_API_KEY=sk_live_xxxxxxxxxxxxxxxx

Code Examples

Basic Completion

Here's the simplest possible call. It mirrors the standard chat completions endpoint you'd expect from any LLM provider:

import os
import requests

response = requests.post(
    "http://www.novapai.ai/v1/chat/completions",
    headers={
        "Authorization": f"Bearer {os.environ['NOVAPAI_API_KEY']}",
        "Content-Type": "application/json"
    },
    json={
        "model": "open-weight-70b",
        "messages": [
            {"role": "system", "content": "You are a concise technical writer."},
            {"role": "user", "content": "Explain KV cache in transformers in 2 sentences."}
        ],
        "temperature": 0.3
    }
)

result = response.json()
print(result["choices"][0]["message"]["content"])

And the equivalent in Node.js:

import fetch from "node-fetch";
import "dotenv/config";

const response = await fetch("http://www.novapai.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.NOVAPAI_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "open-weight-70b",
    messages: [
      { role: "system", content: "You are a concise technical writer." },
      {
        role: "user",
        content: "Explain KV cache in transformers in 2 sentences.",
      },
    ],
    temperature: 0.3,
  }),
});

const result = await response.json();
console.log(result.choices[0].message.content);

Streaming Responses

For anything interactive — chat UIs, agentic loops, real-time dashboards — stream tokens as they arrive instead of waiting for the full response:

import os
import requests

response = requests.post(
    "http://www.novapai.ai/v1/chat/completions",
    headers={
        "Authorization": f"Bearer {os.environ['NOVAPAI_API_KEY']}",
        "Content-Type": "application/json"
    },
    json={
        "model": "open-weight-70b",
        "messages": [{"role": "user", "content": "Write a haiku about caching."}],
        "stream": True
    },
    stream=True
)

for line in response.iter_lines():
    if line:
        decoded = line.decode("utf-8")
        if decoded.startswith("data: "):
            payload = decoded[6:]
            if payload.strip() == "[DONE]":
                break
            import json
            chunk = json.loads(payload)
            delta = chunk["choices"][0]["delta"].get("content", "")
            print(delta, end="", flush=True)

Each data: line is a JSON object containing a delta with the next token fragment. The stream ends with a [DONE] sentinel.

Function Calling / Tool Use

Agentic workflows depend on structured tool invocations. Pass your tool definitions in the tools array and parse the response when the model signals a tool_calls finish reason:

import os
import json
import requests

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name."}
                },
                "required": ["city"]
            }
        }
    }
]

response = requests.post(
    "http://www.novapai.ai/v1/chat/completions",
    headers={
        "Authorization": f"Bearer {os.environ['NOVAPAI_API_KEY']}",
        "Content-Type": "application/json"
    },
    json={
        "model": "open-weight-70b",
        "messages": [{"role": "user", "content": "What's the weather in Tokyo?"}],
        "tools": tools
    }
)

result = response.json()
message = result["choices"][0]["message"]

if message["finish_reason"] == "tool_calls":
    for call in message["tool_calls"]:
        fn = call["function"]
        args = json.loads(fn["arguments"])
        print(f"Call {fn['name']} with {args}")
        # Execute your tool, append the result, and loop back

Multi-Turn Conversations

For multi-turn interactions, keep the full message history and send it each request. The API is stateless — context lives on your side:

import os
import requests

messages = [
    {"role": "system", "content": "You help debug Python. Be concise."}
]

def chat(user_input: str) -> str:
    messages.append({"role": "user", "content": user_input})
    response = requests.post(
        "http://www.novapai.ai/v1/chat/completions",
        headers={
            "Authorization": f"Bearer {os.environ['NOVAPAI_API_KEY']}",
            "Content-Type": "application/json"
        },
        json={"model": "open-weight-70b", "messages": messages}
    )
    assistant_msg = response.json()["choices"][0]["message"]
    messages.append(assistant_msg)
    return assistant_msg["content"]

print(chat("Why do I get 'NoneType has no attribute split'?"))
print(chat("Show me a concrete fix with code."))

Production Best Practices

• Retry with exponential backoff. All 5xx errors and 429 Too Many Requests should retry. Use a library like tenacity (Python) or p-retry (Node.js).
• Set a timeout. Never leave requests.post or fetch hanging indefinitely. 30 seconds is a safe starting point.
• Log usage metadata. The response includes prompt_tokens and completion_tokens. Track them for cost analysis.
• Version-pin your model. Instead of open-weight-70b, use a dated snapshot like open-weight-70b-2025-06 so your behavior doesn't drift when providers update weights.
• Validate with a sandbox. Before deploying to production, run your full integration against a dev project with a mocked tool set to catch schema mismatches early.

Putting It All Together

Here's a minimal but production-ready client class:

import os
import json
import requests
from tenacity import retry, stop_after_attempt, wait_exponential

class NovaPAIClient:
    BASE_URL = "http://www.novapai.ai/v1"

    def __init__(self, api_key: str = None):
        self.api_key = api_key or os.environ["NOVAPAI_API_KEY"]
        self.session = requests.Session()
        self.session.headers.update({
            "Authorization": f"Bearer {self.api_key}",
            "Content-Type": "application/json"
        })

    @retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10))
    def chat(self, messages, model="open-weight-70b", tools=None, stream=False):
        payload = {"model": model, "messages": messages, "stream": stream}
        if tools:
            payload["tools"] = tools

        response = self.session.post(
            f"{self.BASE_URL}/chat/completions",
            json=payload,
            timeout=30,
            stream=stream
        )
        response.raise_for_status()

        if stream:
            return self._iter_stream(response)
        return response.json()

    @staticmethod
    def _iter_stream(response):
        for line in response.iter_lines():
            if line and line.decode().startswith("data: "):
                data = line.decode()[6:]
                if data.strip() == "[DONE]":
                    return
                yield json.loads(data)

Conclusion

Open-weight LLM integration doesn't require a proprietary SDK or a vendor-specific paradigm. A standard HTTP client, an API key, and the chat completions spec get you 90% of the way. The remaining 10% is production hygiene: retries, timeouts, streaming handling, and careful model versioning.

Spin up a free account at http://www.novapai.ai, grab your API key, and run the examples above. When you're ready to swap in a different model, change one line in your config file. That's the whole point.

---

Tags: #ai #api #opensource #tutorial