Five Gates to a Public Model Endpoint: A Deployment Tutorial

You can deploy a free-model proxy to a free server in five stages. Each stage ends with a gate. If a gate fails, you stop and fix. This loop turns "it works on my machine" into "it works from anywhere."

This walkthrough uses two free resources. A free model endpoint. A free server option. Both come from MonkeyCode.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What you need

• A free model endpoint with a key. MonkeyCode provides one.
• A free server option with SSH or console access. MonkeyCode provides one.
• Python 3.11 or newer on your laptop.
• curl and a terminal.

Why stage gates

Free endpoints fail quietly. Free servers cold-start. A proxy that works locally can break the moment it moves. Stage gates catch each failure at the cheapest point. You verify before you deploy. Not after.

Each gate is a command you can run. Each gate has a pass condition. If the condition fails, you stop. This is the opposite of deploy-and-pray.

Stage 1: Pin the contract

Write the contract before the code. Two shapes matter. The request you accept. The response you return. Keep both small.

A small contract is easy to test. It is also easy to change. A fat contract drags every consumer into every change.

# contract.py
from pydantic import BaseModel


class PromptIn(BaseModel):
    prompt: str
    max_tokens: int = 256


class CompletionOut(BaseModel):
    text: str
    model: str
    latency_ms: int

Gate: import the module and build both models.

python -c "from contract import PromptIn, CompletionOut; print(PromptIn(prompt='x')); print(CompletionOut(text='y', model='m', latency_ms=1))"

If that prints two objects, the contract is valid.

Stage 2: Build the proxy

The proxy has one job. Take a prompt. Forward it upstream. Normalize the response. Return your contract shape.

# proxy.py
import os
import time

import httpx
from fastapi import FastAPI
from contract import CompletionOut, PromptIn

app = FastAPI()

UPSTREAM_URL = os.environ["UPSTREAM_URL"]
UPSTREAM_KEY = os.environ["UPSTREAM_KEY"]
TIMEOUT = float(os.environ.get("TIMEOUT_SECONDS", "30"))


@app.get("/health")
def health():
    return {"status": "ok"}


@app.post("/complete", response_model=CompletionOut)
def complete(body: PromptIn):
    payload = {"prompt": body.prompt, "max_tokens": body.max_tokens}
    headers = {"Authorization": f"Bearer {UPSTREAM_KEY}"}
    started = time.monotonic()
    with httpx.Client(timeout=TIMEOUT) as client:
        resp = client.post(UPSTREAM_URL, json=payload, headers=headers)
        resp.raise_for_status()
        data = resp.json()
    latency_ms = int((time.monotonic() - started) * 1000)
    return CompletionOut(
        text=data["text"], model=data["model"], latency_ms=latency_ms
    )

Save this as requirements.txt.

fastapi
uvicorn
httpx

Upstream response shapes vary by provider. Replace data["text"] and data["model"] with your endpoint's real fields. Normalize in one place. Never scatter field names across the codebase.

Stage 3: Run the local gate

Start the proxy. Run three checks. Health, happy path, bad input.

export UPSTREAM_URL="..."   # your free endpoint URL
export UPSTREAM_KEY="..."    # your key
uvicorn proxy:app --port 8000

In a second terminal:

curl -s http://localhost:8000/health
curl -s -X POST http://localhost:8000/complete -H "Content-Type: application/json" -d '{"prompt": "def add(a, b):", "max_tokens": 64}'
curl -s -o /dev/null -w "%{http_code}\n" -X POST http://localhost:8000/complete -H "Content-Type: application/json" -d '{"prompt": ""}'

Expect 200, a JSON completion, and 422. The happy path proves the upstream works. The bad input proves validation works. If any check fails, fix it locally. Deploying a broken proxy multiplies the debugging cost.

Stage 4: Deploy to the free server

Copy the project to the host. Install dependencies. Start the service. Expose the port.

Exact commands depend on your host. The pattern is what matters.

# from your laptop
rsync -az --exclude '.git' --exclude '__pycache__' ./ user@host:/srv/model-proxy/

# on the host
cd /srv/model-proxy
pip install -r requirements.txt
export UPSTREAM_URL="..." UPSTREAM_KEY="..."
nohup uvicorn proxy:app --host 0.0.0.0 --port 8000 > proxy.log 2>&1 &

Replace user@host with your server's address. Set environment variables from the host's secret store. Never commit keys to the repo. A leaked key is a revoked key.

Gate: confirm the process is listening.

curl -s http://localhost:8000/health   # run on the host

Stage 5: Smoke test from the public internet

Local checks prove nothing about the public path. DNS, firewalls, and cold starts sit between your laptop and the server. Test the public URL directly.

The script below runs three gates. Health, completion shape, input validation. It exits non-zero if any gate fails. That makes it usable in CI or a cron job.

# smoke_test.py
import sys
import time

import httpx

BASE_URL = sys.argv[1] if len(sys.argv) > 1 else "http://localhost:8000"
TIMEOUT = 30.0
failures = 0


def check(name, condition, detail=""):
    global failures
    mark = "PASS" if condition else "FAIL"
    print(f"[{mark}] {name} {detail}")
    if not condition:
        failures += 1


started = time.monotonic()
r = httpx.get(f"{BASE_URL}/health", timeout=TIMEOUT)
check("health returns 200", r.status_code == 200, f"(got {r.status_code})")
check("health answers in < 5s", time.monotonic() - started < 5.0)

r = httpx.post(
    f"{BASE_URL}/complete",
    json={"prompt": "def add(a, b):", "max_tokens": 64},
    timeout=TIMEOUT,
)
check("complete returns 200", r.status_code == 200, f"(got {r.status_code})")
if r.status_code == 200:
    data = r.json()
    check("response has text", isinstance(data.get("text"), str) and data["text"])
    check("response has latency_ms", isinstance(data.get("latency_ms"), int))

r = httpx.post(f"{BASE_URL}/complete", json={"prompt": ""}, timeout=TIMEOUT)
check("empty prompt rejected", r.status_code == 422, f"(got {r.status_code})")

if failures:
    print(f"{failures} gate(s) failed.")
    sys.exit(1)
print("All gates passed.")

Run it against the public URL:

python smoke_test.py https://your-proxy.example.com

Expect five PASS lines. The first cold call may be slow. That is normal on free servers. The health gate allows a five-second budget.

Limitations

This loop targets prototypes and internal tools. Free servers are best-effort infrastructure. They have no SLA. They share IPs. They can pause without notice. Free model endpoints rate-limit and fail quietly. The proxy adds a timeout and a shape check. It does not add retries, auth, or rate limiting.

Add those as separate layers. Do not bolt them onto this proxy. Each layer needs its own tests. Each layer needs its own failure mode.

Who should not use this

Skip this approach for production traffic. Free servers cannot promise uptime. Skip it for sensitive data. Free endpoints should not see private payloads. Skip it for sub-second latency. Cold starts will miss your budget.

The sweet spot is a demo, a prototype, or an internal tool with tolerant users. If that is your situation, the loop will save you time.

The takeaway

Five stages. Five gates. Each gate is a command or a script. The total cost is one afternoon. The payoff is a public endpoint that fails loudly.

Copy the files, adapt the contract, and run the gates. The whole loop is small enough to own.