I built an open-source tool to turn your random home GPUs & Ollama into an OpenAI-compatible cloud (no port forwarding)

abhinav anand — Thu, 03 Sep 2026 08:24:29 +0000

hey everyone 👋

so basically i was super annoyed with a classic problem:

i have a decent GPU at home running Ollama & vLLM, but whenever i wanted to use it from my laptop at a cafe or connect it to my side-projects hosted on Render/Vercel, things got messy real fast.

either you mess around w/ router port forwarding (which is annoying & unsafe), pay for ngrok/cloudflare tunnels that randomly drop when sessions timeout, or rent overpriced cloud GPUs on AWS/RunPod just for basic testing.

so over the past few weeks i built Viento — an ultra-lightweight distributed AI inference mesh SDK in Python.

github: https://github.com/abhinav00anand/viento

docs: https://viento.readthedocs.io/en/latest/

pypi: pip install viento

What it actually does (tl;dr)

instead of exposing your home IP or opening incoming ports, Viento runs as a tiny background node daemon on your machine.

it opens an outbound WebSocket tunnel (wss://) to a lightweight cloud gateway mesh. Whenever your app makes a standard OpenAI-format API call (https://viento.onrender.com/v1/chat/completions), the gateway routes the job down through teh websocket to your local engine, streams tokens back via SSE, and handles all backpressure / queueing automatically.

Basically turns any random laptop, homelab, or cloud spot GPU into a private serverless LLM endpoint.

DEV Community: abhinav anand

is this a good product?