Local LLM Studio: chat and a speed check that stay on the laptop

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

I wanted a local model for the boring stuff: drafting a reply I do not want in someone else's log, checking whether a screenshot is readable, and finding out if a 3B model is actually usable on this laptop. What I had instead was three apps, three ports, and a terminal I had to babysit.

Local LLM Studio is the slow version of that workflow. It sits on 127.0.0.1 and talks to whatever open model server is already running.

What I Built

Local LLM Studio is a small private front end for a model server on your own machine. It speaks the OpenAI-compatible /v1 API, so the same install works with Ollama, LM Studio, and llama.cpp's llama-server.

Four jobs, and then it gets out of the way:

• Setup tells you whether the saved server answers, and prints the install and start commands for macOS, Windows, or Linux. You copy those commands into a terminal yourself.
• Chat streams a reply and reloads the same transcript after a refresh. Image attach shows up only when that model is marked as able to see.
• Models lists whatever GET /v1/models returns. On Ollama it can also pull a model, delete one, and read the native vision flag. Pull and delete stay hidden for LM Studio and llama.cpp, because those runtimes load weights themselves.
• Benchmarks run one model at a time and record time to first token and tokens per second for this server and this tokenizer. A discarded warmup is on by default, because the first sample is mostly the runtime loading weights.

The person who gets to be lazier is anyone who keeps a local model around for private drafts and then loses twenty minutes remembering which port is which. Studio binds to 127.0.0.1:8000. Chats, uploaded images, and benchmark rows stay in a ./data folder next to the app. There is no account and no telemetry.

A loopback server URL is saved immediately. A private-network URL is saved only after you confirm it, because the prompt then leaves the machine. Public cloud hosts, including https://api.openai.com/v1, are rejected.

Demo

There is no deployed link, and that is the product. Studio listens only on loopback, and the URL check turns a public host away.

python -m pip install -e ".[dev]"
cd web && npm install && npm run build && cd ..
python -m studio

Open http://127.0.0.1:8000. On Windows, localhost often resolves to IPv6 while Studio listens on IPv4, so use the numeric address.

Setup

Benchmark

Chat

Code

MIT licensed. The repo is tusharv/local-llm.

tusharv / local-llm

Tool to Setup and Run LLM Locally

Local LLM Studio

Local LLM Studio is a private chat, model list, vision check, and speed benchmark for a model server running on your own machine. It speaks the OpenAI-compatible /v1 API, so it works with Ollama, LM Studio, llama.cpp, and any other local server that implements those routes. Ollama is a preset that can also pull a model, delete a model, and read native vision flags.

Studio binds to 127.0.0.1:8000. Chats, images, and benchmark rows stay in ./data. It does not install or start Ollama, LM Studio, or llama.cpp. The first run prints the commands for your operating system. You copy them into a terminal yourself.

The same Studio install is used on macOS, Windows, and Linux.

Install Studio

You need Python 3.11 or newer and Node.js 20 or newer.

python -m pip install -e ".[dev]"
cd web
npm install
npm run build
cd
python -m

…

View on GitHub

The privacy claim is a URL check in studio/url_policy.py. Loopback is saved immediately. A private LAN address needs an explicit confirm, because the prompt then leaves the machine. A public host is refused and the database is left unchanged.

How I Built It

The open-source AI in this project is the model server, not an agent framework. Studio ships no weights. It assumes you already run an open-weight model through one of three local runtimes:

Server	URL you save	What Studio may do
Ollama	http://127.0.0.1:11434/v1	Chat, list, pull, delete, read vision capability
LM Studio	http://127.0.0.1:1234/v1	Chat and list. Load the model in LM Studio.
llama.cpp	http://127.0.0.1:8080/v1	Chat and list. Start llama-server yourself.

Suggested Ollama pulls in the UI include Llama 3.2, Qwen 2.5 Coder, and Gemma 3 4B. Those are starting points for an 8 GB or 16 GB machine. Any other tag in the Ollama library works the same way.

flowchart LR
  browser["Browser at 127.0.0.1:8000"] --> api["FastAPI"]
  api --> db["SQLite and uploads in ./data"]
  api --> v1["OpenAI-compatible /v1"]
  v1 --> ollama["Ollama :11434"]
  v1 --> lm["LM Studio :1234"]
  v1 --> llama["llama-server :8080"]

The API is Python 3.11, FastAPI, and uvicorn. Chat, images, and benchmark rows go into SQLite under ./data. The UI is React and Vite, built once and served by the same process. python -m studio --dev runs the API and Vite together while the interface is changing.

Ollama gets a second, native client on the same port. Its OpenAI routes hide pull progress, delete, and the vision capability flag, so those calls use Ollama's own HTTP API. For every other server, vision comes from a known family name, a one-shot vision probe, or an override you set.

Benchmarks are a fixed suite, version 2, with three profiles: a general mix, code generation, and one image-generation prompt. Generated code is shown as text and never executed. Image URLs are never fetched. Repeats are 1 or 3. Reports show median response time and effective throughput: output tokens divided by the full request duration, including the wait for the first token.

Two write-ups shaped those choices. Theo Slater built a desktop chat app that talks only to Ollama, and wrote that an OpenAI-compatible API should have been there from the first day: I Built a Desktop Chat App for Running Local LLMs Offline. Studio starts from that API, so LM Studio and llama.cpp are a URL change. Sam Hartley is blunt that local models still lose on long reasoning, stale knowledge, and finicky vision: I Replaced All My Cloud AI With Local Models — Here's What Actually Broke. The benchmark page exists so that failure shows up on your own hardware.

Why Does Open Innovation Matter?

A closed chat API can answer the prompt. It cannot be the thing this project is for.

The weights are open, so Llama, Qwen, and Gemma can be swapped with no new vendor SDK and no per-token bill. The servers are open, so the same client reaches three runtimes. Inference stays on the machine, so a transcript about a half-finished idea stays in ./data.

A hosted API would also make the speed number meaningless. Tokens per second is a memory bandwidth number, as @lowspecai put it: each token reads the weights, so the ceiling is how fast your RAM or VRAM can be read. A leaderboard from someone else's GPU does not say whether llama3.2:3b is pleasant on this laptop. The same prompt, run locally with the warmup thrown away, does.

Open weights are also why the lazy version can be picky. If a 3B model rambles, or a vision model calls a screenshot a flowchart, delete it and pull another one.

The Lazy Payoff

The model does not do the job. It removes the errand around the model.

Before Studio, trying one local model meant remembering the port, remembering whether this runtime wants ollama run or a loaded model in a Developer tab, pasting the prompt, then doing it again for the next model because nothing kept the transcript. Checking vision meant attaching an image and reading a confused paragraph. Checking speed meant watching the terminal and guessing.

With Studio, that errand is one saved URL. The transcript is still there after refresh. The attach button is absent when the model cannot see. One benchmark row replaces the guess. The page labels that number as speed for this server and tokenizer. Whether the answer is any good is still yours to read.

I do not have a stopwatch figure, and I am not going to invent one. If you timed a real run this week, put that row here. If you did not, delete this section before publishing.

Thanks

Thanks for the challenge, and for a theme that fits a tool whose whole job is to sit on loopback and wait.