I've been using local LLMs for a while now, but the tools still feel rough. Ollama makes running models easy. However, when you want to do more than just a basic prompt, like comparing two models, refining a system prompt, or checking what was actually sent, you end up piecing things together.
So, I started building Openbench.
What it is
This is a desktop app for experimenting with local models. It’s not just another chat wrapper; it’s more of a workspace. The goal is to focus on prompt iteration, model comparison, and being able to see what’s happening under the hood.
Here are the main features I’m working on:
- Side-by-side model comparison
- A prompt playground with system prompt editing, temperature controls, and raw request/response inspection
- Temporary "incognito" sessions that don’t get saved
- Markdown, code highlighting, and LaTeX rendering in outputs
- Everything stored locally in SQLite, ensuring that nothing leaves your machine unless you choose to share it
Stack
The frontend uses React 19, TypeScript, and Vite, with Zustand for state management and Tailwind plus MUI for user interface. The backend is built with Rust through Tauri 2, SQLx, and ollama-rs.
Where it's at
It’s still early. The core architecture is starting to take shape, including model switching, chat persistence, and basic Ollama integration. A lot of the UI structure is set up, but there’s still much to create.
Why I’m posting this
Mainly for accountability. I often tend to overbuild, and writing this down forces me to focus on what really matters. If you’re working with local LLMs and have thoughts on what the tools are missing, I would genuinely like to hear them.
Top comments (0)