A question we keep getting from business owners: "my staff are already pasting customer emails and contracts into ChatGPT, can we stop that without banning AI?" For a lot of offices the honest answer is a local model on one decent machine, so nothing leaves the building.
It's like having a typist in the back room instead of posting your documents to a typing service across town. They're a bit slower, but the papers never leave the office.
What you need
- One machine with a recent GPU (or an Apple Silicon Mac with enough memory). A small model also runs on CPU for light use, just slowly.
- Ollama to run the models
- Open WebUI for the chat screen your team uses in a browser
- Docker for Open WebUI
Setup
- Install Ollama from its site and pull a model sized to your hardware. Start small and go up only if the answers aren't good enough.
- Run Open WebUI in Docker and point it at Ollama.
- Create the admin account, then turn off open sign-ups so only people you invite can use it.
- Give the team the machine's local address. It works over the office network, and nothing goes out to the internet unless you add tools that do.
Where it works well
Summarising long emails, rewriting replies politely, pulling details out of documents and drafting first versions. Most office writing doesn't need the biggest model in the world.
Where it doesn't
Hard reasoning, very long documents on small hardware, and anything that needs fresh knowledge from the internet. For those, use a paid AI service under a business plan whose data terms you've actually read.
Rules we give clients
- One shared machine, set up properly, beats ten laptops each running something different.
- Back up the Open WebUI data folder, because chats and settings live there.
- Write down which model is approved for which kind of document.
I run Arwahyn AI, an AI automation agency in Dhaka. Bangla setup guides for Ollama and Open WebUI are on our blog.
Top comments (0)