DEV Community

niuniu
niuniu

Posted on

Quick Tip: Run a Local Vision Model with Ollama in 3 Commands (Free, No API Key)

Quick tip that saved me a $20/month bill this week.

I needed to batch-caption ~800 product photos. The hosted vision APIs quote you per image, so I tried the local route with Ollama instead. Total setup time: under 3 minutes.

# 1. Pull a vision model (one time, ~5GB)
ollama pull llama3.2-vision

# 2. Serve it locally (OpenAI-compatible API on :11434)
ollama serve

# 3. Call it from Python — no API key, no internet
Enter fullscreen mode Exit fullscreen mode 
import ollama

resp = ollama.chat(
    model="llama3.2-vision",
    messages=[{
        "role": "user",
        "content": "Describe this product photo in one sentence for an alt tag.",
        "images": ["photo_042.jpg"],
    }],
)
print(resp["message"]["content"])
Enter fullscreen mode Exit fullscreen mode

The math that made me switch:

Option 800 images Privacy
Hosted vision API ~$8-16 one-off, metered forever Photos leave your machine
Ollama local $0.00, unlimited reruns Never leaves localhost

On my RTX 3060 (12GB) it does ~2.3s/image. The whole batch finished in ~31 minutes while I made coffee.

Downsides, honestly: you need a GPU with 8GB+ VRAM (or accept slow CPU inference), and accuracy on fine-grained text-in-image is still behind the top hosted models. For alt-text, tagging, and rough classification? More than good enough.

The part that surprised me: the ollama Python lib is OpenAI-API-compatible, so the same code swaps between local and hosted with a one-line base-URL change. I sketched the whole script in MonkeyCode, which also runs fully local — same "your code stays on your machine" philosophy: https://ly.cyberserval.tech/iIETXiF

Have you moved any production workloads off metered APIs to local models? What was the breaking point — cost, privacy, or latency?

Top comments (0)