DEV Community

Pingredsai
Pingredsai

Posted on

LocalRAG Desktop: a Docker-free local document Q&A app for Windows

#ai #opensource #llm #rag

Most local RAG demos still start with the same obstacle:

cp .env.example .env
docker compose up -d
docker exec localrag-ollama ollama pull ...
Enter fullscreen mode Exit fullscreen mode

That is fine for developers. It is not fine for a lawyer, accountant, or small-business owner who just wants to search their own documents.

So I rebuilt the user-facing setup around a different question:

Can a Windows user run a private document Q&A system without installing Docker, WSL, or Python?

This beta is my answer.

What it does

LocalRAG Desktop is a local document knowledge base:

  • upload text-based PDF, Word, TXT, or Markdown files
  • parse and chunk documents locally
  • create embeddings with a local Chinese embedding model
  • retrieve relevant passages from Qdrant
  • generate an answer with a local Qwen model
  • show the source file and chunk used for the answer

Everything stays on the local machine. The services bind to 127.0.0.1 only.

What changed in this beta

The earlier version was a Docker Compose stack. The Windows beta now uses:

  • native Windows Qdrant
  • native llama.cpp CPU runtime
  • Qwen2.5 1.5B for generation
  • bge-small-zh-v1.5 for embeddings
  • a packaged FastAPI backend
  • a native Windows launcher

The user flow is now:

Download ZIP
→ verify SHA256
→ extract to a data drive
→ double-click the launcher
→ wait for the local services
→ use the local web UI
Enter fullscreen mode Exit fullscreen mode

The launcher performs environment checks, starts the services, waits for health checks, and opens the browser.

Why I did not just wrap Docker

A wrapper can hide Docker commands, but it cannot remove the Docker installation, WSL2 requirement, virtualization settings, admin permissions, and large image downloads.

For a technical user, Docker is convenient. For a non-technical user, it is a second product they have to install before they can try the first one.

The goal here was to remove that second product entirely. A real one-click deployment is not a script that hides Docker commands. It is a setup where the user never needs Docker in the first place.

Architecture 

Windows launcher
├── Qdrant native
├── llama.cpp generation server
├── llama.cpp embedding server
├── packaged FastAPI backend
├── local web UI
└── local data directory
Enter fullscreen mode Exit fullscreen mode

Security-related choices:

  • services bind to 127.0.0.1
  • the local API uses a random session token for non-health endpoints
  • CORS is restricted to local origins
  • diagnostics exclude documents, models, vector data, and access tokens
  • no default telemetry

What I verified

On the development machine:

  • native Qdrant and llama.cpp started successfully
  • a sample document was indexed into 3 chunks
  • a policy question returned the correct answer with a source
  • streaming answers worked
  • the native stack continued working after Docker containers were stopped
  • the packaged backend ran without system Python
  • no-token API requests returned 401

This is still a beta. It has not yet been tested on a wide matrix of clean Windows machines.

Download and try it

Release page:

https://github.com/Pingredsai/local-rag/releases/tag/v0.3.0-beta.1

Direct ZIP:

https://github.com/Pingredsai/local-rag/releases/download/v0.3.0-beta.1/LocalRAG-Beta-v0.3.zip

SHA256:

f35d424fedd98e9f8b143e4a72c6dfd2ee350c03c3338582449c2c479a287035
Enter fullscreen mode Exit fullscreen mode

Current limitations

  • Windows 10 22H2 / Windows 11 x64 only
  • 8GB RAM recommended for the included 1.5B model
  • text-based PDFs only; OCR is not included
  • CPU inference in this beta
  • single knowledge base
  • no multi-user permissions yet
  • the beta is not code signed yet, so Windows may show an “Unknown publisher” or SmartScreen warning

The ZIP contains a Windows prompt guide and a diagnostic tool. Please do not disable Defender, SmartScreen, or Application Control to run it.

Feedback I am looking for

I am mainly interested in:

  • installation/startup failures
  • unclear Windows prompts
  • first-answer latency
  • document types that fail to parse
  • whether source references are clear enough
  • whether the setup feels simple enough for a non-technical user

If you test it, please include your Windows version, RAM, the step where you got stuck, and the diagnostic ZIP generated by the launcher.

Contact: siquhjh@163.com

I will keep the technical limitations and test results honest as the project evolves.

Top comments (0)