langchain-rust: Build LLM apps with Ollama + local models in pure Rust — no Python needed

lili — Mon, 03 Aug 2026 09:24:01 +0000

If you're running local models through Ollama and tired of Python's overhead, check out langchain-rust.

It's a full LLM framework in pure Rust that works great with local models:

Ollama support — first-class integration with tool calling, vision, and streaming
9 vector store backends — InMemory, SQLite, Qdrant, ChromaDB, Redis, PGVector, MongoDB, Pinecone, FileVectorStore
BM25 keyword search — with Chinese/English tokenization, no external dependency
Hybrid retrieval — BM25 + Vector with RRF fusion for better recall
GraphRAG — Knowledge graph construction + community detection, all local
CorrectiveRAG — Self-correcting retrieval with hallucination detection
Code Interpreter — LocalSandbox (subprocess), E2B cloud, or WASM sandbox
LocalEmbeddings — Run embeddings without calling an API

Plus: LangGraph workflows, MCP client/server, 7 memory types, guardrails, and 12+ built-in tools.

Single binary, no virtualenv, no pip conflicts. Just cargo add langchainrust and go.

DEV Community: lili