Last week I published how I made ~500 horticulture papers queryable without hallucination (passion project on DEV). That vertical worked. This post is about what came next: extracting the repeatable parts into an open platform — and publishing a spec other teams can conform to.
The problem nobody demos on Twitter
Enterprise teams don't need another ChatGPT wrapper. They need assistants that:
- Answer only from internal documents (policies, handbooks, runbooks).
- Cite sources — filename + chunk — in every response.
- Refuse when retrieval cannot support an answer.
- Run on infrastructure they control (Docker, K8s, private cloud).
- Ship with measurable quality — not "it looked fine in a notebook once."
I learned this building for scientific PDFs. The interesting work was never the chat bubble. It was making archives answerable without lying — and proving retrieval quality before burning LLM tokens.
That discipline became Grounded LLM.
What I'm trying to standardize (and what I'm not)
I'm not building Dify, LangGraph, or a visual agent constructor.
I'm standardizing one narrow class of systems:
Document-grounded assistants — internal Q&A with citations, numeric verification, and regression-tested retrieval.
The positioning line:
Open standard for document-grounded assistants with citations, numeric verify, and measurable retrieval quality — deployable on your infrastructure.
Non-goals (by design):
- Arbitrary tool/agent graphs
- General chat without a knowledge base
- Cloud-only lock-in
- Feature parity with Glean or Microsoft Copilot SaaS
We compete on trust + reproducible quality + conformance — not feature count.
Five pillars of the standard
| Pillar | What it means | Today in the repo |
|---|---|---|
| 1. Spec & conformance | Published rules + tests anyone can run |
Grounded Spec v1, python -m conformance
|
| 2. Quality science | Numbers, not demos | 89 retrieval cases, retrieval gate in CI, adversarial pack |
| 3. Reference deploy | Reproducible install | Docker, Helm, Terraform (AWS/GCP/Azure) |
| 4. Template marketplace | Growth without forking core | HR, IT Support, Legal FAQ packs + registry |
| 5. Governance | Standard outlives one author | RFC process, RFC-0001 Grounded-compatible |
Horizon 1 success metric: any engineer runs conformance on a fresh deploy in under 15 minutes.
What the platform is today (not a slide deck)
Grounded LLM v0.1.0 is the first tagged release — reference implementation of Grounded Spec v1 with hybrid retrieval (BM25 + dense + RRF), pgvector/Chroma/Qdrant backends, 89 retrieval eval cases in CI, and published GHCR images (ghcr.io/kantik001/grounded-llm-*:0.1.0). Landing page: https://kantik001.github.io/grounded-llm/
After 11 delivery phases merged to main, this is a working reference implementation, not a manifesto.
Architecture
Clients (Web / API / Telegram / embed widget)
↓
Go server — auth, sessions, LLM, verify, admin, quotas, OIDC/RBAC
↓ POST /rag/context
Python RAG — hybrid BM25 + dense + RRF; Chroma / Qdrant / pgvector
↓
data/{tenant}/{domain}/ + Postgres (sessions, audit; pgvector optional)
Split on purpose: Go owns trust boundaries and orchestration; Python owns retrieval only.
What ships out of the box
| Capability | Why it matters |
|---|---|
| Citations in every answer | Audit trail for HR/legal |
| Numeric verify layer | Dosages, vacation days, SLA numbers must match retrieved context |
| Retrieval eval gate in CI | Catches silent RAG regressions on every PR |
| Multi-tenant API |
X-Tenant-ID, API keys, OpenAPI v1 |
| Enterprise admin | RBAC, OIDC SSO, audit log, async reindex, analytics |
| Template packs | python scripts/init_pack.py install hr |
| Ingest connectors | SharePoint, Google Drive, Confluence |
| Conformance CLI | Offline spec check + live HTTP check against any deployment |
| Embeddable widget | Intranet embed, not only Telegram |
Quick start
git clone https://github.com/kantik001/grounded-llm.git
cd grounded-llm
pip install -r conformance/requirements.txt
cp .env.example .env
docker compose up -d --build
python -m conformance spec # offline OpenAPI contract
python -m conformance check --url http://localhost:8080
If your product is Grounded-compatible, these tests should pass without forking my codebase.
From one vertical to a platform (the story arc)
| Horticulture proof | Grounded LLM platform | |
|---|---|---|
| Repo | grounded_horticulture_en | grounded-llm |
| Domain | Apple rootstocks, disease IDs | Any internal documents |
| Retrieval | Tuned on ~500 papers, eval 68/68 | Hybrid BM25+RRF + 89 eval cases, CI gate |
| Deliverable | Vertical proof + passion story | v0.1.0 spec + conformance + packs + deploy |
The horticulture project answered: "Can we make scientific PDFs queryable safely?"
Grounded LLM answers: "Can we ship the next assistant in days without rebuilding auth, verify, eval, and deploy?"
Media: what to show
Screenshot 1 — Chat with citations
Screenshot 2 — Conformance CLI
📸 Terminal recording:
python -m conformance spec
python -m conformance check --url http://localhost:8080
Screenshot 3 — CI retrieval gate
Screenshot 4 — Template packs
Screenshot 5 — Admin panel
Where this sits in the industry (including Google)
Big tech is solving adjacent problems:
| Product / area | Focus | Grounded LLM difference |
|---|---|---|
| NotebookLM | Research / consumer grounding on uploads | We target enterprise on-prem, API contract, CI gates |
| Vertex AI Search | Managed cloud retrieval | We target self-hosted, MIT core, conformance badge |
| Gemini + Workspace | SaaS copilot inside Google | We target any LLM endpoint, any infra |
I'm not competing with Google on consumer UX. I'm saying: when procurement asks "is your internal assistant grounded and testable?" — there should be a published spec and CLI answer, not a vendor slide.
If you work on enterprise RAG, OSS standards, or ML platform conformance — I'd genuinely value feedback on RFC-0001.
Call to action
-
Try v0.1.0: Release notes ·
docker compose upor pull GHCR:0.1.0 - Star / watch the repo if enterprise grounding interests you: github.com/kantik001/grounded-llm
- Run conformance on your deploy and open an issue if something should be in Spec v2
- Contribute an eval case when you fix a retrieval bug — see GOOD_FIRST_ISSUES.md
- Building in horticulture or another vertical? The passion repo is still the deep retrieval story; this platform is the generalization
Summary
| Question | Answer |
|---|---|
| What is it? | Open platform + Spec v1 for cited, verified document assistants |
| What is it not? | Agent builder, ChatGPT clone, Glean competitor |
| Why now? | Vertical proof worked; standard + conformance is the multiplier |
| What's next? | External conformance adopters, Spec v2 feedback, RFP-grade positioning |
I started with my father's papers. I want to end with a checkable standard any team can implement — and prove — on their own infrastructure.
Top comments (2)
I appreciate the emphasis on building a standard for document-grounded assistants that prioritize trust, reproducible quality, and conformance. The five pillars of the standard, particularly the focus on quality science and measurable retrieval quality, resonate with me. The use of a hybrid retrieval approach, combining BM25, dense, and RRF, is also noteworthy. Have you considered exploring other retrieval algorithms, such as BERT-based methods, to further improve the assistant's accuracy and robustness?
Thanks, Luis.
On BERT-family methods: I already use that lineage in two places, with more modern checkpoints than classic BERT:
Dense retrieval — Transformer embeddings (intfloat/multilingual-e5-small), same bi-encoder idea as Sentence-BERT-style search. Cross-encoder rerank — optional via RAG_RERANKER=cross_encoder (cross-encoder/ms-marco-MiniLM-L-6-v2): score query–document pairs after a wider retrieve. Hybrid mode covers the keyword/BM25 side.
What we hold back for now (ColBERT, per-customer BERT fine-tunes) is mostly ops cost vs the “15-minute conformance / template pack in days” goal.