DEV Community: Tony Henein

How to Build a Robust RAG System: Lessons from a Live Production Architecture

Tony Henein — Tue, 06 Oct 2026 17:07:44 +0000

Most RAG demos are animations; we built a live system to surface the hard truths of production AI. This article shares the lessons learned, focusing on how retrieval quality depends on structural chunking and explicit refusal gates.

This figure illustrates how a production RAG pipeline orchestrates components like Document Store, Retrieval, and LLM to build grounded answers.

Most RAG demos are animations. We wanted one you could actually break.

Our How RAG Works walkthrough explains retrieval-augmented generation step by step: chunk, embed, retrieve, generate. It is a good explainer. It is also a simulation, and a simulation can't show you the parts of RAG that only appear when real questions hit real content.

So we built the real thing. Below the walkthrough is a live panel: ask a data-architecture question, and it answers from the 50+ articles and glossary entries we've published, shows exactly what it retrieved and how closely each passage matched, and cites a source for every claim. When our content doesn't cover the question, it says so.

This article walks through how it's built, layer by layer, and the lessons that changed the RAG architecture design along the way. If you're planning a retrieval-augmented generation implementation over your own documents, the lessons are the useful part. (New to RAG? Start with RAG Explained: Giving Language Models a Library Card.)

The RAG architecture at a glance

Like every RAG system, it runs in two phases that happen at completely different times.

Indexing runs once, offline. We split our published articles and glossary into chunks, turn each chunk into an embedding (a vector that captures its meaning), and store the vectors in a vector index. We re-run it after publishing new content.

Querying runs on every question. The question passes a bot check, is embedded with the same model, matched against the index, filtered through RAG guardrails, and — only if it survives — answered by a language model that may use nothing but the retrieved passages.

Component

What we used

Why

Embeddings

BGE base (open source, 768 dimensions)

Strong retrieval quality, runs on managed GPUs

Vector index

Cloudflare Vectorize (~960 chunks)

Same platform as the site; no extra service to run

Answer model

Llama 3.3 70B (open-weight)

Good at the "is this answerable?" judgment, supports structured JSON output

Runtime

A Cloudflare Pages Function

Deploys with the website; no separate backend

Bot protection

Turnstile + a rate-limit rule

Stops scripts before any AI cost

Everything runs on open-source or open-weight models. That was deliberate: it keeps the system portable, and it's a fair test of whether you need a proprietary API to do RAG well. For a vector search strategy of this shape, you don't.

Indexing: chunking is where quality is decided

It's tempting to treat chunking as plumbing. It isn't. Retrieval can only ever return the chunks you created, so their boundaries decide what an answer can be built from.

Three decisions mattered most:

Split by section, not just by size. We split each article at its headings first, then into roughly 800-character chunks with about 150 characters of overlap, breaking at a paragraph, then a sentence, then a space. A chunk never crosses a heading, so every chunk knows which section it came from — which makes its citation meaningful.
Leave the noise out. We excluded reference lists and series-navigation sections. Indexed, they let a question match a list of links instead of the argument it was looking for.
Embed with context, store without it. Each chunk is embedded with its article title and section heading prefixed, so a passage that never names its subject ("it also cuts cost…") still retrieves for questions about that subject. The stored text stays clean for display.

Two operational details saved us later. Every chunk gets a deterministic ID derived from its URL and position, so re-indexing overwrites in place instead of duplicating. After each run, we delete any vector the current content no longer produces—a shortened or retired article— with a guard that refuses to run cleanup on an empty corpus.

Querying: the same model, or nothing works

The question must be embedded with the same model — and the same settings — used for the documents. Otherwise, query and documents live in different mathematical spaces, and similarity scores are quietly meaningless. No errors. Answers get worse.

We nearly hit a subtle version of this. The embedding API offers two "pooling" modes, and its default isn't the one the BGE models were trained with. Choosing one mode at indexing time and inheriting the default at query time would have produced exactly that silent failure. The fix was to pin the setting in one shared constant that both the indexer and the query path import.

Retrieval then fetches the 20 closest chunks and keeps only the single best chunk per article. Early tests returned five passages from the same post, which looks broken to a reader and starves the answer of breadth. The top four distinct articles become the numbered sources the model can use.

The hardest part: deciding what it may not say

Here is the finding that reshaped the enterprise AI architecture design. Retrieval always returns something. There is no natural "no results" — just less relevant results. So a system that answers whenever it retrieves will answer everything, confidently.

The obvious fix is a minimum similarity score. We measured it on the real index, and it doesn't work on its own:

Question

Best match score

Should it answer?

Should we use dbt or SQLMesh?

0.879

Yes

What does Acacia charge for a migration?

0.668

No — we've never published pricing

How do we keep an LLM from making things up about our data?

0.641

Yes

Our warehouse spend keeps climbing every quarter

0.597

Yes

How do I fix a leaking kitchen faucet?

~0.43

The pricing question outscores two questions our content genuinely answers. Any threshold strict enough to refuse it would also refuse real questions asked casually.

Then testing surfaced a worse failure. Asked whether we had SAP experience, the model found an article about SAP and answered "yes." It wasn't inventing from nothing — it was over-reading a real source. That's the failure mode that pulls RAG systems out of production, because it looks exactly like a grounded answer.

So refusal became three layers of RAG guardrails, each enforced in code rather than left to a prompt:

An "ask us" gate, before any AI call. Questions about us — pricing, staffing, timelines, our own experience, clients or certifications — go to a person. Content can only ever answer those wrongly, and "talk to us" is the outcome we want anyway.
A relevance floor, before generation. Clearly unrelated questions (the faucet scored 0.43) are refused without calling the model at all, so they cost nothing.
A structured answerability check, after. The model returns JSON with its answer, its citations, and an explicit "answerable" flag. Code then verifies that every citation points to a source that was actually retrieved; an answer that cites nothing real is refused. The refusal wording is ours, not the model's.

One prompt-design detail mattered more than expected. With the "answerable" field first in the output schema, the model decided "no" before reading closely, and refused a cost question while looking at a source titled "Cost Risk: No Guardrails = 2–3× Your Expected Spend." Moving the answer field first — draft, then judge — fixed it.

If you take one idea from this article: grounding is necessary, but not sufficient. A trustworthy RAG architecture needs explicit rules about what it may claim. This is the same principle behind AI governance that actually works, applied at the level of a single answer.

Show the Retrieval, not just the answer.

The live panel shows the retrieved passages and their similarity scores before the answer, with weak matches dimmed. Anyone can wire up a chatbot; showing what was retrieved, and why, is what lets a reader judge whether the answer deserves trust. For a technical evaluator, it's the most interesting part of the system.

It also makes failures legible. When the answer is thin, you can usually see why in the Retrieval: the right article scored fourth, or the question needed a page that isn't indexed yet. That's the signal for the next improvements, such as hybrid search to catch exact terms that embeddings blur, and reranking to reorder candidates with a more precise model.

Cost, abuse, and keeping it running

A public endpoint backed by a 70-billion-parameter model is a free GPU for whoever finds it, so the controls came before launch:

Bot check per question. Every question carries a fresh, single-use token that is verified server-side before any AI call. Scripts are stopped at no cost.
Rate limit. A network-level rule caps each visitor at a handful of requests every few seconds.
Hard caps. Question length, request size, sources per answer, and answer length are all bounded.
Exact cost tracking. The inference API reports its own billing unit with every answer, and we log each request with it, so our monitoring shows real spend against the daily free allowance rather than an estimate. A typical answer costs about a tenth of a cent; refusals that skip the model cost nothing.
A kill switch. One setting turns the live panel off; the page falls back to the walkthrough, and nothing else changes.

The request log keeps the questions people ask (so we can see what's useful and tune refusals) and a one-way, salted code instead of any IP address, and deletes everything after 90 days.

A checklist for your own retrieval-augmented generation implementation

Chunk by structure first (headings, sections), then by size — and exclude navigation and reference noise.
Pin the embedding model and its settings in one place that indexing and querying both use.
Deduplicate retrieval by source document before generation.
Don't rely on a similarity threshold alone. Measure in- and out-of-scope questions on your real index first.
Route questions about your own organization — pricing, commitments, experience — to people, before any model sees them.
Ask for structured output with an explicit answerability flag, and validate citations in code.
Show the retrieved sources to users. It builds trust and makes failures diagnosable.
Add bot protection, rate limits, and cost tracking before the endpoint goes public.

RAG reduces hallucination; it doesn't eliminate it. What turns a demo into a production-grade RAG architecture is the layer around the model: what it may retrieve, what it may claim, and what it must refuse.

References

Medallion Architecture Explained: Bronze, Silver & Gold Layers for the Enterprise

Tony Henein — Mon, 07 Sep 2026 06:45:09 +0000

Medallion architecture provides a powerful framework for organizing enterprise data into Bronze, Silver, and Gold layers. Learn how this layered model enhances data quality, governance, reliability, and cost-efficiency, and discover its implementation on platforms like Databricks and Snowflake.

Medallion Architecture Explained: Bronze, Silver & Gold Layers for the Enterprise

Medallion architecture has become the default blueprint for organizing data at enterprise scale. It structures every dataset into three progressive layers — bronze silver gold — so that raw, untrusted inputs are refined step by step into governed, business-ready assets. For leaders investing in a modern data platform, understanding this pattern is the difference between a warehouse that accumulates cost and one that compounds value.

This guide explains how a medallion data architecture works in an enterprise setting, why the layered model reduces risk, and what it takes to implement it well. For a less technical primer, see our companion piece, Medallion Architecture Explained: A Guide for Business Leaders.

What Medallion Architecture Is

A medallion architecture is a data design pattern that progressively improves the quality and structure of data as it moves through three named layers. Each layer has a clear contract: what enters it, what transformations are applied to it, and who is allowed to consume it. The result is a single, auditable path from source system to executive dashboard.

The layers are tiered by trust and refinement — Bronze at the foundation, Silver in the middle, and Gold at the top:

The Three Layers

Bronze — Raw Ingestion

The Bronze layer captures source data exactly as it arrives, with no business logic applied. It is append-only and immutable, preserving a complete history of every record within the medallion data architecture.

Purpose: a durable, replayable record of source truth.
Enterprise value: full audit lineage for compliance, and the ability to reprocess history whenever business rules change.

Silver — Cleansed and Conformed

The Silver layer validates, deduplicates, and conforms data into consistent, well-modeled entities. This is where data quality rules, schema enforcement, and master-data alignment live.

Purpose: a trustworthy, integrated view across systems.
Enterprise value: eliminates the "my numbers don't match yours" problem and creates one operational source of truth.

Silver is also the natural home for disciplined modeling approaches. If your organization needs full historical traceability and auditability, this layer of the bronze silver gold model pairs well with Data Vault 2.0 modeling.

Gold — Business-Ready Products

The Gold layer delivers curated, use-case-specific datasets: KPIs, dashboards, machine learning features, and executive reporting. These are the data products the business actually consumes from the modern data platform.

Purpose: fast, governed access to decision-ready metrics.
Enterprise value: shorter time to insight and a stable interface that shields consumers from upstream change.

Why the Layered Model Matters at Enterprise Scale

The discipline of separating raw, refined, and curated data is what makes a platform governable. Each layer becomes a control point where you can independently apply quality checks, access policies, and reliability targets across the medallion data architecture.

Risk isolation: a bad source feed contaminates Bronze, not your executive dashboards.
Cost control: heavy transformation happens once in Silver, not repeatedly in every report.
Reliability: service-level objectives can be set per layer. See our reliability playbook on SLOs and data freshness.

Implementing Medallion Architecture

The pattern is platform-agnostic, but the implementation details differ by engine. Most enterprises leverage the bronze silver gold framework on a lakehouse or cloud warehouse:

Databricks: Delta Lake tables map cleanly to Bronze, Silver, and Gold, with ACID guarantees across layers.
Snowflake: schemas or databases per layer, with streams and tasks driving incremental refinement. See why businesses choose Snowflake.

Regardless of engine, success for any modern data platform depends on three practices: explicit data contracts between layers, automated quality testing in Silver, and clear ownership of Gold data products.

Common Pitfalls

Skipping Bronze: transforming on ingestion destroys your ability to reprocess history.
Logic crept into Gold: business rules belong in Silver; Gold should assemble, not cleanse.
No contracts: without defined interfaces between layers, the architecture degrades into an undocumented pipeline.

The Bottom Line

A well-built medallion architecture turns a sprawling collection of pipelines into a governed, auditable, and cost-efficient platform. The bronze silver gold model is not bureaucracy — it is the structure that lets an enterprise scale data with confidence. Explore our case studies to see the pattern applied in production.

Unstructured Data in a Medallion Architecture: Storing and Managing Images, Diagrams, and Designs

Tony Henein — Thu, 09 Jul 2026 16:53:04 +0000

Discover how to extend your data lakehouse to govern and activate unstructured assets like images, CAD models, and diagrams. This guide shows you how to integrate metadata-driven Bronze, Silver, and Gold layers for robust unstructured data management.

This diagram illustrates the adaptation of the Medallion Architecture for unstructured data, showing how files remain intact while undergoing refinement through AI-driven metadata extraction and vectorization across Bronze, Silver, and Gold layers.

Unstructured Data in a Medallion Architecture: How to Store, Govern, and Activate Images, Diagrams, and Design Files

Most unstructured data medallion architecture guidance assumes tidy rows and columns. But a growing share of enterprise value is locked inside unstructured data — product photos, engineering diagrams, building designs, CAD and BIM models, scanned contracts, and architectural renderings. These assets rarely fit a table, yet they carry exactly the context that AI and analytics teams now need.

This guide explains how to extend the Bronze, Silver, and Gold layers to handle unstructured data management without abandoning the governance and lineage that make the medallion pattern work. Within a data lakehouse, the goal is to treat a 200 MB building design the same way you treat a customer record — versioned, described by metadata, quality-checked, and ready to serve.

Why Unstructured Data Breaks the Classic Medallion Model

The unstructured data medallion architecture was designed to progressively refine data from raw to curated. With structured data, each layer transforms columns. With unstructured data, the file itself usually stays intact — a photo is still a photo in Gold — so the refinement happens in the metadata and derived signals that surround it, not in the bytes of the file.

That shift changes three things:

Storage of record moves to object storage. When you storing images in data lake environments rely on, the binary lives in a bucket or volume; the table only holds a pointer, a checksum, and descriptive metadata.
Refinement means enrichment, not rewriting. Effective unstructured data management in Silver and Gold adds extracted text, embeddings, classifications, and quality flags rather than reshaping the original asset.
Governance follows the metadata. Access control, lineage, and retention are enforced on the catalog record that describes each file, which in turn governs the file behind it.

The Three Layers, Adapted for Files

Bronze — Raw Landing for Binaries

Bronze is the immutable landing zone. For unstructured data, that means the original file lands in object storage exactly as received, and a Bronze table captures one row per file with the bare facts:

A stable object path or URI to the binary (never the binary inside the row).
A content hash (for deduplication and integrity), file size, and MIME type.
Source system, ingestion timestamp, and the original filename.

Nothing is interpreted yet. A blurry site photo, a superseded floor plan, and a final rendering all land side by side. Bronze's job is to guarantee you never lose the source of truth and can always replay downstream processing.

Silver — Cleaned, Described, and Enriched

Silver is where unstructured data management turns raw files into usable assets through metadata enrichment. The file stays put; you layer signals on top of it:

Content extraction: OCR text from scanned drawings, EXIF and geolocation from photos, layer and object lists from CAD/BIM, page text from PDFs.
Enrichment: AI-generated captions, tags, and object detection for images; entity extraction from contracts; classification of a diagram as "electrical," "structural," or "HVAC."
Vector embeddings: numeric representations of each image or document that power semantic search and retrieval-augmented generation.
Quality and conformance: deduplication by content hash, resolution and readability checks, and standardized metadata schemas so a "building design" from any source describes itself the same way.

The result is a Silver table that is fully queryable — you can filter, join, and search across files even though the assets themselves are unstructured.

Gold — Curated Products for Analytics and AI

Gold packages Silver into trusted, purpose-built products. For organizations that storing images in data lake repositories contain, this typically looks like:

A curated asset catalog — every current building design with its project, discipline, revision, and approval status, ready for a dashboard or portal.
A vector index of approved documents and diagrams that powers "find similar designs" or an AI assistant that answers questions grounded in your own drawings.
Aggregated signals — counts of assets by project phase, defect photos by site, or coverage gaps in as-built documentation.

Because Gold is governed and versioned in the unstructured data medallion architecture, an AI model or business user consumes only approved, current assets — not the raw noise sitting in Bronze.

A Worked Example: Building Design and Architecture Files

Consider an engineering and construction organization managing thousands of drawings, renderings, and BIM models across active projects in a data lakehouse.

Bronze: Every uploaded file — a revised floor plan, a drone photo of the site, a Revit model export — lands in object storage with a hash and a Bronze row recording where it came from and when.
Silver: OCR pulls title-block text and revision numbers from drawings; AI captions and classifies photos ("facade, north elevation"); BIM metadata is extracted; embeddings are generated so any drawing can be found semantically. Superseded revisions are flagged, and duplicates collapse by hash.
Gold: A "current approved drawings" product exposes only the latest revision per sheet per project, feeds a search portal, and grounds an AI assistant that answers "show me the latest structural plans for Tower B" with the right file — with full lineage back to the source.

Governance, Lineage, and Cost

The same disciplines that protect structured data apply here, enforced through the catalog:

Access control on the metadata table gates who can retrieve the underlying file; sensitive designs stay restricted by role.
Lineage ties every Gold asset back to its Bronze original and the enrichment steps in between, so you can audit how a file was classified or captioned.
Retention and tiering move cold binaries to cheaper storage while keeping their metadata hot and searchable — you don't pay premium rates to store a five-year-old rendering you rarely open.
Open table formats (Delta, Iceberg) give ACID transactions and time travel over the metadata layer, so the catalog of your unstructured data estate is as trustworthy as any other table in your data lakehouse.

Implementation Principles

Store binaries in object storage, pointers in tables. Never embed large files in rows when you storing images in data lake style.
Refine metadata, preserve the original. The Bronze binary is immutable; metadata enrichment lives alongside it.
Standardize a metadata contract early so every asset type describes itself consistently.
Add embeddings in Silver to make unstructured content searchable and AI-ready by default.
Govern the catalog, and the files follow. Access, lineage, and retention all key off the metadata record in your unstructured data medallion architecture.

Frequently Asked Questions

Where do the actual image or design files get stored in a medallion architecture?

The binary files live in object storage (a data lake bucket or lakehouse volume), not inside database rows. Each medallion table stores a pointer to the file, a content hash, and descriptive metadata. This keeps tables fast and queryable while the large binaries stay in cost-efficient storage.

How does the Silver layer refine unstructured data if the file doesn't change?

For unstructured data management, refinement happens in the metadata and derived signals rather than the file itself. Silver adds extracted text (OCR), AI captions and classifications, vector embeddings, and quality flags — turning an opaque file into something you can filter, join, and search.

How do you make images and diagrams searchable?

Generate vector embeddings for each asset in the Silver layer and store them in a vector index in Gold. This enables semantic search ("find similar facade designs") and retrieval-augmented AI assistants grounded in your own diagrams and drawings.

Can the same governance apply to files and structured data?

Yes. Access control, lineage, and retention are enforced on the catalog record that describes each file. Because that record governs the file behind it, unstructured data assets inherit the same governance model as your structured tables in a data lakehouse.