The medallion architecture organizes a data platform into three layers, Raw, Clean, and Ready, that take source data and refine it step by step into trusted, business-ready tables. It is the most common blueprint for the modern data lakehouse, and it is the foundation OptimaFlo is built on. This guide explains each layer in plain English, then shows how to stand one up without hiring a data team.
What is the medallion architecture?
The medallion architecture is a data design pattern popularized by Databricks. It splits data processing into three progressively cleaner layers. You may know these layers by their traditional names, bronze, silver, and gold; we use clearer terms: Raw, Clean, and Ready. Each stage adds structure, quality, and business meaning:
- Raw holds data exactly as it arrived.
- Clean holds cleaned, validated, joined data.
- Ready holds business-ready metrics for dashboards and reports.
The point is simple. You never transform data in one big, fragile step. You move it through three small, reliable ones, and every stage stays queryable and auditable.
The three layers
Raw: untouched source data
Raw is your system of record. Data lands here exactly as the source sent it, with zero transforms and full history. Nothing is thrown away. If a downstream rule turns out to be wrong, you can always reprocess from Raw.
Because Raw keeps everything, the table format matters. OptimaFlo stores every layer in an open lakehouse table format, which gives Raw ACID transactions, schema evolution, and time travel. You can query the table as it looked last Tuesday, not just today.
Clean: cleaned and conformed
Clean is where source data becomes reliable. This layer validates types, removes duplicates, fixes nulls, and joins sources into clean, conformed tables. It is the layer most analysts actually build on.
Clean is also where SQL lives. In a traditional stack this is the dbt layer, written and maintained by data engineers. The transforms are version-controlled and reviewed, because a bad Clean query quietly poisons everything downstream.
Ready: business-ready
Ready is the layer your business sees. It holds aggregated metrics, star schemas, and the tables that power dashboards and reports. Ready tables are usually updated incrementally, so a daily metric refreshes in seconds instead of rebuilding from scratch.
Here is how the three layers compare:
| Layer | Holds | Transforms | Typical consumer |
|---|---|---|---|
| Raw | Source data, full history | None | Reprocessing, audits |
| Clean | Cleaned, validated, joined | Type, dedupe, join | Analysts, data scientists |
| Ready | Metrics, star schema | Aggregate, model | Dashboards, executives |
Why data teams adopt it
The pattern is popular because it solves real problems at once:
- Reliability. Failures are isolated to one hop, not the whole pipeline.
- Reproducibility. Keeping Raw plus lakehouse time travel means any result can be rebuilt exactly.
- Incremental refinement. Each layer does one job, so changes are small and safe.
- Auditability. You can trace any Ready number back through Clean to the original source record.
The catch: it usually takes a whole stack and a team
The architecture is elegant. Building it is not. A typical stack stitches together six to ten tools: ingestion (Fivetran), a warehouse (Snowflake), transforms (dbt), orchestration (Airflow), a BI tool (Tableau), and data observability (Monte Carlo). Each one is a contract, a bill, and a thing that breaks at 2 a.m.
It also takes people. Someone has to write the Clean and Ready SQL, wire up the orchestration, and own the schemas. For a data-rich team without a dedicated data engineering function, that is the wall most projects hit.
How OptimaFlo builds the medallion architecture for you
OptimaFlo is one platform that runs the full Raw, Clean, Ready flow inside your own cloud. It is built to remove the stack and help overwhelmed teams at the same time.
- LLM-native. Describe the table you want in plain language. OptimaFlo generates the Clean and Ready SQL, you review and approve it, and it ships. The AI writes the data engineering, you keep the judgment.
- Open lakehouse foundation. Every layer is an open table, so ACID guarantees, schema evolution, and time travel apply end to end, not just in one tool. No proprietary lock-in.
- Bring your own cloud (BYOC). The platform runs in your environment. Your source data is stored in your account, never ours.
- The right compute, automatically. OptimaFlo matches the compute to the size of your data: a light local engine for small jobs, your own warehouse as they grow. You never size or manage clusters.
- One platform, not ten. Ingestion, transforms, orchestration, dashboards, and data quality checks live in one place.
The result: you can go from your first data source connection to a live, Ready-backed dashboard in a single afternoon. See the getting started guide for the five-minute version, or the architecture overview for the full picture.
What a Clean transform looks like
A Clean model is just SQL: take the Raw table, cast the types, drop the bad rows, and write a clean table. OptimaFlo generates SQL like this, and you approve it before it runs:
SELECT
order_id,
customer_id,
CAST(order_total AS DECIMAL(12, 2)) AS order_total,
DATE_TRUNC('day', ordered_at) AS order_day
FROM raw.orders
WHERE order_total IS NOT NULL;
That clean.orders table then feeds a Ready metric, like daily revenue by customer segment, which lands in your dashboard.
Key takeaways
- The medallion architecture refines data through three layers: Raw source data, Clean validated data, and Ready business-ready tables.
- Its real cost is the multi-tool stack and the data team needed to run it.
- OptimaFlo collapses that into one LLM-native platform on an open lakehouse, running in your own cloud.
Ready to build your first Raw-to-Ready pipeline? Start building with OptimaFlo or read the getting started guide.
Top comments (0)