If you work with data, you will be asked this sooner or later: what is a lakehouse, and why not just use a warehouse? Here is the answer I use, with a short animated diagram at the end.
The data warehouse
A data warehouse (Snowflake, Redshift, BigQuery) stores structured tables in its own optimised format. It is very good at three things:
- ACID transactions
- enforced schemas and governance
- fast BI queries
The trade-off is cost and reach. Storage is proprietary and expensive at scale, and it was never designed for raw files, images or logs, so machine learning teams struggle to work in it.
The data lake
A data lake is cheap object storage, such as S3 or ADLS, holding open file formats. It takes any data type, costs very little, and Python and ML tools can read it directly.
But a plain lake has no transactions. Two writers can corrupt a table, nothing enforces a schema, and without a catalog it slowly becomes a data swamp nobody trusts.
Why most companies ran both
So teams loaded everything into the lake, then copied the modelled part into the warehouse. That means two copies of the data, two pipelines to maintain, and two sets of numbers that drift apart.
The lakehouse
A lakehouse is a data lake with an open table format, such as Delta Lake or Apache Iceberg, on top. That table format gives you warehouse behaviour directly on cheap object storage.
From the warehouse it takes:
- ACID transactions on the table files
- schema enforcement and a catalog for governance
- fast BI, because the engine keeps statistics on those same files and skips what a query does not need
From the lake it keeps:
- the same cheap, open storage
- every data type, structured or not
- direct access for Python and ML, with no export step
The one-line answer
One copy of the data, in open formats, serving BI and machine learning at the same time.
Watch it drawn step by step
This is part of a free series that builds Databricks up one layer at a time. Watch it in order from episode 1: https://www.youtube.com/watch?v=B5iHmoYgnqY&list=PLDB5WDkDOYF4
Top comments (0)