I'll be honest — I put this guide together because I kept needing it myself. Enough meetings where someone throws out "Lambda vs. Kappa" or "we should really move to a Lakehouse" and I'm nodding along while quietly rebuilding the mental model from scratch. Enough times explaining to an exec why a data mesh isn't just "microservices for data" and watching the analogy land wrong. So I wrote down the guide I wished I'd had, and I've been coming back to it ever since. That's the Data Architecture Field Guide, and if any of that sounds familiar, it's worth a spot in your bookmarks.
Who it's for
This one's for anyone who needs to operate in the data world, not just build in it — engineers and architects who want the "why" behind a pattern, sure, but just as much for technical leads, product managers, and anyone who has to sit in the same room as the developer explaining why Kafka is the right call and the executive asking what it costs and when it ships. The real skill isn't memorizing tool names — it's knowing the jargon well enough to translate it, in either direction, without losing the room.
It's not hands-on — and that's the point
I want to be upfront about what this is not. It's not a "spin up Kafka and follow along" tutorial (there's a small set of AWS labs tucked in at the end for that itch, but they're not the main event). This is a foundation — the concepts, the tradeoffs, the vocabulary — so that when someone says "eventual consistency" or "medallion architecture" in a design review, you're not just nodding, you actually know what's being traded away. That kind of fluency is what lets you hold your own in any forum, whether it's a whiteboard session with engineers or a budget conversation with leadership.
What it covers
The subtitle says it best: one argument, in eight parts. Each part builds on the last, walking from fundamentals to the kind of judgment calls you actually get paid for:
- Foundations — ACID vs. BASE, CAP theorem, OLTP vs. OLAP, data modeling basics, the ETL-to-ELT shift, batch vs. streaming, and why legacy warehouse migrations happen at all.
- Architecture Patterns & Decision Framework — Lambda, Kappa, Lakehouse, Medallion, Data Mesh, and Data Fabric, laid side by side with a framework for actually choosing between them.
- Design & Operations — storage and table formats, dimensional modeling, ingestion, streaming, transformations with dbt, DataOps, metadata, quality, governance, and security.
- Cost & Performance Economics — the tradeoffs that don't show up in architecture diagrams but absolutely show up in the cloud bill.
- Delivery — serving layers (BI, APIs, reverse ETL), reliability (SLAs, disaster recovery), and what it actually means to run data as a product.
- Data Mesh Operating Models — organizational patterns for scaling ownership across teams.
- AI & Defense — feature stores, vector databases, RAG architecture, architecture decision records (ADRs), common anti-patterns, and a board-level capstone scenario to pressure-test what you've learned.
- A Few Hands-on Tutorials — a small set of AWS labs (Glue, Athena, Iceberg, Hudi, dbt, Kinesis, and more) if you want to ground a concept in something real — not the point of the guide, just a bonus.
How it's organized (so you'll actually come back)
I built this to be reread, not read once and closed:
- A left sidebar with the full topic map, so you can jump straight to the one thing you need at 4:45pm before a 5pm meeting.
- Previous/Next links for the days you do want to sit down and go deep.
- A master decision map up front — a flowchart to help you find your question before you go digging.
Why I keep coming back to it
Most architecture content online is either a vendor pitch dressed up as advice, or a comparison post that reads great once and teaches you nothing the second time. I didn't want that. I wanted something I could brush up against before a review, skim before a stakeholder call, or just wander through when I forget the difference between data fabric and data mesh for the hundredth time. If that's the kind of resource you're after too, bookmark it — you'll be back.
Find it in the tutorials sidebar on this site, or go straight to it here.
If you go through it and something's missing, unclear, or just plain wrong — tell me. This guide gets better every time someone pushes back on it, and that's good for both of us: you get the answer you actually needed, and the next person who bookmarks it gets a sharper guide. You can raise a pull request or reach out.
Top comments (0)