Databricks Lakebase: Native OLTP Layer for the Lakehouse

databricks lakebase is the pick-one architectural announcement of 2025 that finally collapses the decades-old operational-vs-analytical database split into a single Databricks-managed plane — a Postgres-compatible, serverless OLTP layer that sits inside the lakehouse, shares governance with Delta through Unity Catalog, and eliminates the reverse-ETL glue jobs that senior data engineers have been maintaining for a decade. Every operational read your application makes — a user profile fetch, a recommendation-feature lookup, an order-status query — has historically lived in a separate transactional store (Postgres, MySQL, DynamoDB) and been kept in sync with the warehouse via CDC pipelines out and reverse-ETL jobs back; Lakebase's bet is that one platform can serve both patterns, cheaply, with the same Unity Catalog RBAC and masking policies covering both worlds.

This guide is the senior-DE walkthrough you wished existed the first time an interviewer asked "walk me through Databricks Lakebase and how it differs from Snowflake Unistore," or "your team runs a reverse-ETL pipeline from Delta to RDS Postgres for the recommendation service — what changes with Lakebase?", or "how does Unity Catalog govern a Postgres wire-protocol database?" It walks through the five things every senior engineer must know: why Lakebase changes the lakehouse story in 2026, the serverless postgres databricks architecture built on the Neon acquisition with copy-on-write branching and separated compute + storage, the bidirectional sync that makes postgres lakehouse a real thing (near-zero-lag CDC into Delta, MERGE-back into Lakebase), the three operational lanes where Lakebase wins (reverse-ETL replacement, feature serving, operational dashboards) plus the one anti-pattern (very-high-QPS bid paths), and the migration playbooks from RDS Postgres, Neon, and Snowflake Unistore that senior interviewers probe for. Each section pairs a teaching block with a Solution-Tail interview answer — code, a step-by-step trace, an output table, then a concept-by-concept breakdown of why it works.

When you want hands-on reps immediately after reading, drill the SQL practice library →, rehearse on the design practice library →, and sharpen the streaming axis with the streaming practice library →.

---

On this page

• Why Lakebase changes the lakehouse story in 2026
• Lakebase architecture — separation of compute + storage
• OLTP + OLAP in one platform
• Operational patterns + when to use Lakebase
• Migration + interview signals
• Cheat sheet — Lakebase recipes
• Frequently asked questions
• Practice on PipeCode

---

1. Why Lakebase changes the lakehouse story in 2026

One platform for OLTP and OLAP — the historical divide collapses, reverse-ETL becomes a legacy pattern

The one-sentence invariant: Databricks Lakebase is a Postgres-compatible, serverless OLTP layer that lives inside the Databricks lakehouse, shares Unity Catalog governance with Delta tables, and syncs bidirectionally with Delta so that operational reads (single-row lookups, joins on a primary key) and analytical reads (aggregate scans over hundreds of millions of rows) both happen against the same governed data without CDC glue jobs, reverse-ETL pipelines, or dual-write dances between systems. The historical split between OLTP (row-store, single-row latency-critical, Postgres/MySQL) and OLAP (column-store, aggregate-scan-optimised, Snowflake/BigQuery/Delta) has forced every modern data team to maintain two databases plus the plumbing between them; Lakebase's bet — echoed by Snowflake Unistore and to a lesser extent by MotherDuck — is that one platform can serve both, and that governance parity (one Unity Catalog policy applied everywhere) is worth more than the last 5 percent of tail latency the specialised systems still win on.

Where it shows up.

• Reverse-ETL replacement. Every team with a Delta warehouse and an operational app has a reverse-ETL job today: compute recommendation features in Delta, ship them back to Postgres so the app can read them with sub-50ms latency. Lakebase collapses this into a single MERGE from Delta into Lakebase.
• Real-time feature serving. ML teams compute features from event streams, land them in Delta, and need to serve them from a low-latency store for model inference. Lakebase's Postgres wire protocol is the serve side; no Redis or DynamoDB required.
• Operational dashboards over fresh warehouse data. Ops teams want Grafana / Metabase over "the state of the warehouse right now" without ETL lag. Lakebase syncs from Delta with near-zero lag; the dashboard queries Postgres.
• Application state for AI agents. Databricks positioned Lakebase specifically as the state store for agents built on Mosaic AI — the agent's short-term memory needs OLTP semantics but must share Unity Catalog policy with the training data.

What interviewers listen for.

• Do you name Data + AI Summit 2025 as the launch context and the Neon acquisition as the foundation without prompting? — senior signal.
• Do you say "Postgres wire compatible" in the first sentence when Lakebase comes up? — required answer.
• Do you push back on "just use RDS Postgres" with the Unity Catalog governance argument — "the whole point is one policy plane across OLTP and OLAP"? — senior signal.
• Do you describe Lakebase as "OLTP for the lakehouse" rather than as "a Postgres replacement"? — required answer.
• Do you name the anti-pattern — very-high-QPS bid-path workloads where a specialised OLTP is still cheaper — unprompted? — senior signal.

The 2026 reality — the OLTP-in-lakehouse category is real, and there are three serious entrants.

• Databricks Lakebase — the subject of this guide. Postgres-compatible, serverless, built on the Neon acquisition (mid-2025), Unity-Catalog governed, sits inside the Databricks workspace next to Delta.
• Snowflake Unistore + Hybrid Tables — Snowflake's answer, generally available for a year longer than Lakebase, uses Snowflake's own row-store engine (not Postgres wire), integrates with Snowflake's Horizon governance. Different wire protocol; interview probes often compare feature parity.
• MotherDuck + DuckDB Cloud — a smaller-scale entrant focused on developer ergonomics; not a Lakebase competitor at enterprise scale but often mentioned in the same breath.
• Neon (standalone) — the Postgres-branching database Databricks acquired to build Lakebase. Still runs as a standalone service (AWS/Azure/GCP) for teams that want Neon without the Databricks workspace.
• PlanetScale, Aurora Postgres Serverless v2, Aiven — the incumbent serverless-Postgres players that Lakebase displaces on the "Postgres for apps that also touch the warehouse" workload.

The four axes senior interviewers probe.

• OLTP semantics. Does the store give you real Postgres — transactions, indexes, row-level locks, MVCC — or a warehouse pretending to be OLTP? Lakebase is real Postgres (the Neon fork), so the answer is "yes; row-level UPDATE + SELECT with proper index behaviour."
• OLAP integration. How does the OLTP layer talk to the analytical layer? Lakebase's answer is bidirectional sync with Delta under one Unity Catalog namespace; Unistore's is Hybrid Tables that live in Snowflake's column store; standalone Postgres's is Debezium + reverse-ETL glue.
• Governance parity. Do you apply the same masking / RBAC / lineage policy across OLTP and OLAP without duplicating it in two systems? Lakebase claims yes via Unity Catalog; standalone Postgres cannot.
• Cost model. Is the OLTP layer priced per-query-per-second (specialised OLTP) or as a slice of the lakehouse compute budget? Lakebase is serverless-consumption-priced; the branch cost is O(1) rather than O(data), which changes dev-environment economics.

Worked example — the three-pattern comparison table

Detailed explanation. The single most useful artifact for a Lakebase interview is a memorised 3-column comparison table across Lakebase, Snowflake Unistore, and standalone Postgres + reverse-ETL. Every senior discussion converges on this table within the first ten minutes; having it in your head is what separates a fluent answer from a stumbling one. Walk through building the table for a hypothetical e-commerce team with a Delta warehouse for analytics and a customer-facing recommendation service that needs sub-50ms feature reads.

• Analytical layer. 500 TB of order history and clickstream data in Delta tables under Unity Catalog.
• Operational layer. ~5000 queries/sec of SELECT features FROM user_features WHERE user_id = ? from the recommendation service.
• Sync. Recommendation features are computed nightly (or streamed hourly) from Delta and need to reach the operational store within minutes of computation.
• Governance. PII masking must apply identically to analyst queries against Delta and to service reads against the operational store.

Question. Build the three-way comparison for the recommendation-feature workload and pick the pattern each requirement should map to.

Input.

Dimension	Lakebase	Snowflake Unistore	Postgres + reverse-ETL
Wire protocol	Postgres	Snowflake SQL	Postgres
OLTP engine	Neon fork	Snowflake row-store	native Postgres
Analytical sync	bidirectional Delta	Hybrid Tables	Debezium + Fivetran
Governance plane	Unity Catalog	Horizon	separate per system
Branching	O(1) copy-on-write	not supported	pg_dump snapshot

Code.

-- Lakebase — the same DDL a Postgres engineer already writes
CREATE TABLE app.user_features (
    user_id       BIGINT       PRIMARY KEY,
    features      JSONB        NOT NULL,
    model_version TEXT         NOT NULL,
    updated_at    TIMESTAMPTZ  NOT NULL DEFAULT clock_timestamp()
);

CREATE INDEX idx_user_features_updated_at
    ON app.user_features (updated_at);

-- Bidirectional sync — declared once as a Unity Catalog-managed sync
-- (illustrative Databricks CLI; syntax evolves with the product)
-- databricks lakebase sync create \
--   --source delta://analytics.recommendations.user_features_v2 \
--   --target lakebase://app.user_features \
--   --mode merge --primary-key user_id --lag-target-ms 60000

Step-by-step explanation.

1. The Lakebase DDL is deliberately identical to Postgres DDL — that's the point of wire compatibility. Any Postgres engineer already knows how to model user_features, and every psycopg2 / pgx / JDBC driver works out of the box.
2. The updated_at index on Lakebase is still the right choice for a serving table — it lets you cheaply query "what features have changed in the last N minutes" without a full table scan, which is exactly what the sync-monitoring dashboard needs.
3. The sync declaration is Databricks-specific — a single Unity Catalog artifact that says "keep this Delta table and this Lakebase table in sync in this direction on this key with this lag target." The sync engine is what replaces the Airflow + Debezium + Fivetran stack a standalone-Postgres team would build.
4. --lag-target-ms 60000 (60 seconds) is the SLO for the sync. Lakebase's engine keeps the target Postgres table within one minute of the Delta source in steady state; on backlog, the sync throttles the Delta side and never violates OLTP query latency.
5. In practice, most senior architectures pick Lakebase for the reverse-ETL and feature-serving lanes, keep specialised OLTP (Aurora, DynamoDB) for very-high-QPS bid paths, and use Unity Catalog as the single lineage / policy plane across both.

Output.

Requirement	Recommended pattern	Why
Sub-50ms feature reads	Lakebase	real Postgres + index; ~10-30ms p50
5000 QPS burst	Lakebase	serverless autoscale absorbs bursts
Nightly Delta → OLTP refresh	Lakebase sync (Delta → Lakebase)	no Airflow / Fivetran glue
Unity Catalog PII masking	Lakebase	policy applies to both worlds
Dev/staging isolation	Lakebase branches	O(1) branch cost
100k QPS bid path	still Aurora / DynamoDB	Lakebase not tuned for this

Rule of thumb. Never pick an OLTP-in-lakehouse product based on "which vendor is trendy this quarter." Pick it based on (wire protocol × governance × analytical sync × cost model × QPS envelope) — the five axes. Write the table on a whiteboard first; the choice falls out of the constraints.

Worked example — the five-minute senior interview monologue

Detailed explanation. The senior data-engineering Lakebase interview has a predictable structure: the interviewer opens with an ambiguous framing ("what did Databricks announce at Data + AI Summit 2025 that matters for data engineers?"), then progressively narrows to test whether you understand the architecture and the trade-offs. The candidates who name the pattern in sentence one score highest; the candidates who describe "some new Postgres thing" score lowest. Walk through the interview grading rubric.

• Ambiguous opener. "What's Lakebase and why should our team care?" — invites you to name the pattern.
• Follow-up 1. "How is it different from RDS Postgres?" — probes serverless + branching + Unity Catalog.
• Follow-up 2. "How does it sync with Delta?" — probes bidirectional sync semantics.
• Follow-up 3. "What breaks with Lakebase that RDS wouldn't?" — probes cold-start, connection pooling, QPS limits.
• Follow-up 4. "When would you not use Lakebase?" — probes anti-patterns.

Question. Draft a 5-minute senior Lakebase monologue that covers all five axes without waiting to be asked.

Input.

Interview signal	Weak answer	Senior answer
Naming	"the new Databricks Postgres"	"Postgres-compatible OLTP inside the lakehouse, built on the Neon acquisition"
Architecture	"it's serverless"	"serverless compute + shared object-store storage + copy-on-write branching"
Sync	"it syncs with Delta somehow"	"bidirectional; CDC into Delta, MERGE back from Delta, one Unity Catalog policy plane"
Anti-pattern	"not sure"	"very-high-QPS bid paths; specialised OLTP still cheaper for 100k QPS+"
Migration	"restore a pg_dump"	"pg_dump + Delta CDC for RDS; native path from Neon; schema translate from Unistore"

Code.

Senior Lakebase answer template (5 minutes)
==========================================

Minute 1 — name the pattern up front
  "Lakebase is Databricks' Postgres-compatible OLTP layer,
   announced at Data + AI Summit 2025 and built on the Neon
   acquisition. It sits inside the lakehouse and shares Unity
   Catalog governance with Delta."

Minute 2 — architecture
  "Serverless compute pods talking to a shared object-store page
   store — same architecture that made Neon novel. Copy-on-write
   branching means dev/staging branches are O(1) — you get a
   full-fidelity dev copy without paying for the storage twice."

Minute 3 — bidirectional sync
  "Native bidirectional sync with Delta under Unity Catalog:
   near-zero-lag CDC from Lakebase into Delta for analytics,
   MERGE / UPSERT from Delta back into Lakebase to serve computed
   features and aggregates. This is the reverse-ETL kill move."

Minute 4 — anti-pattern
  "Not for very-high-QPS bid-path OLTP. Ad-tech real-time bidding,
   sub-millisecond trading, 100k+ QPS write workloads — the
   serverless Postgres model has cold-start and connection-pooling
   tails that a dedicated Aurora / DynamoDB deployment doesn't."

Minute 5 — migration + governance
  "Migration paths: from RDS Postgres via pg_dump + a Delta CDC
   backfill for history; from Neon via a native path (same engine
   family); from Snowflake Unistore via schema translation. On
   governance: one Unity Catalog policy — say, PII masking on
   customers.email — applies identically to Delta scans and
   Lakebase reads. That's the point."

Step-by-step explanation.

1. Minute 1 is the crucial framing. Naming the pattern immediately — "Postgres-compatible OLTP inside the lakehouse, built on the Neon acquisition, announced at Data + AI Summit 2025" — signals you're a decision-maker who tracks the industry, not someone who just read the product page.
2. Minute 2 addresses architecture. The serverless-compute + shared-storage + copy-on-write branching triple is what distinguishes Lakebase from RDS or Aurora — RDS is provisioned compute plus local disk; Lakebase is elastic compute plus shared object store. Branching is the operational win: dev/staging environments cost pennies.
3. Minute 3 is the sync story. Every senior architect wants to know how OLTP data reaches OLAP and vice versa. Bidirectional sync under Unity Catalog is Lakebase's differentiator; naming the direction (CDC out, MERGE in) shows you understand the plumbing.
4. Minute 4 is the anti-pattern. Every candidate who describes Lakebase as "the future for all OLTP" loses credibility. Senior answers name the workloads where Lakebase is not the right tool — very-high-QPS bid paths, sub-millisecond trading — because knowing where a technology doesn't fit is a bigger senior signal than knowing where it does.
5. Minute 5 covers migration and governance. Every enterprise deployment starts with a migration; every enterprise cares about policy. Naming three migration paths (RDS, Neon, Unistore) and one policy story (Unity Catalog masking spans both worlds) demonstrates deployment-ready thinking.

Output.

Grading criterion	Weak score	Senior score
Names pattern in minute 1	rare	mandatory
Names Neon acquisition	rare	required
Names bidirectional sync	occasional	mandatory
Names anti-pattern	rare	senior signal
Names governance parity	rare	senior signal

Rule of thumb. The senior Lakebase answer is a 5-minute monologue that covers all five axes without waiting for the follow-ups. Rehearse it once; deploy it every time you're asked "what's new at Databricks this year?"

Worked example — the "pick the OLTP layer" decision tree

Detailed explanation. Given a new operational workload, the senior architect runs a 5-question decision tree in their head. Codifying the tree makes the interview answer reproducible: any interviewer can hand you a scenario and you can walk it out loud. Walk through the tree with three canonical scenarios: a Delta-warehouse team adding a customer-facing app, a Snowflake shop evaluating Unistore vs Lakebase, and a bid-path ad-tech team.

• Q1. Do you already run the Databricks lakehouse with Unity Catalog? → yes = strong Lakebase candidate; no = evaluate on Postgres-compat vs governance trade-off.
• Q2. Is the workload aggregate scans (OLAP) or single-row reads/writes (OLTP)? → OLTP = Lakebase / Unistore / RDS; OLAP = Delta / Snowflake.
• Q3. Do you need Postgres wire compatibility (existing psycopg2 apps, JDBC connectors, Postgres extensions)? → yes = Lakebase over Unistore.
• Q4. Is the peak write QPS > 50k with sub-ms tail-latency requirements? → yes = specialised OLTP (Aurora / DynamoDB) still wins; no = Lakebase fits.
• Q5 (parallel branch). Do you need dev / staging branches of production data for testing? → yes = Lakebase's O(1) copy-on-write branching is the differentiator.

Question. Walk the decision tree for the three scenarios and record the pattern each ends up with.

Input.

Scenario	Q1 (Databricks?)	Q2 (OLTP?)	Q3 (Postgres wire?)	Q4 (100k+ QPS?)	Q5 (branching?)
Delta shop + rec-service	yes	yes	yes	no	yes
Snowflake shop	no	yes	maybe	no	maybe
Ad-tech bidder	maybe	yes	no	yes	no

Code.

# Decision-tree helper (illustrative)
def pick_oltp_layer(uses_databricks: bool,
                    is_oltp: bool,
                    needs_postgres_wire: bool,
                    peak_qps: int,
                    needs_branching: bool) -> list[str]:
    """Return the primary OLTP layer(s) for a new workload."""
    if not is_oltp:
        return ["Delta / Snowflake — analytical workload"]

    candidates: list[str] = []

    if peak_qps >= 50_000:
        candidates.append("Aurora Postgres / DynamoDB (specialised OLTP)")
        return candidates

    if uses_databricks:
        candidates.append("Lakebase (Postgres-compat + Unity Catalog + branching)")
    elif needs_postgres_wire:
        candidates.append("Neon (standalone) or RDS Postgres")
    else:
        candidates.append("Snowflake Unistore Hybrid Tables")

    if needs_branching and "Lakebase" not in candidates[0]:
        candidates.append("consider Lakebase for branching workflow")

    return candidates


# Walk the three scenarios
print(pick_oltp_layer(True,  True, True,  5_000,   True))
# -> ['Lakebase (Postgres-compat + Unity Catalog + branching)']

print(pick_oltp_layer(False, True, True,  2_000,   False))
# -> ['Neon (standalone) or RDS Postgres']

print(pick_oltp_layer(True,  True, False, 200_000, False))
# -> ['Aurora Postgres / DynamoDB (specialised OLTP)']

Step-by-step explanation.

1. Scenario 1 — Delta warehouse with Unity Catalog, adding a customer-facing recommendation service at ~5000 QPS, wants dev/staging branches. Q1 = yes, Q2 = yes, Q3 = yes, Q4 = no, Q5 = yes → Lakebase is the clean answer. The governance parity + branching alone justify the choice.
2. Scenario 2 — Snowflake shop with no Databricks footprint. The decision tree still asks OLTP-vs-OLAP first, but Q1 rules out Lakebase without a workspace migration. The realistic answer is Unistore or standalone Neon, depending on the Postgres-wire requirement.
3. Scenario 3 — an ad-tech real-time bidder at 200k QPS with sub-millisecond tail-latency SLO. Q4 short-circuits the tree: specialised OLTP (Aurora Postgres, DynamoDB, ScyllaDB) still wins. Lakebase's serverless cold-start tail is a non-starter for the bid path.
4. The parallel Q5 branch (branching) is orthogonal to the primary pick. Even in a Snowflake or standalone-Postgres shop, "we need cheap dev copies of production" is a signal to look at Neon or Lakebase specifically.
5. If none of Q1-Q4 pass cleanly, default to the incumbent (RDS Postgres) and revisit when Databricks or Snowflake enters the roadmap. Do not force a lakehouse-integrated OLTP onto a team that doesn't have a lakehouse.

Output.

Scenario	Primary pick	Add-on
Delta shop + rec-service	Lakebase	branching for dev/staging
Snowflake shop	Unistore or Neon	—
Ad-tech bidder	Aurora / DynamoDB	—

Rule of thumb. The five-question decision tree is a whiteboard-friendly answer. Practice walking it end-to-end so an interviewer can hand you any scenario and get a pattern name in under 60 seconds.

Senior interview question on Lakebase adoption

A senior interviewer often opens with: "Your team runs a Delta warehouse under Unity Catalog and a recommendation service that reads user features from a self-managed RDS Postgres, kept in sync by a Fivetran reverse-ETL job. Walk me through the case for migrating to Databricks Lakebase, the architecture changes, the migration mechanics, and what would make you keep RDS instead."

Solution Using Lakebase as the reverse-ETL and feature-serving target with a phased RDS retirement

-- Step 1 — provision a Lakebase Postgres instance in the Databricks workspace
-- (illustrative CLI; syntax evolves with the product)
-- databricks lakebase instance create \
--   --name prod-features \
--   --region us-east-1 \
--   --unity-catalog my_org.app_features

-- Step 2 — create the feature-serving table in Lakebase; same DDL as RDS
CREATE TABLE app_features.user_features (
    user_id       BIGINT       PRIMARY KEY,
    features      JSONB        NOT NULL,
    model_version TEXT         NOT NULL,
    computed_at   TIMESTAMPTZ  NOT NULL,
    updated_at    TIMESTAMPTZ  NOT NULL DEFAULT clock_timestamp()
);

CREATE INDEX idx_user_features_updated_at
    ON app_features.user_features (updated_at);

-- Step 3 — replace the Fivetran job with a Unity-Catalog-managed sync
-- databricks lakebase sync create \
--   --name features-refresh \
--   --source delta://analytics.recommendations.user_features_v2 \
--   --target lakebase://prod-features/app_features.user_features \
--   --mode merge --primary-key user_id --lag-target-ms 60000

# Step 4 — the recommendation service still uses psycopg2; only the DSN changes
import psycopg2

# Before (RDS)
# conn = psycopg2.connect("host=rds-prod.internal port=5432 dbname=features user=svc")

# After (Lakebase — Postgres wire, workspace-managed hostname)
conn = psycopg2.connect(
    host="prod-features.lakebase.us-east-1.databricks.com",
    port=5432,
    dbname="prod",
    user="svc",
    sslmode="require",
)

def get_features(user_id: int) -> dict | None:
    with conn.cursor() as cur:
        cur.execute("""
            SELECT features, model_version, updated_at
            FROM   app_features.user_features
            WHERE  user_id = %s
        """, (user_id,))
        row = cur.fetchone()
    return {"features": row[0], "model_version": row[1], "updated_at": row[2]} if row else None

-- Step 5 — Unity Catalog policy on PII (applies across Delta + Lakebase)
CREATE MASK app_features.mask_email
    ON app_features.customers.email
    RETURN CASE
        WHEN is_member('pii_readers') THEN email
        ELSE regexp_replace(email, '(.).*(@.*)', '\1***\2')
    END;

Step-by-step trace.

Step	Before (RDS + Fivetran)	After (Lakebase + sync)
Feature refresh mechanism	Fivetran reverse-ETL job	Unity Catalog sync (Delta → Lakebase)
Refresh lag	15-30 min	~60 s
PII masking	duplicated in RDS view + Delta view	single Unity Catalog MASK
Dev environment	shared staging + pg_dump copies	O(1) Lakebase branch
Wire protocol	Postgres	Postgres (unchanged)
Governance plane	RDS IAM + Unity Catalog (separate)	Unity Catalog only
Fivetran cost	~$3k/month	$0 (bundled)

After the migration, the recommendation service sees no code changes beyond the DSN swap; feature refresh lag drops from 15-30 minutes to ~60 seconds; PII masking is defined once in Unity Catalog and applies to both analyst SQL against Delta and service reads against Lakebase; and dev/staging environments become O(1) branches instead of nightly pg_dump copies.

Output:

Metric	Before	After
Feature freshness	15-30 min	60 s
Reverse-ETL vendor cost	~$3k/mo	$0 (bundled)
PII masking definitions	2 (Delta + RDS)	1 (Unity Catalog)
Dev env spin-up time	hours (pg_dump)	seconds (branch)
Wire-protocol change	none	none

Why this works — concept by concept:

• Postgres wire compatibility — Lakebase speaks the Postgres frontend protocol, so every existing psycopg2 / JDBC / pgx client works with only a DSN swap. Zero application-code migration cost — the load-bearing decision that makes the whole thing feasible.
• Unity Catalog sync — a declarative one-line sync replaces the Airflow / Fivetran / Debezium reverse-ETL stack. The Databricks-managed sync engine keeps lag inside an SLO target, throttles the source on backlog, and reports lineage into Unity Catalog automatically.
• Copy-on-write branches — dev and staging environments become O(1) branch operations against the shared page store. No storage duplication, no long pg_dump / pg_restore cycles, no drift between staging and prod because branches share history until write divergence.
• Unity Catalog policy parity — a single MASK definition applies to both the Delta scan an analyst runs in a notebook and the Postgres SELECT the recommendation service issues. Removes the "we defined the policy in two places and they drifted" bug class entirely.
• Cost — one Lakebase instance replaces the RDS instance + Fivetran subscription; per-QPS pricing is serverless-consumption-based; the branch cost is bounded by write-divergence bytes, not by full table size. Net O(delta) per commit versus O(N) per Fivetran refresh, at strictly lower vendor cost.

Design
Topic — design
Design problems on lakehouse and OLTP-in-lakehouse systems

Practice →

SQL Topic — sql SQL problems on Postgres-compatible OLTP

Practice →

---

2. Lakebase architecture — separation of compute + storage

Serverless Postgres compute + shared object-store page store + O(1) copy-on-write branching — the Neon fork inside the Databricks workspace

The mental model in one line: Lakebase's architecture is the Neon fork Databricks acquired in mid-2025 — a Postgres binary whose page store has been detached from local disk and rehomed onto a shared object-store layer, with a serverless compute plane that spins Postgres processes up and down on demand and a copy-on-write branching primitive that lets you create a full-fidelity branch of production in milliseconds by sharing the underlying pages until they diverge — all wrapped inside the Databricks workspace and governed by Unity Catalog like any other lakehouse asset. Every senior discussion of Lakebase eventually converges on the branching workflow, because that's what makes the dev/staging economics qualitatively different from RDS or Aurora.

The three layers of the Lakebase stack.

• Compute layer (serverless Postgres). Elastic Postgres processes that spin up on demand for each connection or workload. Autoscales with load; scales to zero on idle (with a cold-start penalty measured in low seconds). Each compute pod attaches to a page-store endpoint over the wire, not to local disk.
• Storage layer (shared page store on object storage). The Neon innovation Databricks inherited: Postgres's 8KB pages live in a service-managed object-store layer (S3 / ADLS / GCS underneath) with a fast in-memory + local-SSD cache in front of it. Compute reads pages over the network; the page-store returns the correct version for the compute's snapshot.
• Branching primitive (copy-on-write). Creating a branch is a metadata operation: the new branch shares every existing page with its parent and only stores new / modified pages. Cost of creating a branch: bytes and milliseconds. Cost of holding a branch: only the diverged pages.

The branch-per-environment workflow — the operational headline.

• main branch. Production. Pinned to a compute pod that is always warm. Every write commits here; downstream consumers (Delta CDC, applications) read from here.
• staging branch. Created from main on demand (or nightly). Runs the same schema; developers point staging services at it. Storage cost = only the writes since the branch was created.
• dev-{feature} branches. Feature branches, one per engineer or per pull request. Created in seconds, destroyed just as fast. This is the Neon workflow senior engineers migrate to Lakebase for.
• analytics branch. A read-only branch fed to long-running analytical queries or ad-hoc data-science workloads without impacting production compute.
• Reset. A branch can be reset to its parent's current head; effectively a "throw away my dev changes and re-fork from prod."

Where it shows up.

• Dev environments that mirror prod. Every enterprise has struggled to keep staging in sync with production; branches solve this by construction.
• PR-level integration tests. CI spins up a per-PR branch, runs migrations + tests, tears the branch down. Cost per PR is measured in cents, not dollars.
• Point-in-time recovery. Branches can be created from a past timestamp (PITR), giving instant "what did the DB look like yesterday at 3pm?" for incident forensics.
• Blue/green migrations. Migrate on a branch, promote branch to main. The Neon community has built this pattern; it maps directly onto Lakebase.

The four axes for Lakebase architecture.

• Latency. Steady-state OLTP reads: sub-10ms p50, ~50ms p99 (dominated by page-store cache hits). Cold-start on scale-to-zero: 1-5 seconds first connection.
• Throughput. Scales horizontally with compute-pod addition; single-pod throughput is modern-Postgres-comparable (tens of thousands of TPS on modest hardware, roughly).
• Consistency. Full Postgres semantics — MVCC, serializable isolation available, row-level locks. Not a "warehouse pretending to be OLTP" — this is real Postgres.
• Durability. Object-store durability (11 nines) for storage; writes are ack'd after WAL is durable in the shared page store. No local-disk data-loss window.

What interviewers listen for.

• Do you name "serverless compute + shared page store + copy-on-write branching" as the three layers unprompted? — senior signal.
• Do you say "the Neon fork" rather than "some new Databricks-built engine"? — required accuracy.
• Do you push back on "branches are just database copies" with the copy-on-write mechanics — "the branch shares pages until write divergence"? — senior signal.
• Do you name cold-start latency as the trade-off for scale-to-zero — 1-5 seconds first connection? — required answer.

The failure modes to name.

• Cold-start tail. First connection after scale-to-zero pays a 1-5 second penalty. For always-on services, pin the compute to prevent scale-to-zero; for scheduled workloads, accept the tail.
• Connection pooling. Serverless Postgres and long-lived pooled connections are a common footgun. Use PgBouncer / built-in Lakebase pooler; do not open a fresh connection per query from a low-latency service.
• Branch lag. Branches created from a running primary can lag by seconds until they catch up. Read-your-own-writes on a fresh branch is a defined-timing operation, not instantaneous.
• Page-store hot spots. Very hot pages (top of a monotonically-increasing index, e.g. BIGSERIAL PK) hit page-store cache but can still be a bottleneck. UUID keys or hash-partitioned indexes mitigate.

Common interview probes on Lakebase architecture.

• "How is Lakebase's storage architecture different from RDS?" — required answer: shared page store on object storage vs local EBS disk.
• "What does a branch cost?" — required answer: O(1) creation; ongoing cost proportional to divergent pages only.
• "How does the compute plane scale?" — required answer: serverless autoscale; scale-to-zero option with cold-start penalty.
• "Is it real Postgres?" — required answer: yes; Neon fork, full wire protocol, full MVCC semantics, most extensions supported.

Worked example — branch-per-environment DDL and CI wiring

Detailed explanation. The canonical Lakebase environment workflow: main is production; staging is a branch reset nightly from main; per-PR branches spin up in CI, run migrations + tests, tear down on PR close. Walk through the branch DDL and the CI wiring.

• Provisioning. One Lakebase instance per production database; branches are children.
• Staging reset. Nightly cron: destroy staging branch, re-create from main.
• PR branch. Named pr-{number}; created on PR open; destroyed on PR close.
• Compute. main has a pinned always-on pod; branches use scale-to-zero pods.

Question. Show the branch commands and the CI pipeline that spins up a per-PR Lakebase branch, runs migrations, then tears down.

Input.

Object	Purpose
main branch	production; pinned compute
staging branch	nightly reset from main
pr-{n} branch	ephemeral per-PR branch
CI hook	on-open create; on-close destroy

Code.

# Illustrative Databricks CLI for Lakebase branch management
# (syntax evolves with the product)

# 1. Create the production instance once
databricks lakebase instance create \
    --name prod-features \
    --region us-east-1 \
    --unity-catalog my_org.app_features \
    --compute-mode pinned \
    --min-compute-cu 2

# 2. Nightly staging reset (cron in Airflow / Databricks Workflows)
databricks lakebase branch delete --instance prod-features --branch staging || true
databricks lakebase branch create \
    --instance prod-features \
    --branch staging \
    --parent main \
    --compute-mode serverless

# 3. Per-PR branch (invoked from GitHub Actions on PR open)
databricks lakebase branch create \
    --instance prod-features \
    --branch "pr-${PR_NUMBER}" \
    --parent main \
    --compute-mode serverless \
    --idle-timeout-seconds 900

# 4. Get the branch DSN for tests
BRANCH_DSN=$(databricks lakebase branch get \
    --instance prod-features \
    --branch "pr-${PR_NUMBER}" \
    --output dsn)

# 5. Run migrations + tests against the branch
psql "${BRANCH_DSN}" -f migrations/latest.sql
pytest tests/integration/ --dsn "${BRANCH_DSN}"

# 6. Tear down on PR close
databricks lakebase branch delete \
    --instance prod-features \
    --branch "pr-${PR_NUMBER}"

# .github/workflows/pr-integration.yml — the CI half of the above
name: Lakebase PR branch
on:
  pull_request:
    types: [opened, synchronize, closed]

jobs:
  branch-and-test:
    if: github.event.action != 'closed'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Create Lakebase branch
        env:
          DATABRICKS_TOKEN: ${{ secrets.DATABRICKS_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
        run: ./scripts/pr-branch-up.sh
      - name: Run integration tests
        env:
          PR_NUMBER: ${{ github.event.pull_request.number }}
        run: ./scripts/pr-tests.sh

  branch-teardown:
    if: github.event.action == 'closed'
    runs-on: ubuntu-latest
    steps:
      - name: Delete Lakebase branch
        env:
          DATABRICKS_TOKEN: ${{ secrets.DATABRICKS_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
        run: ./scripts/pr-branch-down.sh

Step-by-step explanation.

1. The production instance is created once with --compute-mode pinned so the primary Postgres process is always warm. Cold-start is a bad look for a customer-facing service; pinning trades a small always-on cost for zero cold-start.
2. The nightly staging reset is the operational win over RDS/Aurora: branch delete + branch create is a metadata operation, not a full data restore. Cost is measured in the write bytes since the last reset, not in the full data volume.
3. The per-PR branch uses --compute-mode serverless --idle-timeout-seconds 900. When no query has run for 15 minutes, the compute scales to zero; when the next CI step queries the branch, it pays a short cold-start penalty. This is the right cost model for ephemeral branches.
4. The DSN is retrieved via CLI and passed to psql for migrations and pytest for integration tests. Every existing Postgres tooling works — this is the wire-compatibility payoff.
5. Teardown on PR close reclaims the divergent-page cost. In steady state, an org running 50 open PRs pays the storage cost of only the divergent pages of those 50 branches, plus one always-on production pod. The economics don't exist on RDS.

Output.

Environment	Storage cost	Compute cost	Cold-start
main (prod)	full data	always-on pod	0 s
staging	divergent pages since last reset	serverless	1-3 s first hit
pr-1234	divergent pages since branch create	serverless + auto-idle	1-5 s first hit
pr-1235	divergent pages	serverless	1-5 s first hit

Rule of thumb. For any Lakebase deployment, pin main, reset staging nightly, and let per-PR branches be ephemeral with a 15-minute idle timeout. This is the operational pattern the Neon community established; it applies directly to Lakebase.

Worked example — point-in-time recovery via branching

Detailed explanation. The Neon-derived branching primitive makes point-in-time recovery a first-class operation: create a branch from a past timestamp, mount it as a live Postgres instance, run forensic queries against it, and destroy it when done. This replaces the traditional PITR workflow of "restore an entire pg_basebackup from S3, apply WAL until the target time, wait 4 hours." Walk through the incident-forensics workflow.

• Scenario. A bad application deploy at 14:23 UTC accidentally set status = 'cancelled' on 12,000 orders. You need to identify which orders were affected and restore them.
• Old way. Restore last night's backup, replay WAL to 14:22, extract the row states, diff against current state, write a repair script. Total time: hours.
• New way. Create a branch of main at timestamp 14:22:00 UTC, run a diff query against production, generate the repair script from the diff. Total time: minutes.

Question. Create a PITR branch at 14:22 UTC, diff against production, and generate the repair script.

Input.

Component	Value
Instance	prod-orders
PITR timestamp	2026-07-19 14:22:00 UTC
Branch name	forensic-2026-07-19-1422
Diff query	affected orders between the two branches

Code.

# 1. Create a branch of main at the pre-incident timestamp
databricks lakebase branch create \
    --instance prod-orders \
    --branch forensic-2026-07-19-1422 \
    --parent main \
    --parent-timestamp "2026-07-19T14:22:00Z" \
    --compute-mode serverless \
    --read-only

# 2. Get DSNs for both branches
FORENSIC_DSN=$(databricks lakebase branch get \
    --instance prod-orders \
    --branch forensic-2026-07-19-1422 \
    --output dsn)

PROD_DSN=$(databricks lakebase branch get \
    --instance prod-orders \
    --branch main \
    --output dsn)

-- 3. Diff the two branches to find affected rows
-- (Run from a notebook / psql that can attach to both DSNs;
--  shown here as a cross-DB query using dblink for illustration)

CREATE EXTENSION IF NOT EXISTS dblink;

-- All orders whose status is 'cancelled' now
-- but was NOT 'cancelled' at 14:22 UTC
SELECT prod.id,
       prod.status AS current_status,
       hist.status AS pre_incident_status
FROM   public.orders AS prod
JOIN   dblink('forensic_conn',
              'SELECT id, status FROM public.orders')
       AS hist(id BIGINT, status TEXT)
       ON hist.id = prod.id
WHERE  prod.status = 'cancelled'
  AND  hist.status <> 'cancelled';

-- 4. Generate the repair script from the diff
-- (Save the result to affected_orders.sql, review, apply during
--  a change window)
INSERT INTO orders_repair_20260719 (id, restore_status)
SELECT prod.id, hist.status
FROM   public.orders AS prod
JOIN   dblink('forensic_conn',
              'SELECT id, status FROM public.orders')
       AS hist(id BIGINT, status TEXT)
       ON hist.id = prod.id
WHERE  prod.status = 'cancelled'
  AND  hist.status <> 'cancelled';

-- The actual repair (guarded, in a controlled transaction)
BEGIN;
UPDATE public.orders AS o
   SET status     = r.restore_status,
       updated_at = clock_timestamp()
  FROM orders_repair_20260719 AS r
 WHERE o.id = r.id
   AND o.status = 'cancelled';
COMMIT;

Step-by-step explanation.

1. The branch create --parent-timestamp operation is the Neon PITR primitive Lakebase inherits: given a timestamp inside the retained WAL window, the branch is a read-only Postgres instance whose visible state is what production looked like at that timestamp. The branch is available in seconds because it's a metadata operation over the page store.
2. Both branches expose standard Postgres DSNs, so the forensic diff runs from any tool that speaks Postgres wire — psql, a notebook, a Python script. The dblink extension lets you join across branches in a single query; alternatives include running two queries and diffing in application code.
3. The diff query surfaces the exact rows the incident touched: prod.status = 'cancelled' AND hist.status <> 'cancelled'. This is orders of magnitude faster than manually inspecting last night's backup.
4. The repair script uses a staging table (orders_repair_20260719) so a reviewer can inspect the diff before running the UPDATE. Guarded UPDATEs with AND o.status = 'cancelled' prevent re-repairing rows a user has legitimately cancelled since the incident.
5. When the incident is resolved, databricks lakebase branch delete --branch forensic-2026-07-19-1422 reclaims the branch cost. Total storage cost of the entire forensic session: bounded by the divergent pages during the session (probably kilobytes).

Output.

Step	Elapsed	Cost
Branch create at PITR timestamp	5-15 s	O(1) metadata
Diff query (12k affected rows)	1-3 s	bounded page-store reads
Repair script generation	seconds	negligible
Repair transaction on prod	seconds	one small UPDATE
Branch delete	1 s	reclaim
Total incident-to-repair time	~5 min	vs ~4h for pg_basebackup PITR

Rule of thumb. For any Lakebase incident-response runbook, teach the on-call rotation to reach for branch create --parent-timestamp before any pg_restore workflow. The point-in-time branch is the single most-underused Lakebase feature; every senior on-call should have it memorised.

Worked example — connection pooling for a serverless Postgres

Detailed explanation. Serverless Postgres and naive per-request connection opening are a well-known footgun. Each new Postgres connection costs a fork (or fork-equivalent) on the compute plane; at 1000 QPS this saturates. The fix is a connection pooler in front of Lakebase — PgBouncer, or Lakebase's built-in pooler if it exists in your workspace. Walk through the config.

• Failure mode. Application opens a fresh psycopg2 connection per HTTP request; at 500 QPS the connection-open rate saturates the Lakebase compute pod.
• Fix. Deploy PgBouncer sidecar in transaction pooling mode; app opens a persistent pool to PgBouncer; PgBouncer multiplexes to Lakebase.
• Trade-off. Transaction-mode pooling breaks Postgres session state (SET, prepared statements, cursors); app code must not rely on session state across statements.

Question. Configure PgBouncer in front of Lakebase and update the application to pool through it.

Input.

Component	Value
Pooler	PgBouncer 1.22+
Pool mode	transaction
App pool size	5 per app pod
Upstream pool size	20 per pooler

Code.

; pgbouncer.ini — deployed as a sidecar next to the app
[databases]
prod = host=prod-features.lakebase.us-east-1.databricks.com port=5432 dbname=prod

[pgbouncer]
listen_addr = 127.0.0.1
listen_port = 6432
auth_type = scram-sha-256
auth_file = /etc/pgbouncer/userlist.txt

; The critical setting for serverless Postgres
pool_mode = transaction

; Sizing
default_pool_size = 20        ; connections to Lakebase per user/db
reserve_pool_size = 5
max_client_conn = 200         ; from the app side

; Long-lived pooler ↔ Lakebase connections
server_lifetime = 3600
server_idle_timeout = 600

; Sensible timeouts for a serverless upstream
query_wait_timeout = 30

# Application code — pool through PgBouncer, not directly to Lakebase
import psycopg2
from psycopg2.pool import ThreadedConnectionPool

# Note: sslmode='disable' to the local sidecar; PgBouncer handles TLS upstream
POOL = ThreadedConnectionPool(
    minconn=2,
    maxconn=5,               # per-pod cap; scale via pod count
    host="127.0.0.1",
    port=6432,
    dbname="prod",
    user="svc",
    sslmode="disable",
)

def get_features(user_id: int) -> dict | None:
    conn = POOL.getconn()
    try:
        with conn.cursor() as cur:
            # Transaction-mode pooling: no SET, no prepared statements
            # that cross statement boundaries, no session-local state
            cur.execute("""
                SELECT features, model_version, updated_at
                FROM   app_features.user_features
                WHERE  user_id = %s
            """, (user_id,))
            row = cur.fetchone()
        conn.commit()
        return {"features": row[0], "model_version": row[1], "updated_at": row[2]} if row else None
    finally:
        POOL.putconn(conn)

Step-by-step explanation.

1. The failure mode is that every fresh Postgres connection triggers backend-process work on Lakebase's compute plane. At 500 QPS with no pooling, the compute pod spends more time forking connections than serving queries. Latency p99 climbs into hundreds of milliseconds.
2. PgBouncer in pool_mode = transaction multiplexes short-lived app connections onto a small number of long-lived upstream connections. Each app query claims an upstream connection for the duration of one transaction, then releases it — a 100x reduction in upstream connection churn.
3. default_pool_size = 20 caps upstream connections per user/db combination; this must be tuned against Lakebase's per-pod connection limit. The pooler-to-Lakebase connections stay warm (server_lifetime = 3600), which keeps cold-start out of the request path.
4. The app opens a small local pool to PgBouncer (maxconn = 5 per pod) and treats it as free. The sslmode='disable' is safe because the pooler is a sidecar on the same host; PgBouncer opens TLS to Lakebase.
5. The trade-off: transaction-mode pooling breaks SET LOCAL, PREPARE, session-scoped GUCs, and any cursor that outlives a single transaction. Application code must be reviewed for these; most modern app code (which relies on parameterised queries, not prepared statements) is fine.

Output.

Metric	Direct connections	With PgBouncer
Per-query connection open	yes	no (pooled)
p50 latency at 500 QPS	40-80 ms	8-15 ms
p99 latency at 500 QPS	200-800 ms	30-60 ms
Compute-pod connection churn	500/s	~1/s
Cold-start in request path	possible	never

Rule of thumb. Never talk to a serverless Postgres directly from a request-path service. Deploy PgBouncer (or the Lakebase-native pooler if available in your workspace) with pool_mode = transaction, keep upstream pool warm, and audit application code for session-state assumptions. Skipping this is the #1 Lakebase performance footgun.

Senior interview question on Lakebase architecture

A senior interviewer might ask: "Design the environment topology for a team of 20 engineers running 30-40 concurrent pull requests against a Lakebase-backed service. Cover the branch strategy, the compute-mode choices, the CI wiring, the cost envelope, and the on-call runbook when a branch's compute is unhealthy."

Solution Using pinned main + nightly staging reset + ephemeral per-PR branches + PgBouncer sidecar

# 1. Production instance — pinned compute, always-on
databricks lakebase instance create \
    --name prod-orders \
    --region us-east-1 \
    --unity-catalog my_org.app_orders \
    --compute-mode pinned \
    --min-compute-cu 4 \
    --max-compute-cu 32 \
    --pitr-retention-hours 168      # 7 days of PITR

# 2. Staging branch — recreated nightly from main
databricks lakebase branch delete --instance prod-orders --branch staging || true
databricks lakebase branch create \
    --instance prod-orders \
    --branch staging \
    --parent main \
    --compute-mode serverless \
    --idle-timeout-seconds 3600

# 3. Per-PR branches — created on PR open, deleted on close
# (see .github/workflows/pr-integration.yml above)

# 4. Deployment manifest — PgBouncer sidecar next to every app pod
apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-svc
spec:
  replicas: 6
  template:
    spec:
      containers:
        - name: app
          image: my-org/orders-svc:v1.42
          env:
            - name: DB_HOST
              value: "127.0.0.1"
            - name: DB_PORT
              value: "6432"
        - name: pgbouncer
          image: bitnami/pgbouncer:1.22
          env:
            - name: POSTGRESQL_HOST
              value: "prod-orders.lakebase.us-east-1.databricks.com"
            - name: POSTGRESQL_PORT
              value: "5432"
            - name: POOL_MODE
              value: "transaction"
            - name: POSTGRESQL_DATABASE
              value: "prod"

-- 5. Monitoring queries — pod-side + Lakebase-side
-- pgbouncer stats (from app node)
SHOW STATS;
SHOW POOLS;

-- Lakebase-side connection count + query latency
SELECT count(*) AS active_conns,
       max(now() - query_start) AS longest_running_query
FROM   pg_stat_activity
WHERE  state = 'active';

Step-by-step trace.

Layer	Config	Result
main compute	pinned, 4-32 CU	zero cold-start; autoscale to burst
staging	serverless, 1h idle	pennies per day when unused
PR branches	serverless, 15m idle	seconds to spin up; cents to hold
PITR	7 days	forensic branch anytime in window
PgBouncer	transaction mode, 20 upstream	100x connection-churn reduction
Monitoring	pg_stat_activity + SHOW POOLS	pod-level + Lakebase-level visibility

After the rollout, main never cold-starts because compute is pinned; staging reflects prod schema within one day; per-PR branches spin up in under 15 seconds and cost cents each; PITR forensic branches are available for any point in the last 7 days; PgBouncer keeps the compute-pod connection count flat regardless of app-pod count.

Output:

Metric	Value
main cold-start	0 s (pinned)
staging cold-start on first hit	1-3 s
PR branch creation latency	5-15 s
PR branch hourly cost	~$0.05 idle, ~$0.20 active
Compute-pod connection count	~20 (via PgBouncer)
PITR forensic-branch window	7 days
p99 query latency (steady)	30-60 ms

Why this works — concept by concept:

• Pinned main compute — --compute-mode pinned trades a small always-on cost for zero request-path cold-start. Non-negotiable for customer-facing services; the cold-start tail on scale-to-zero is what makes serverless-Postgres a footgun for latency-sensitive OLTP.
• Nightly staging branch reset — branch delete + branch create is a metadata operation, not a full data restore. Staging schema always matches prod within one day; drift is impossible. Cost is bounded by the write bytes since the last reset.
• Ephemeral per-PR branches — CI creates on PR open, tears down on PR close. Each PR gets a full-fidelity prod copy without paying for full data storage. This is the Neon workflow that changes the dev-loop economics; 30 concurrent PRs cost the same as 3 provisioned staging environments would on RDS.
• PgBouncer transaction pooling — serverless Postgres + per-request connection open is a footgun. PgBouncer in transaction mode multiplexes short app connections onto a small warm upstream pool, taking cold-start out of the request path and cutting p99 latency by 5-10x under load.
• Cost — one pinned pod for prod + serverless-consumption for staging and PR branches + shared page store (cheap object storage) + 7-day PITR retention. The eliminated cost is the RDS-multi-AZ + nightly pg_dump + Fivetran + per-PR staging environments the traditional stack requires. Net O(1) per branch operation versus O(data) per RDS clone.

Design
Topic — design
Design problems on serverless database architectures

Practice →

SQL Topic — sql SQL problems on branching and PITR patterns

Practice →

---

3. OLTP + OLAP in one platform

Bidirectional sync between Lakebase and Delta under Unity Catalog — CDC out, MERGE in, one policy plane covering both

The mental model in one line: postgres lakehouse in the Lakebase sense means a single Databricks workspace where Postgres OLTP tables and Delta OLAP tables are peers under one Unity Catalog namespace, connected by bidirectional sync — near-zero-lag CDC that streams Lakebase mutations into Delta for analytics, and a MERGE / UPSERT reverse flow that pushes Delta-computed aggregates and features back into Lakebase for OLTP serving — with masking, RBAC, and lineage policies defined once in Unity Catalog and enforced automatically in both worlds. Every senior architect who has maintained a Fivetran-plus-Debezium-plus-reverse-ETL stack recognises immediately what this eliminates.

The two sync directions.

• Lakebase → Delta (CDC out). Every commit on a Lakebase table is captured (log-based, WAL-derived, similar to Debezium's Postgres connector) and appended to a target Delta table with near-zero lag. Downstream analytics — dashboards, ad-hoc SQL, ML training — read the Delta side.
• Delta → Lakebase (MERGE in). Aggregate or feature computations that produce a Delta output can be declared as a reverse sync into a Lakebase target. MERGE / UPSERT semantics: existing rows update, new rows insert. This is the reverse-ETL replacement.
• Bidirectional (both). Common for feature-store patterns: features computed in Delta land in Lakebase for serving; app writes to Lakebase flow back to Delta for training-data freshness.

Where it shows up.

• Feature stores. Compute features from event streams in Delta; serve them from Lakebase via MERGE-in sync. The app reads from Lakebase; the training pipeline reads from Delta. One store, two access patterns.
• Operational dashboards over warehouse data. Grafana / Metabase against Postgres wire is a mature ecosystem; connecting it to Lakebase — with Delta aggregates synced in — gives sub-minute-fresh operational dashboards without a query engine on Delta.
• Reverse-ETL replacement. Recommendations, propensity scores, customer segments computed in Delta pushed to Lakebase for the customer-facing service. Kills Hightouch / Census / Fivetran reverse-ETL subscriptions.
• AI-agent state. Databricks Mosaic AI agents that need short-term memory or session state land it in Lakebase; training data drawn from the same state via Delta CDC.

The three sync modes.

• snapshot mode. One-time full copy from source to target. Used to bootstrap a new sync or reset after schema break.
• incremental mode. Streaming CDC on the source; target is kept in near-lockstep. Latency SLO configurable (typical: 30s-5min lag target).
• merge mode. Batch UPSERT from source into target on a primary key. Used for Delta → Lakebase reverse flow where source is a periodic aggregate, not a stream.

What interviewers listen for.

• Do you name bidirectional sync rather than "some CDC pipeline"? — required accuracy.
• Do you say "Unity Catalog governs both" in the first sentence when governance comes up? — senior signal.
• Do you push back on "just build Debezium yourself" with the "one declared sync vs an Airflow DAG" argument? — senior signal.
• Do you name the primary-key requirement for MERGE-in sync — required answer.

Common interview probes on Lakebase + Delta sync.

• "How does the CDC out sync handle deletes?" — required answer: WAL-derived; deletes propagate as tombstones into Delta as _change_type = 'delete'.
• "What's the lag SLO?" — configurable per sync; 30s-5min typical; enforced by the sync engine which throttles on backlog.
• "How does a Unity Catalog masking policy apply to Lakebase?" — the policy is defined in Unity Catalog and enforced by both the Delta scan engine and the Lakebase Postgres query layer.
• "What breaks the sync?" — schema evolution without backward compatibility, primary-key changes, source table drops.

Worked example — Lakebase → Delta CDC for analytics

Detailed explanation. The canonical CDC-out flow: an orders table on Lakebase is the operational source of truth; a bronze.orders_cdc Delta table receives every mutation with near-zero lag; downstream analytics build silver / gold tables from the bronze. Walk through the sync declaration and the downstream MERGE.

• Source. app.orders on Lakebase (Postgres).
• Target. bronze.orders_cdc on Delta / Unity Catalog.
• Mode. Incremental streaming CDC; lag target 60 seconds.
• Downstream. A silver-layer MERGE that dedupes CDC events into a current-state table.

Question. Declare the CDC-out sync, show the bronze Delta table shape, and write the silver MERGE.

Input.

Component	Value
Source	Lakebase app.orders
Target	Delta bronze.orders_cdc
Mode	incremental (CDC)
Lag target	60 s
Silver	silver.orders (current state per order_id)

Code.

# 1. Declare the CDC-out sync as a Unity-Catalog artifact
databricks lakebase sync create \
    --name orders-cdc-out \
    --source lakebase://prod-orders/app.orders \
    --target delta://main.bronze.orders_cdc \
    --mode incremental \
    --lag-target-ms 60000 \
    --include-before-image true

-- 2. The bronze CDC Delta table shape (auto-created by the sync)
-- CREATE TABLE main.bronze.orders_cdc (
--     _change_type      STRING,           -- 'insert' | 'update' | 'delete'
--     _commit_ts        TIMESTAMP,        -- source-side commit time
--     _lsn              STRING,           -- WAL LSN for ordering
--     id                BIGINT,
--     customer_id       BIGINT,
--     total_cents       BIGINT,
--     status            STRING,
--     created_at        TIMESTAMP,
--     updated_at        TIMESTAMP,
--     _before           STRUCT<...>       -- pre-image (UPDATE / DELETE)
-- ) USING DELTA;

-- 3. Silver-layer MERGE — dedupe CDC events into current state
-- Runs on a Databricks Workflow every 5 min; idempotent

MERGE INTO main.silver.orders AS tgt
USING (
    -- Take the latest event per order_id from the last 10 min of CDC
    SELECT id, customer_id, total_cents, status, updated_at,
           _change_type, _commit_ts
    FROM (
        SELECT *,
               row_number() OVER (
                   PARTITION BY id
                   ORDER BY _lsn DESC
               ) AS rn
        FROM   main.bronze.orders_cdc
        WHERE  _commit_ts > current_timestamp() - INTERVAL 10 MINUTES
    )
    WHERE rn = 1
) AS src
ON tgt.id = src.id
WHEN MATCHED AND src._change_type = 'delete'
    THEN DELETE
WHEN MATCHED
    THEN UPDATE SET
        customer_id = src.customer_id,
        total_cents = src.total_cents,
        status      = src.status,
        updated_at  = src.updated_at
WHEN NOT MATCHED AND src._change_type <> 'delete'
    THEN INSERT (id, customer_id, total_cents, status, updated_at)
    VALUES     (src.id, src.customer_id, src.total_cents,
                src.status, src.updated_at);

Step-by-step explanation.

1. The sync create command declares the CDC-out sync as a single Unity-Catalog-managed artifact. Behind the scenes, the sync engine attaches to Lakebase's WAL (same primitive Debezium uses against standalone Postgres), reads each committed change, transforms it into the CDC-event shape, and appends to the target Delta table.
2. The bronze CDC table carries the full row on inserts and updates plus a _before struct for the pre-image on updates and deletes. _lsn is the durable ordering key — always dedupe on _lsn DESC in silver, never on _commit_ts (which can tie across transactions).
3. The silver MERGE runs every 5 minutes as a Databricks Workflow. It reads the last 10 minutes of CDC (a 5-minute safety buffer past its own schedule), picks the latest event per id via row_number() OVER (PARTITION BY id ORDER BY _lsn DESC), and merges into the current-state table.
4. The three MERGE branches handle the three CDC event types: delete → DELETE, update on existing → UPDATE SET, insert on non-matched → INSERT. The _change_type = 'delete' guard prevents a delete event from being re-inserted.
5. The 10-minute CDC read window trades a small overwrite of already-merged rows (idempotent — same result) for tolerance of the sync's up-to-60-second lag plus the Workflow's schedule jitter. In practice, silver reflects Lakebase within 5-8 minutes of the source commit.

Output.

Layer	Cadence	Row lifecycle	Purpose
bronze.orders_cdc	streaming (< 60s)	append-only	audit / replay
silver.orders	5 min batch	current state per PK	dashboards / joins
gold.orders_daily	daily	aggregated	reporting

Rule of thumb. For any Lakebase → Delta CDC deployment, keep bronze append-only + _lsn-ordered, dedupe to current-state silver via row_number() OVER (PARTITION BY pk ORDER BY _lsn DESC), and build gold from silver on a slower cadence. This is the classic bronze/silver/gold medallion pattern, adapted for Lakebase CDC.

Worked example — Delta → Lakebase MERGE for reverse-ETL

Detailed explanation. The reverse flow: recommendation features computed nightly in Delta land in Lakebase's user_features serving table via a declared MERGE-in sync. The customer-facing service reads from Lakebase over Postgres wire; the training pipeline reads from Delta. Walk through the declaration and the serving pattern.

• Source. analytics.recommendations.user_features_v2 (Delta) — output of a nightly Databricks job.
• Target. app.user_features on Lakebase — served to the recommendation service.
• Mode. MERGE UPSERT on user_id.
• Cadence. Nightly (or 4x/day for higher freshness).

Question. Declare the MERGE-in sync and show the serving-side pattern that reads through PgBouncer.

Input.

Component	Value
Source	Delta analytics.recommendations.user_features_v2
Target	Lakebase app.user_features
Primary key	user_id
Mode	merge (UPSERT)
Cadence	on-completion of Delta job

Code.

# 1. Declare the MERGE-in sync — Delta → Lakebase
databricks lakebase sync create \
    --name features-refresh \
    --source delta://analytics.recommendations.user_features_v2 \
    --target lakebase://prod-features/app.user_features \
    --mode merge \
    --primary-key user_id \
    --trigger on-source-commit

-- 2. Lakebase target schema — Postgres DDL, one time
CREATE TABLE app.user_features (
    user_id       BIGINT       PRIMARY KEY,
    features      JSONB        NOT NULL,
    model_version TEXT         NOT NULL,
    computed_at   TIMESTAMPTZ  NOT NULL,
    updated_at    TIMESTAMPTZ  NOT NULL DEFAULT clock_timestamp()
);

# 3. Serving-side pattern — the app reads from Lakebase via PgBouncer
import psycopg2
from psycopg2.pool import ThreadedConnectionPool

POOL = ThreadedConnectionPool(
    minconn=2, maxconn=5,
    host="127.0.0.1", port=6432,        # PgBouncer sidecar
    dbname="prod", user="rec-svc",
)

def score_recommendations(user_id: int, candidate_items: list[int]) -> list[tuple[int, float]]:
    conn = POOL.getconn()
    try:
        with conn.cursor() as cur:
            cur.execute("""
                SELECT features, model_version
                FROM   app.user_features
                WHERE  user_id = %s
            """, (user_id,))
            row = cur.fetchone()

        if row is None:
            return _cold_start_fallback(candidate_items)

        features, model_version = row[0], row[1]
        return _score(features, model_version, candidate_items)
    finally:
        POOL.putconn(conn)

-- 4. Monitoring — sync lag from the Lakebase side
SELECT max(updated_at) AS last_refresh,
       now() - max(updated_at) AS staleness
FROM   app.user_features;

Step-by-step explanation.

1. The sync create --mode merge --primary-key user_id --trigger on-source-commit declaration says: "whenever the source Delta table gets a new commit, run a MERGE UPSERT into the Lakebase target on user_id as the join key." No Airflow DAG, no Fivetran configuration, no reverse-ETL vendor.
2. The Lakebase target table is defined once in Postgres DDL — standard Postgres, standard indexing decisions (user_id as PRIMARY KEY gives the sync engine and the serving reads both a clustered access path). The sync engine will match columns by name; extra columns on the target (like updated_at) are populated by defaults.
3. The serving-side code is pure Postgres. psycopg2 opens a pooled connection through the PgBouncer sidecar and issues a single-row SELECT by primary key. Latency for this read is ~5-10ms p50 under normal load.
4. When a user has no features (cold-start), the code falls back to a non-personalised scoring path. This is a serving-side concern; the sync doesn't try to invent features for users who don't exist in the Delta source.
5. The monitoring query now() - max(updated_at) from the Lakebase side gives an on-call-friendly staleness metric. Alert on staleness > 2 * refresh_interval — a broken sync surfaces as a stale updated_at, not as a silent failure.

Output.

Access pattern	Layer	Latency	Freshness
Rec-service read by user_id	Lakebase	5-10 ms	last MERGE (minutes)
Analyst SQL over feature history	Delta	seconds	streaming
Feature-computation job read	Delta	tens of seconds	streaming
Backfill of new feature	one MERGE sync run	minutes-hours	one-shot

Rule of thumb. For any feature-serving deployment, compute features in Delta (cheap analytical compute), MERGE them into Lakebase (cheap OLTP serving), read from Lakebase over Postgres wire (mature ecosystem). This is the Lakebase reverse-ETL kill pattern; every feature store should use it once Lakebase is available.

Worked example — Unity Catalog masking policy across both worlds

Detailed explanation. The killer governance feature: a single Unity Catalog MASK definition on a PII column applies identically to Delta scans (analyst queries, ML training pipelines) and Lakebase reads (service queries, ops dashboards). Walk through the masking-policy definition and prove it enforces on both sides.

• Column. customers.email — PII.
• Policy. pii_readers group sees the raw email; everyone else sees a masked version.
• Enforcement. Same definition applies to SELECT email FROM customers from a Delta notebook and from a Lakebase psycopg2 query.

Question. Define the mask, verify it applies on both a Delta scan and a Lakebase read.

Input.

Component	Value
Masked column	customers.email
Group with clear access	pii_readers
Mask function	regexp_replace(email, ...)
Enforcement scope	Delta + Lakebase (both)

Code.

-- 1. Define the mask function in Unity Catalog
CREATE FUNCTION my_org.security.mask_email(email STRING)
RETURN CASE
    WHEN is_account_group_member('pii_readers') THEN email
    ELSE regexp_replace(email, '(.).*(@.*)', '\1***\2')
END;

-- 2. Bind the mask to the column — one definition, both worlds
ALTER TABLE my_org.app_customers.customers
    ALTER COLUMN email
    SET MASK my_org.security.mask_email;

-- 3. Delta side — analyst notebook query
SELECT customer_id, name, email
FROM   my_org.app_customers.customers
WHERE  customer_id IN (1, 2, 3);
-- Non-member sees: e***@example.com
-- Member of pii_readers sees: [e-mail]

# 4. Lakebase side — service read via psycopg2
import psycopg2

conn = psycopg2.connect(
    host="prod-customers.lakebase.us-east-1.databricks.com",
    port=5432, dbname="prod", user="rec-svc", sslmode="require",
)

with conn.cursor() as cur:
    cur.execute("""
        SELECT customer_id, name, email
        FROM   app_customers.customers
        WHERE  customer_id = %s
    """, (1,))
    row = cur.fetchone()

# The 'rec-svc' user is not a member of pii_readers,
# so the returned email is masked: e***@example.com
print(row)

Step-by-step explanation.

1. The mask function is defined once in Unity Catalog under my_org.security. It's a SQL function with a CASE that branches on group membership: pii_readers members see the raw email; everyone else sees a redacted version.
2. ALTER TABLE ... ALTER COLUMN email SET MASK binds the function to the column at the Unity Catalog level. From that moment on, any query — from any engine — that reads customers.email under Unity Catalog governance gets the masked value unless the caller is a pii_readers member.
3. On the Delta side, the enforcement happens at query-planning time: the Delta scan engine rewrites SELECT email into SELECT mask_email(email). Analysts see the mask; PII team members bypass it. No code change on the analyst side.
4. On the Lakebase side, the enforcement happens at the Postgres query layer: the service's SELECT is rewritten similarly. The service user rec-svc is not a pii_readers member, so the returned email is masked. No code change on the service side — the mask is invisible to application code.
5. The critical property is that the same policy definition covers both worlds. Compared to the traditional pattern of defining a Postgres VIEW for the service and a Snowflake / Databricks masking policy for the analyst — and hoping the two definitions stay in sync — this is a decisive win.

Output.

Query origin	Caller group	Returned value
Delta notebook	pii_readers	[e-mail]
Delta notebook	analyst	e***@example.com
Lakebase (service)	rec-svc	e***@example.com
Lakebase (dba)	pii_readers	[e-mail]
Airflow (ETL job)	etl-svc	e***@example.com

Rule of thumb. For any Lakebase deployment with PII, define masks once in Unity Catalog and let them apply across both worlds. Never define a separate Postgres VIEW for masking on the Lakebase side — the whole point of Lakebase-in-lakehouse is that you don't have to.

Senior interview question on Lakebase + Delta sync

A senior interviewer might ask: "You need to serve real-time recommendations at 5000 QPS from a customer-facing service, while also training a fresh recommendation model daily on the same feature history. Design the sync topology using Lakebase and Delta, define the governance policy for the PII columns, and describe what happens when the feature-computation job produces a schema-incompatible output."

Solution Using bidirectional Lakebase-Delta sync + Unity Catalog masking + schema-compatibility gates

# 1. Two syncs — one each direction

# CDC out — Lakebase reads (raw event captures) → Delta for training data
databricks lakebase sync create \
    --name events-cdc-out \
    --source lakebase://prod-features/app.raw_events \
    --target delta://main.bronze.events_cdc \
    --mode incremental \
    --lag-target-ms 60000 \
    --include-before-image true

# MERGE in — Delta-computed features → Lakebase for serving
databricks lakebase sync create \
    --name features-refresh \
    --source delta://main.gold.user_features_v2 \
    --target lakebase://prod-features/app.user_features \
    --mode merge \
    --primary-key user_id \
    --trigger on-source-commit \
    --schema-compat-mode backward

-- 2. Unity Catalog mask on PII (applies to both directions)
CREATE FUNCTION my_org.security.mask_email(email STRING)
RETURN CASE
    WHEN is_account_group_member('pii_readers') THEN email
    ELSE regexp_replace(email, '(.).*(@.*)', '\1***\2')
END;

ALTER TABLE my_org.app_customers.customers
    ALTER COLUMN email
    SET MASK my_org.security.mask_email;

-- 3. Lakebase serving table
CREATE TABLE app.user_features (
    user_id       BIGINT       PRIMARY KEY,
    features      JSONB        NOT NULL,
    model_version TEXT         NOT NULL,
    computed_at   TIMESTAMPTZ  NOT NULL,
    updated_at    TIMESTAMPTZ  NOT NULL DEFAULT clock_timestamp()
);

# 4. Serving-side read — Postgres wire through PgBouncer
def get_features(user_id: int) -> dict | None:
    conn = POOL.getconn()
    try:
        with conn.cursor() as cur:
            cur.execute("""
                SELECT features, model_version, updated_at
                FROM   app.user_features
                WHERE  user_id = %s
            """, (user_id,))
            row = cur.fetchone()
        return _row_to_dict(row)
    finally:
        POOL.putconn(conn)

Step-by-step trace.

Component	Config	Purpose
CDC out (Lakebase→Delta)	incremental, 60s lag, before-image	fresh training data
MERGE in (Delta→Lakebase)	on-source-commit, PK user_id, schema-compat backward	fresh serving features
Unity Catalog mask	one CREATE FUNCTION, one ALTER TABLE	one PII policy, both worlds
Lakebase serving	Postgres DDL, PgBouncer front	5-10ms p50 reads
Schema-compat gate	backward mode	breaks the sync loudly on incompatible change

After deployment, raw events flow Lakebase → Delta with sub-minute lag for training; features computed in Delta flow back into Lakebase within minutes of each computation; the recommendation service reads from Lakebase at 5000 QPS with 5-10ms p50 latency; PII masking is defined once in Unity Catalog and enforced identically across both engines; a schema-incompatible feature output fails the MERGE-in sync at declaration time rather than corrupting the serving table.

Output:

Metric	Value
CDC-out lag (steady)	< 60 s
MERGE-in freshness	minutes after Delta commit
Serving p50 latency	5-10 ms
Serving p99 latency	30-60 ms
PII masking definitions	1 (Unity Catalog)
Failure mode on bad schema	sync fails at compat check

Why this works — concept by concept:

• Bidirectional sync as a first-class primitive — CDC-out and MERGE-in are both declared as Unity-Catalog-managed sync artifacts, not built as bespoke Airflow DAGs or Fivetran connectors. The sync engine handles WAL attachment, backpressure, retries, and lineage automatically.
• Schema-compatibility gate — --schema-compat-mode backward refuses to run the MERGE if the source Delta schema changes in a non-backward-compatible way. This catches the "someone dropped a column and now serving is broken" bug at declaration time.
• Unity Catalog masking spanning both — one MASK definition covers Delta scans and Lakebase queries. Removes an entire class of "the two policy definitions drifted" bugs; also removes the audit-checkbox work of proving they match.
• PgBouncer + Postgres wire on the serving path — nothing exotic on the app side. The recommendation service opens a pooled Postgres connection to a sidecar, issues a single-row SELECT by PK, gets back a JSONB blob. Every existing library, ORM, and monitoring tool works out of the box.
• Cost — one Lakebase instance, two syncs, one MASK policy, one PgBouncer sidecar per app pod. Compared to the equivalent Debezium + Fivetran + Redis serving + two masking policies stack, this is dramatically fewer moving parts and one governance plane. Net O(delta) per commit on the CDC side and O(changed features) per MERGE on the reverse side, at a single-vendor operational surface.

Streaming
Topic — streaming
Streaming CDC and bidirectional-sync problems

Practice →

SQL Topic — sql SQL problems on MERGE and UPSERT patterns

Practice →

---

4. Operational patterns + when to use Lakebase

Three winning lanes (reverse-ETL replacement, real-time feature serving, operational dashboards) and one anti-pattern (very-high-QPS bid paths)

The mental model in one line: oltp lakehouse gives you three operational patterns Lakebase does dramatically better than the incumbent stack — reverse-ETL replacement (Delta-computed data pushed to a Postgres-wire serving store), real-time feature serving (compute in Delta, serve from Lakebase), and operational dashboards over fresh warehouse data — and one anti-pattern where you should still reach for a specialised OLTP store, namely very-high-QPS bid paths where cold-start tails and connection-pool churn matter more than governance parity. Every senior architect needs the four-lane map in their head before defending or challenging a Lakebase adoption.

Lane 1 — Reverse-ETL replacement.

• Where it shows up. Every team with a Delta warehouse and a customer-facing app has a reverse-ETL job today: compute segments, propensity scores, next-best-action recommendations in Delta; ship them to the operational Postgres so the app can read them.
• The old stack. Airflow DAG → dbt model in Delta → Hightouch / Census / Fivetran sync → RDS Postgres → app read. Three vendors, four hops, per-record cost, 15-30 minute latency, separate governance in each system.
• The Lakebase pattern. Declare a --mode merge --primary-key <pk> sync from the Delta gold table to the Lakebase target. One artifact; Unity Catalog handles lineage; latency drops to minutes; vendor cost drops to zero (bundled).
• The win. Removes an entire operational tier from the stack; consolidates governance; lets the analytics team own the reverse flow directly without a separate ops team.

Lane 2 — Real-time feature serving.

• Where it shows up. ML inference systems that need to score against features computed from event streams. Traditional stack: Redis or DynamoDB in front of a feature-compute pipeline.
• The old stack. Kafka → Spark Streaming → Delta (features) → Feast/Tecton feature store → Redis for serving → model. Multi-hop, multiple stores, feature-store-vendor lock-in, complex online/offline parity story.
• The Lakebase pattern. Features computed in Delta land in a Lakebase serving table via MERGE-in sync. Model reads via Postgres wire; sub-50ms p50 latency; online store is real Postgres so SQL joins across features are trivial.
• The win. Eliminates the online/offline parity problem — the same MERGE that lands features online can be reversed for offline reads. One store, two access patterns.

Lane 3 — Operational dashboards over warehouse data.

• Where it shows up. Ops teams want Grafana / Metabase / Superset dashboards over "the state of the warehouse right now" — order volumes, error rates, SLO burn, financial reconciliation.
• The old stack. Warehouse → materialised view → BI tool with 15-30 minute cache. Or, warehouse → Trino / Presto → dashboard, with query concurrency limits and cold-cache latency spikes.
• The Lakebase pattern. A Delta-computed aggregate (e.g. orders_last_hour_by_region) syncs to a Lakebase table; the dashboard queries Lakebase over Postgres wire; sub-second query latency; fresh within minutes.
• The win. Turns the "warehouse dashboard latency" complaint into a solved problem. Grafana against Postgres is a mature ecosystem; the freshness is what was hard to get.

Anti-pattern — Very-high-QPS OLTP bid paths.

• Where NOT to use Lakebase. Sub-millisecond tail-latency workloads at very high QPS: ad-tech real-time bidding, high-frequency trading pre-trade paths, sub-ms leaderboard writes, chat message fan-out at 100k+ QPS.
• Why. Serverless Postgres has cold-start tails, connection-pool sizing constraints, and network hops that a locally-provisioned specialised OLTP (Aurora, DynamoDB, ScyllaDB) can beat by tens of milliseconds at the tail. When the workload's SLO cares about p99.9 and every millisecond, Lakebase is not the right tool.
• Signals. If your target SLO includes p99.9 or p99.99 tail-latency figures below 20ms, if your write QPS exceeds 50k sustained, or if your workload is stateful in ways that penalise scale-to-zero cold-start — pick a specialised OLTP and integrate to the lakehouse via CDC.
• The senior signal. Naming this anti-pattern unprompted is a stronger senior signal than praising Lakebase. Every technology has a workload envelope; knowing where it doesn't fit is more valuable than knowing where it does.

Where each lane wins on cost.

• Reverse-ETL. Bundled sync cost vs $2k-$10k/month per reverse-ETL vendor. Win: hundreds to thousands of dollars per month per pipeline.
• Feature serving. Bundled Lakebase cost vs Redis cluster + feature-store vendor. Win: often 2-5x total cost reduction.
• Ops dashboards. Bundled Lakebase cost vs Trino cluster + separate BI-cache. Win: dashboard latency drops, cost neutral or lower.
• Bid-path OLTP. No cost win — the specialised store's per-query cost at 100k+ QPS is genuinely lower than Lakebase's serverless model would be at the same scale.

What interviewers listen for.

• Do you name three lanes plus one anti-pattern unprompted? — senior signal.
• Do you quote specific SLO numbers (sub-50ms serving, sub-second dashboard) rather than "fast"? — required accuracy.
• Do you push back on "use Lakebase for everything" with the QPS-envelope argument? — required senior signal.
• Do you frame reverse-ETL as "an operational tier we remove" rather than "a job we rewrite"? — senior signal.

Worked example — reverse-ETL replacement for a segment-sync pipeline

Detailed explanation. The canonical reverse-ETL job: nightly, compute customer_segment in Delta from purchase history; sync to Postgres so the marketing app can query "customers in segment X." Old stack was Airflow → dbt → Hightouch → RDS. New stack is one Lakebase sync. Walk through the migration.

• Before. Airflow DAG runs a dbt model, materialises analytics.gold.customer_segments, kicks off a Hightouch sync to the RDS crm.customer_segments table.
• After. analytics.gold.customer_segments on Delta syncs directly to crm.customer_segments on Lakebase via a declared MERGE sync.
• Downstream. Marketing app reads Lakebase via Postgres wire.

Question. Show the before/after topology, define the Lakebase sync, and quantify the savings.

Input.

Component	Before	After
Source	Delta gold.customer_segments	Delta gold.customer_segments
Middle tier	Airflow + Hightouch	Lakebase sync (Unity Catalog)
Target	RDS crm.customer_segments	Lakebase crm.customer_segments
Latency	30-60 min	5-15 min
Vendor cost	~$1500/mo Hightouch	$0 (bundled)

Code.

# 1. Declare the sync — one artifact replaces the DAG + Hightouch
databricks lakebase sync create \
    --name segments-refresh \
    --source delta://analytics.gold.customer_segments \
    --target lakebase://prod-crm/crm.customer_segments \
    --mode merge \
    --primary-key customer_id \
    --trigger on-source-commit

-- 2. Lakebase target — same DDL RDS would have used
CREATE TABLE crm.customer_segments (
    customer_id       BIGINT       PRIMARY KEY,
    segment_id        TEXT         NOT NULL,
    segment_score     NUMERIC(6,4) NOT NULL,
    model_version     TEXT         NOT NULL,
    computed_at       TIMESTAMPTZ  NOT NULL,
    updated_at        TIMESTAMPTZ  NOT NULL DEFAULT clock_timestamp()
);

CREATE INDEX idx_customer_segments_segment
    ON crm.customer_segments (segment_id);

# 3. Marketing app read — Postgres wire, unchanged from RDS days
def customers_in_segment(segment_id: str, limit: int = 1000) -> list[int]:
    conn = POOL.getconn()
    try:
        with conn.cursor() as cur:
            cur.execute("""
                SELECT customer_id
                FROM   crm.customer_segments
                WHERE  segment_id = %s
                ORDER  BY segment_score DESC
                LIMIT  %s
            """, (segment_id, limit))
            return [row[0] for row in cur.fetchall()]
    finally:
        POOL.putconn(conn)

Step-by-step explanation.

1. The old topology had four moving parts: Airflow schedule, dbt model, Hightouch connector, RDS target. Each was owned by a different team (data eng, analytics, ops, DBA). The new topology has one artifact — the sync declaration — owned by the analytics team.
2. --trigger on-source-commit means the sync runs whenever a new Delta commit lands on the source, not on a fixed schedule. Freshness improves from "nightly" to "minutes after dbt commits."
3. The Lakebase target DDL is standard Postgres — same DDL the RDS team would have written. The idx_customer_segments_segment index supports the common access pattern (WHERE segment_id = ?), independent of the sync mechanism.
4. The marketing app code doesn't change at all beyond the DSN swap. Same psycopg2, same SQL, same result shape. This is the wire-compatibility payoff.
5. On cost: Hightouch subscription cost ($1.5k/month) disappears; RDS instance cost is displaced by Lakebase serverless (typically lower on segment-sync workloads because the query rate is moderate). Airflow simplifies (no longer needs the Hightouch task). Total operational surface shrinks.

Output.

Metric	Before	After
Freshness	30-60 min	5-15 min
Hightouch cost	~$1.5k/mo	$0
Airflow tasks	3 (dbt, hightouch, monitor)	1 (dbt)
Distinct governance planes	3 (Airflow, Hightouch, RDS)	1 (Unity Catalog)
Marketing app changes	none	none

Rule of thumb. For any reverse-ETL job whose target is a Postgres-family database and whose source is a Delta table, migrate to a Lakebase MERGE sync first — even before considering broader migrations. The reverse-ETL lane is where Lakebase has the sharpest incumbent-vs-Lakebase cost delta.

Worked example — real-time feature serving with online/offline parity

Detailed explanation. ML feature serving is the second lane. The pattern: features are computed streaming into Delta from event streams; a MERGE sync lands them in Lakebase; the online serving path reads from Lakebase; the offline training path reads from Delta; the same MERGE ensures both sides see the same feature vector at the same time. Walk through a fraud-scoring feature-serving deployment.

• Feature. user_txn_stats_60d — computed from 60 days of transaction history, refreshed every 15 minutes.
• Online path. Fraud-scoring service, ~2000 QPS, sub-50ms SLO. Reads user features from Lakebase before scoring a new transaction.
• Offline path. Nightly retraining reads from Delta (same feature history).
• Parity guarantee. Because both sides read from tables kept in sync by the same declared sync, the "training-serving skew" bug class is closed by construction.

Question. Show the sync declaration, the online read path, and the offline train read.

Input.

Component	Value
Feature-compute source	Delta ml.gold.user_txn_stats_60d
Online target	Lakebase ml_online.user_features
Refresh cadence	every 15 min
Online SLO	p99 < 50 ms
Offline train	reads Delta directly

Code.

# Sync — Delta feature output → Lakebase serving table
databricks lakebase sync create \
    --name user-features-serving \
    --source delta://ml.gold.user_txn_stats_60d \
    --target lakebase://prod-ml/ml_online.user_features \
    --mode merge \
    --primary-key user_id \
    --trigger schedule --cron "*/15 * * * *"

-- Lakebase serving table
CREATE TABLE ml_online.user_features (
    user_id           BIGINT       PRIMARY KEY,
    txn_count_60d     INTEGER      NOT NULL,
    txn_amount_60d    NUMERIC(14,2) NOT NULL,
    txn_pct_night     NUMERIC(4,3) NOT NULL,
    txn_pct_intl      NUMERIC(4,3) NOT NULL,
    computed_at       TIMESTAMPTZ  NOT NULL,
    updated_at        TIMESTAMPTZ  NOT NULL DEFAULT clock_timestamp()
);

# Online path — fraud-scoring service
def score_txn(user_id: int, txn: dict) -> float:
    conn = POOL.getconn()
    try:
        with conn.cursor() as cur:
            cur.execute("""
                SELECT txn_count_60d, txn_amount_60d,
                       txn_pct_night, txn_pct_intl
                FROM   ml_online.user_features
                WHERE  user_id = %s
            """, (user_id,))
            row = cur.fetchone()
        if row is None:
            return _cold_start_score(txn)
        features = dict(zip(
            ["txn_count_60d", "txn_amount_60d", "txn_pct_night", "txn_pct_intl"],
            row
        ))
        return _fraud_model(features, txn)
    finally:
        POOL.putconn(conn)

# Offline path — nightly retraining (Databricks notebook)
train_df = spark.sql("""
    SELECT f.user_id,
           f.txn_count_60d, f.txn_amount_60d,
           f.txn_pct_night, f.txn_pct_intl,
           l.is_fraud
    FROM   ml.gold.user_txn_stats_60d f
    JOIN   ml.gold.txn_labels l ON l.user_id = f.user_id
    WHERE  l.event_date BETWEEN date_sub(current_date(), 30) AND current_date()
""")
train_model(train_df)

Step-by-step explanation.

1. The sync is scheduled every 15 minutes — a good balance between feature freshness and MERGE-sync cost. --trigger schedule --cron "*/15 * * * *" runs the MERGE deterministically; you can also use on-source-commit if the feature-compute job commits at a variable cadence.
2. The Lakebase serving table has the exact column set the scoring model expects. When a new feature is added, the compute job produces the new column in Delta, the sync propagates it to Lakebase (backward-compatible schema evolution), and the model can start using it.
3. The online path is a single-row indexed SELECT — ~5-10ms p50 through PgBouncer, well under the 50ms SLO. Cold-start (new user with no features) falls back to a heuristic score; the "unknown user" case is a serving-side concern, not a sync concern.
4. The offline train path reads directly from the Delta source. Because the MERGE sync propagates every row (not a subset), the training features are identical to the serving features at the moment of MERGE — no training-serving skew.
5. The parity guarantee is worth expanding on: in the traditional stack, features are computed once for offline and separately for online, and it's a constant fight to keep them consistent. Here, one compute produces one Delta output, MERGE'd into one Lakebase target, read from both sides. Skew is impossible at the feature-vector level.

Output.

Path	Read from	Latency	Freshness
Online scoring	Lakebase	5-10 ms	last MERGE (< 15 min)
Offline training	Delta	seconds (batch)	streaming
Backfill	Delta → Lakebase MERGE	minutes	one-shot
Debugging	Lakebase (JSON dump)	milliseconds	current serving state

Rule of thumb. For any ML feature-serving deployment, compute once in Delta, MERGE to Lakebase for online, read Delta for offline. The training-serving parity bug class is closed by construction; every ML team should adopt this pattern once Lakebase is in the workspace.

Worked example — operational dashboard with Grafana + Lakebase

Detailed explanation. The third winning lane: ops teams want low-latency dashboards over fresh warehouse data. Traditional pattern was Trino / Presto in front of Iceberg / Delta with per-query scaling issues; better pattern is a Lakebase-served aggregate that Grafana queries via the Postgres data source. Walk through a real deployment for an order-processing team.

• Aggregate. orders_last_hour_by_region — number of orders and total revenue per region for the last 60 minutes.
• Source. Delta bronze table bronze.orders_cdc (CDC out from Lakebase orders table).
• Compute. A Databricks Workflow every 60 seconds refreshes the aggregate.
• Serve. MERGE to Lakebase ops.orders_by_region; Grafana queries Lakebase.

Question. Build the aggregate, declare the sync, wire Grafana.

Input.

Component	Value
Aggregate	orders_last_hour_by_region
Cadence	60 s
Source	Delta bronze.orders_cdc
Grafana data source	Postgres → Lakebase

Code.

-- 1. Delta view / materialised table — 60-second refresh
CREATE OR REPLACE TABLE ops_delta.orders_last_hour_by_region AS
SELECT region,
       count(*)         AS order_count,
       sum(total_cents) AS revenue_cents,
       current_timestamp() AS computed_at
FROM   main.silver.orders
WHERE  created_at > current_timestamp() - INTERVAL 1 HOUR
GROUP  BY region;

# 2. Sync the aggregate to Lakebase — Grafana reads from here
databricks lakebase sync create \
    --name ops-orders-by-region \
    --source delta://ops_delta.orders_last_hour_by_region \
    --target lakebase://prod-ops/ops.orders_by_region \
    --mode merge \
    --primary-key region \
    --trigger schedule --cron "* * * * *"    # every minute

-- 3. Lakebase target
CREATE TABLE ops.orders_by_region (
    region          TEXT         PRIMARY KEY,
    order_count     INTEGER      NOT NULL,
    revenue_cents   BIGINT       NOT NULL,
    computed_at     TIMESTAMPTZ  NOT NULL,
    updated_at      TIMESTAMPTZ  NOT NULL DEFAULT clock_timestamp()
);

# 4. Grafana panel query (Postgres data source)
SELECT region,
       order_count,
       revenue_cents / 100.0 AS revenue_usd,
       computed_at
FROM   ops.orders_by_region
ORDER  BY revenue_cents DESC;

Step-by-step explanation.

1. The aggregate is a single Delta table refreshed every 60 seconds by a lightweight Workflow job. Compute lives in Delta because that's where the CDC-received order data lands; result set is tiny (one row per region) so the sync is cheap.
2. The sync runs on a 1-minute cron. Grafana panels refresh at some cadence (typically 15-60 seconds); pinning the source refresh at 60 seconds keeps the dashboard fresh without over-refreshing the compute.
3. The Lakebase target is a tiny table (dozens of rows for regions worldwide) with region as the primary key. Grafana's Postgres data source connects with a service credential and issues plain SQL — no Databricks SDK, no custom driver, no Trino connector.
4. Grafana's dashboard shows sub-second query latency (because it's hitting a tiny indexed Postgres table) with freshness bounded at ~2 minutes (60s compute + 60s sync). This is dramatically better than the "Trino query takes 15 seconds over a large Delta table" experience.
5. This pattern scales to dozens of dashboards. Each dashboard has its own tiny Lakebase target table populated by a tiny Delta aggregate + tiny sync. The compute cost is proportional to the aggregate work (small); the serving cost is proportional to the query rate (small).

Output.

Component	Latency	Freshness
Delta aggregate refresh	seconds	60 s cadence
Lakebase sync	seconds	60 s cadence
Grafana panel query	< 100 ms	60-120 s stale worst case
Overall dashboard freshness	~2 min	acceptable for ops

Rule of thumb. For any operational dashboard over warehouse data, compute the tiny aggregate in Delta, MERGE it to a tiny Lakebase table, connect Grafana with the Postgres data source. This is dramatically simpler and cheaper than putting Trino / Presto in front of Delta for dashboard workloads.

Senior interview question on Lakebase operational patterns

A senior interviewer might ask: "Your ad-platform team is evaluating Lakebase for three workloads: (1) segment-sync from Delta to the CRM app, (2) sub-millisecond bid-time OLTP at 200k QPS, (3) real-time bidder-performance dashboards. For each workload, pick a store and defend your choice with the four axes we covered earlier."

Solution Using Lakebase for lanes 1 and 3, keeping a specialised OLTP for lane 2

# Lane 1 — segment sync — Lakebase MERGE-in sync
databricks lakebase sync create \
    --name crm-segments \
    --source delta://analytics.gold.customer_segments \
    --target lakebase://prod-crm/crm.customer_segments \
    --mode merge \
    --primary-key customer_id \
    --trigger on-source-commit

# Lane 3 — bidder-performance dashboards — Lakebase MERGE for aggregates
databricks lakebase sync create \
    --name bidder-perf-1m \
    --source delta://ops_delta.bidder_perf_1m \
    --target lakebase://prod-ops/ops.bidder_perf \
    --mode merge \
    --primary-key bidder_id \
    --trigger schedule --cron "* * * * *"

Lane 2 — sub-ms bid-path OLTP — NOT Lakebase
    Store:     Aurora Postgres provisioned + read replicas OR
               DynamoDB on-demand OR ScyllaDB self-hosted
    Reason:    200k QPS + sub-ms p99.9 SLO exceeds Lakebase's
               serverless envelope. Cold-start on scale-to-zero
               is a bidder-drop event; connection-pool churn
               at 200k QPS overwhelms transaction-pool warmup.
    Integration: CDC out (Debezium against Aurora) → Delta
                 for analytics; Lakebase-managed reverse-ETL
                 not needed on the bid path.

-- Lakebase target for lane 1 — CRM segments
CREATE TABLE crm.customer_segments (
    customer_id       BIGINT       PRIMARY KEY,
    segment_id        TEXT         NOT NULL,
    segment_score     NUMERIC(6,4) NOT NULL,
    computed_at       TIMESTAMPTZ  NOT NULL,
    updated_at        TIMESTAMPTZ  NOT NULL DEFAULT clock_timestamp()
);

-- Lakebase target for lane 3 — bidder performance
CREATE TABLE ops.bidder_perf (
    bidder_id     BIGINT       PRIMARY KEY,
    bids_1m       INTEGER      NOT NULL,
    wins_1m       INTEGER      NOT NULL,
    spend_cents_1m BIGINT      NOT NULL,
    updated_at    TIMESTAMPTZ  NOT NULL DEFAULT clock_timestamp()
);

Step-by-step trace.

Lane	Workload	Store	Reason
1	Segment sync (batch, 5-15 min freshness)	Lakebase	wire compat + governance + cost
2	Bid-path OLTP (200k QPS, sub-ms p99.9)	Aurora / DynamoDB	Lakebase envelope exceeded
3	Bidder-perf dashboards (per-minute freshness)	Lakebase	Grafana + Postgres + fresh aggregate
Integration	Lane 2 → analytics	Debezium → Delta	log-based CDC into the lakehouse
Governance	Lane 1 + 3	Unity Catalog	one policy plane
Governance	Lane 2	Aurora IAM + Unity Catalog on Delta	separate policies

After the deployment, lanes 1 and 3 sit inside the Databricks workspace with one governance plane; lane 2 stays on a specialised OLTP for the bid path with CDC out to Delta for analytics; the analytics side is unified because everything lands in Delta / Unity Catalog eventually.

Output:

Workload	Store	p99 latency	Cost lever
Segment sync	Lakebase	seconds (query)	replaces Hightouch
Bid-path OLTP	Aurora	sub-ms	already provisioned
Bidder dashboard	Lakebase	< 100 ms	replaces Trino for dashboards
Analytics (all lanes)	Delta	seconds-minutes	one warehouse

Why this works — concept by concept:

• Lane-based decision making — never adopt Lakebase for "everything" or reject it because "the bid path is too fast." Pick each workload, walk the four axes, land on the right store. Lakebase wins lanes 1 and 3; the specialised OLTP wins lane 2.
• Governance-follows-workload — Unity Catalog governs where the data lives. For lanes 1 and 3 (Lakebase), Unity Catalog policies apply. For lane 2 (Aurora), Aurora IAM + column-level grants govern; the Delta CDC side inherits Unity Catalog policies. Two governance planes exist honestly rather than being papered over.
• CDC out of the specialised OLTP — even where Lakebase is not the primary store, the Databricks lakehouse remains the analytics destination via traditional CDC (Debezium against Aurora, DynamoDB Streams → Kinesis → Delta). Analytics stays unified.
• Anti-pattern discipline — the senior signal is knowing not to force Lakebase onto a bid path. Every vendor tells you their thing does everything; every senior engineer knows every technology has an envelope. Being able to defend the envelope in an interview is worth more than technical Lakebase depth.
• Cost — Lakebase for the lanes it wins; specialised OLTP for the lanes it doesn't; one unified analytics layer downstream. Net operational cost is lower than pure-Lakebase (because you avoid over-paying on the bid path) and lower than pure-specialised-OLTP (because lanes 1 and 3 no longer require reverse-ETL vendors). Right tool per lane, at every workload.

Design
Topic — design
Design problems on feature stores and serving paths

Practice →

Streaming Topic — streaming Streaming problems on operational-dashboard pipelines

Practice →

---

5. Migration + interview signals

From RDS Postgres, Neon, or Snowflake Unistore into Lakebase — three source paths, one target, and the interview probes senior candidates must answer

The mental model in one line: migrating to Lakebase is a three-source problem — from RDS Postgres you pg_dump the schema and use Delta CDC to backfill history, from Neon (the acquisition foundation) you get a native path with near-zero schema translation, from Snowflake Unistore you do the heaviest lift (schema translate, RBAC rewrite, Horizon-to-Unity-Catalog policy migration) — and every senior interviewer will probe the migration mechanics to test whether you understand serverless Postgres semantics, branching, Unity Catalog integration, and the OLTP-plus-OLAP convergence story well enough to lead the migration for their team. Every senior architect who has led a database migration knows the migration mechanics are half the story; the other half is the operational readiness the target platform demands.

Source 1 — from RDS / Aurora Postgres.

• Path. pg_dump --schema-only on RDS → apply on Lakebase; pg_dump --data-only for initial data OR Debezium CDC from RDS into Delta and MERGE into Lakebase; cutover during a maintenance window.
• Cost. ~2 engineer-weeks for a moderate schema (30 tables, 500GB data).
• Gotchas. Postgres extensions — check the Lakebase compatibility matrix; not every RDS extension is supported. Also revoke direct RDS access post-cutover to prevent write-drift.
• Rollback. Reverse-sync from Lakebase back to RDS via MERGE-in during a defined rollback window.

Source 2 — from Neon (native path).

• Path. Neon-to-Lakebase migration is the cleanest — Lakebase is the Neon fork; the storage engine is the same. Databricks provides a native import path that avoids full data copy where possible.
• Cost. ~2 days for a moderate deployment.
• Gotchas. Branch history may not survive migration; recreate branch strategy in Lakebase. Neon-specific extensions must be checked against the Lakebase support matrix.
• Rollback. Trivial while the Neon source remains accessible; just point apps back.

Source 3 — from Snowflake Unistore Hybrid Tables.

• Path. The heaviest migration. Snowflake Hybrid Tables are not Postgres wire — application code that hits Unistore via Snowflake connectors must be rewritten to psycopg2 / pgx. Data migration via COPY INTO to S3 → Delta → MERGE to Lakebase. RBAC translation from Horizon to Unity Catalog.
• Cost. ~3 engineer-weeks for schema + governance; longer if app code has heavy Snowflake dependencies.
• Gotchas. Snowflake-specific SQL dialects (VARIANT, semi-structured functions) map imperfectly to Postgres JSONB.
• Rollback. Keep Unistore in read-only mode during cutover; reverse-sync via MERGE if needed.

The four interview probes on migration.

• Serverless Postgres semantics. "What happens when a Lakebase compute pod scales to zero mid-transaction?" — required answer: transactions are protected by the shared page-store WAL; the transaction survives compute recycle, but the client connection is dropped and must retry.
• Branching cost. "How much does it cost to keep a branch of a 1TB database for 30 days?" — required answer: bounded by divergent-page bytes, not by 1TB × 30d. Typical answer: single-digit dollars for a lightly-written branch, low tens for a heavily-written branch.
• Unity Catalog integration. "Show me the RBAC declaration that grants the recommendation service read-only access to app.user_features and no access to app.customers." — required answer: GRANT SELECT ON app.user_features TO rec-svc; REVOKE ALL ON app.customers FROM rec-svc; under Unity Catalog namespacing.
• OLTP+OLAP convergence. "Why does bidirectional Lakebase-Delta sync matter more than a fast CDC pipeline?" — required answer: because governance parity + one lineage view + one policy plane is worth more than the last N milliseconds of latency the specialised pipelines can save.

Common interview probes on Lakebase migration.

• "How would you migrate from RDS?" — pg_dump + Delta CDC backfill + cutover window.
• "How would you migrate from Neon?" — native path; keep source read-only during cutover.
• "How would you migrate from Unistore?" — schema translate + Horizon-to-UC RBAC rewrite + application code changes.
• "What's the biggest risk?" — extension compatibility on RDS; branch-history loss on Neon; wire-protocol dependency on Unistore.

Worked example — RDS Postgres migration with a zero-downtime cutover

Detailed explanation. The canonical RDS → Lakebase migration: use pg_dump --schema-only for schema, run Debezium against RDS to stream mutations into a Delta staging table, MERGE from Delta into Lakebase to backfill, cut over during a short maintenance window when Debezium is caught up. Walk through the phases.

• Phase 1. Schema migration via pg_dump --schema-only and psql apply.
• Phase 2. Data backfill via Debezium (RDS → Delta) + MERGE (Delta → Lakebase).
• Phase 3. Cutover: freeze RDS writes, drain last Debezium events, switch DSN, unfreeze on Lakebase.
• Phase 4. Decommission RDS after a soak period.

Question. Write the schema-dump command, the Debezium config for the CDC stream, and the cutover script.

Input.

Component	Value
Source	RDS Postgres 15
Target	Lakebase (Postgres 16-compat)
Data volume	500 GB
Downtime budget	< 5 min
Debezium topic	rds-cdc.public.*

Code.

# 1. Schema dump + apply
pg_dump --schema-only \
        --no-owner --no-privileges \
        "postgres://[e-mail]:5432/production" \
        > rds_schema.sql

# Manual review — check extensions against Lakebase compat matrix
# (uuid-ossp OK, postgis check version, pg_trgm OK, ...)

# Apply on Lakebase
psql "postgres://[e-mail]:5432/prod" \
     -f rds_schema.sql

# 2. Debezium — RDS → Delta CDC (via Kafka Connect + Delta sink)
name: rds-to-delta
config:
  connector.class: io.debezium.connector.postgresql.PostgresConnector
  database.hostname: rds-src.internal
  database.port: 5432
  database.dbname: production
  database.user: cdc_reader
  database.password: ${env:CDC_PASSWORD}
  plugin.name: pgoutput
  slot.name: debezium_rds_migration
  publication.name: migration_pub
  snapshot.mode: initial
  topic.prefix: rds_migration
  tombstones.on.delete: true

# 3. Backfill sync — Delta staging → Lakebase target
databricks lakebase sync create \
    --name rds-backfill \
    --source delta://main.staging.rds_migration_orders \
    --target lakebase://prod-orders/app.orders \
    --mode merge \
    --primary-key id \
    --trigger continuous \
    --lag-target-ms 60000

# 4. Cutover script (< 5-min downtime)
set -e

# 4a. Freeze RDS writes — revoke INSERT/UPDATE/DELETE from app roles
psql "$RDS_ADMIN_DSN" <<'SQL'
    REVOKE INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public FROM app_writer;
SQL

# 4b. Wait for Debezium + backfill sync to drain (lag → 0)
until [ "$(check_backfill_lag_ms)" -lt 500 ]; do
    echo "Backfill lag: $(check_backfill_lag_ms) ms — waiting..."
    sleep 2
done

# 4c. Point app config to Lakebase DSN
kubectl set env deployment/orders-svc DB_HOST=prod-orders.lakebase.us-east-1.databricks.com

# 4d. Unblock writes on Lakebase (they were already allowed; app just wasn't sending them)
echo "Cutover complete. Monitor Lakebase for 24h before decommissioning RDS."

Step-by-step explanation.

1. Phase 1 — schema dump — is straightforward pg_dump --schema-only. The manual review is critical: not every RDS extension is supported on Lakebase, and a missed extension turns into a cutover-day surprise. Check pg_extension on RDS against the Lakebase compat matrix in advance.
2. Phase 2 — backfill — uses Debezium's initial snapshot (snapshot.mode: initial) followed by continuous WAL streaming. This captures every existing row and every subsequent mutation into a Delta staging table. The MERGE sync from Delta to Lakebase then applies them to the target on the primary key.
3. Phase 3 — cutover — starts by revoking write permissions on RDS. From that moment, no new mutations happen on the source; the backfill can drain to zero lag. Once check_backfill_lag_ms < 500, Lakebase reflects RDS exactly.
4. The DSN swap in the app deployment (kubectl set env) is a rolling restart of app pods. When each pod restarts, it connects to Lakebase instead of RDS. If a pod is mid-transaction when it's rolled, the transaction rolls back on RDS (which now rejects writes) and the retry lands on Lakebase.
5. Phase 4 — decommission — waits 24 hours to confirm no residual RDS traffic (via pg_stat_activity), then drops the RDS instance. Keep the last RDS snapshot on S3 as a safety net for rollback for at least the compliance-mandated retention window.

Output.

Phase	Elapsed	Downtime
Schema dump + apply	30 min	0
Backfill run (500 GB)	4-8 h	0
Cutover window	3-5 min	3-5 min
Soak period	24 h	0
RDS decommission	after soak	0
Total	~2 wk end-to-end	< 5 min

Rule of thumb. For any RDS → Lakebase migration, split the work into (schema-dump + Debezium-backfill + short-cutover + decommission) phases. The short-cutover window is the only downtime; every other phase runs live. Never attempt a "big-bang" pg_dump + pg_restore cutover on a live database — the downtime is not worth it when the CDC-based approach exists.

Worked example — Neon-to-Lakebase native path

Detailed explanation. Neon customers get the cleanest migration path — same engine family, same storage architecture, same branching primitive. Databricks provides a native import that maps a Neon project into a Lakebase instance without a full data copy. Walk through the mechanics.

• Neon source. Existing Neon project with main branch and 3-5 dev branches.
• Lakebase target. New Lakebase instance in a Databricks workspace with Unity Catalog binding.
• Mechanics. Databricks CLI import command; Neon project is mounted; branches are recreated.
• Cost. ~2 days including validation.

Question. Run the native import and validate branch parity.

Input.

Component	Value
Neon source	project abc-123 with main + 3 dev branches
Target workspace	Databricks workspace X
Unity Catalog binding	my_org.app_from_neon
Extensions	pgvector, uuid-ossp (both Lakebase-supported)

Code.

# 1. Native import — Neon project → Lakebase instance
databricks lakebase import-neon \
    --neon-api-key "$NEON_API_KEY" \
    --neon-project-id abc-123 \
    --workspace-instance-name migrated-from-neon \
    --unity-catalog my_org.app_from_neon \
    --branches main,dev-alice,dev-bob,dev-carol

# 2. Verify branch parity
databricks lakebase branch list --instance migrated-from-neon
# Expected:
#   NAME          PARENT   CREATED_AT
#   main          (root)   ...
#   dev-alice     main     ...
#   dev-bob       main     ...
#   dev-carol     main     ...

# 3. Row-count sanity — main branch of Lakebase vs Neon main
for table in orders customers user_features; do
    NEON_COUNT=$(psql "$NEON_MAIN_DSN" -tAc "SELECT count(*) FROM public.$table")
    LB_COUNT=$(psql "$LAKEBASE_MAIN_DSN" -tAc "SELECT count(*) FROM public.$table")
    echo "$table: neon=$NEON_COUNT lakebase=$LB_COUNT"
done

# 4. Bind masking policies on the Lakebase side (Unity Catalog)
psql "$LAKEBASE_ADMIN_DSN" <<'SQL'
    -- Rewrite Neon RBAC into Unity Catalog policies (manual step)
    -- ... GRANTs, MASKs, RLS policies as defined in your workspace ...
SQL

Step-by-step explanation.

1. The import-neon command is the native path. It talks to Neon's API, mounts the project's storage into the Databricks-managed storage layer, and creates a Lakebase instance whose branch tree mirrors Neon's. Because the storage engine is compatible, no full data copy is needed.
2. Branch parity is the first check. databricks lakebase branch list should show every Neon branch. In practice, only branches you explicitly name via --branches are imported; wildcard is not supported (as a safety measure).
3. Row counts on main should match Neon exactly. If they don't, the import failed silently on one of the tables; re-run with a scoped --tables flag to import just the missing ones. In practice, the native path is reliable at row-count parity because it's not doing a logical copy.
4. Neon RBAC / masking policies do NOT auto-translate to Unity Catalog. You must rewrite them explicitly on the Lakebase side. This is the biggest non-cleanup work of the migration and is where most teams spend their 2 days.
5. Once validated, cut over apps by DSN swap. Because Neon remains available, rollback is trivial for at least the first soak period.

Output.

Step	Elapsed	Notes
Import command	15-60 min	scales with branch count
Branch parity check	seconds	metadata check
Row-count validation	minutes	one query per table
RBAC / masking rewrite	hours	manual, per-policy
Cutover	minutes	DSN swap
Total	~2 days	mostly RBAC work

Rule of thumb. For any Neon → Lakebase migration, use the native import path (not pg_dump), plan for RBAC / masking rewrite as the bulk of the work, and keep Neon in read-only mode for a soak period before decommissioning. The mechanical migration is fast; the governance-translation work is the real cost.

Worked example — Snowflake Unistore migration with schema translation

Detailed explanation. The hardest migration: Snowflake Unistore Hybrid Tables are not Postgres wire, so applications must be rewritten. Data migration goes through S3 as an intermediate; RBAC translation from Horizon to Unity Catalog is manual. Walk through the phased plan.

• Phase 1. Schema translation — Unistore SQL DDL to Postgres DDL.
• Phase 2. Data migration — COPY INTO from Snowflake to S3, then MERGE from Delta staging into Lakebase.
• Phase 3. Application code rewrite — Snowflake connectors to psycopg2 / pgx.
• Phase 4. RBAC translation — Horizon policies to Unity Catalog.
• Phase 5. Cutover with a read-only soak on Snowflake.

Question. Show the schema-translation delta for one table and the data-migration pipeline.

Input.

Component	Value
Source	Snowflake Unistore Hybrid Table
Target	Lakebase Postgres
Schema differences	VARIANT → JSONB, VARIANT paths translate
Data pipeline	Snowflake COPY INTO → S3 → Delta → Lakebase MERGE

Code.

-- 1. Snowflake source DDL (Unistore Hybrid Table)
CREATE HYBRID TABLE crm.customers (
    customer_id     NUMBER      NOT NULL PRIMARY KEY,
    email           VARCHAR     NOT NULL,
    profile         VARIANT     NOT NULL,          -- semi-structured
    created_at      TIMESTAMP   NOT NULL,
    updated_at      TIMESTAMP   NOT NULL
);

-- 2. Translated Lakebase DDL
CREATE TABLE crm.customers (
    customer_id     BIGINT       PRIMARY KEY,
    email           TEXT         NOT NULL,
    profile         JSONB        NOT NULL,          -- Postgres native semi-structured
    created_at      TIMESTAMPTZ  NOT NULL,
    updated_at      TIMESTAMPTZ  NOT NULL DEFAULT clock_timestamp()
);

CREATE INDEX idx_customers_email ON crm.customers (email);
-- Optional GIN index on JSONB for path-based queries
CREATE INDEX idx_customers_profile_gin ON crm.customers USING GIN (profile);

-- 3. Snowflake side — export to S3 as Parquet
COPY INTO @s3_export/customers_20260719/
FROM   crm.customers
FILE_FORMAT = (TYPE = PARQUET)
HEADER = TRUE;

# 4. Databricks notebook — S3 Parquet → Delta staging
spark.read.parquet("s3://.../customers_20260719/") \
     .write.mode("overwrite") \
     .saveAsTable("main.staging.snowflake_customers")

# 5. MERGE into Lakebase via declared sync
# (illustrative CLI)
# databricks lakebase sync create \
#     --name migrate-customers \
#     --source delta://main.staging.snowflake_customers \
#     --target lakebase://prod-crm/crm.customers \
#     --mode merge --primary-key customer_id --trigger on-demand

# 6. Application code — before (Snowflake connector)
# import snowflake.connector
# conn = snowflake.connector.connect(...)
# cur.execute("SELECT profile:preferences.language FROM crm.customers WHERE customer_id = %s", (cid,))

# After (Postgres wire — psycopg2 with JSONB path)
import psycopg2
conn = psycopg2.connect(host="prod-crm.lakebase.us-east-1.databricks.com", ...)

with conn.cursor() as cur:
    cur.execute("""
        SELECT profile->'preferences'->>'language'
        FROM   crm.customers
        WHERE  customer_id = %s
    """, (cid,))
    row = cur.fetchone()

Step-by-step explanation.

1. Schema translation is mostly mechanical but has traps. VARIANT maps to JSONB (Postgres native semi-structured type). NUMBER typically maps to BIGINT or NUMERIC depending on precision; VARCHAR maps to TEXT. TIMESTAMP maps to TIMESTAMPTZ if the Unistore column carried timezone.
2. Snowflake-specific SQL doesn't survive the translation. profile:preferences.language (Snowflake VARIANT path syntax) becomes profile->'preferences'->>'language' (Postgres JSONB operators). Every application query that touches semi-structured columns must be rewritten.
3. Data migration goes through S3 as an intermediate because Snowflake and Databricks share object-store as the lingua franca. Snowflake COPY INTO @s3_export writes Parquet; Databricks reads Parquet into Delta staging; Lakebase sync MERGEs into the target.
4. Application code changes are the biggest single-line-of-effort item. Every Snowflake connector call → psycopg2 call; every VARIANT path → JSONB path; every Snowflake-specific function (OBJECT_CONSTRUCT, PARSE_JSON) → Postgres equivalent (jsonb_build_object, to_jsonb).
5. RBAC translation from Horizon (Snowflake's governance) to Unity Catalog is manual and detail-heavy. Every ROLE, every GRANT, every masking policy must be re-expressed in Unity Catalog syntax. Budget hours per role; total time is often the migration's critical path.

Output.

Migration cost	Value
Schema translation	1-2 days per 30-table module
Data migration	proportional to volume
Application rewrite	proportional to Snowflake-connector footprint
RBAC / masking rewrite	1-3 days per major role
Cutover window	minutes
Total	~3 wk end-to-end for a moderate deployment

Rule of thumb. For any Snowflake Unistore → Lakebase migration, plan for the schema translation to be mechanical, the data migration to be routine, and the application-rewrite plus RBAC-translation to be the real cost. Do not underestimate the application rewrite — Snowflake connector code is often deeply embedded in service repos.

Senior interview question on Lakebase migration

A senior interviewer might ask: "You've been asked to lead a migration of a 30-table, 500 GB Postgres database currently on Aurora, along with its accompanying reverse-ETL pipeline from a Delta warehouse, into Lakebase. Walk me through the phased plan, the cutover mechanics, the governance-translation work, and the interview probes you'd expect from Databricks solution architects reviewing the plan."

Solution Using a phased pg_dump + Debezium + declared-sync migration with a 24-hour soak

# Phase 1 — schema-only migration (days 1-2)
pg_dump --schema-only --no-owner --no-privileges \
        "postgres://[e-mail]:5432/production" \
        > schema.sql

# Review + fix extensions against Lakebase compat matrix

# Provision Lakebase target
databricks lakebase instance create \
    --name migrated-orders \
    --region us-east-1 \
    --unity-catalog my_org.app_orders \
    --compute-mode pinned \
    --min-compute-cu 4 \
    --max-compute-cu 32

psql "$LAKEBASE_ADMIN_DSN" -f schema.sql

# Phase 2 — Debezium CDC from Aurora (days 3-4)
name: aurora-migration-cdc
config:
  connector.class: io.debezium.connector.postgresql.PostgresConnector
  database.hostname: aurora-src.internal
  database.port: 5432
  database.dbname: production
  database.user: cdc_reader
  plugin.name: pgoutput
  slot.name: aurora_migration
  publication.name: migration_pub
  snapshot.mode: initial
  topic.prefix: aurora_migration
  tombstones.on.delete: true

# Phase 3 — declared sync — Delta staging → Lakebase (days 3-4)
name: aurora-backfill
config:
  # (Databricks CLI equivalent shown earlier in section)
  # source: delta://main.staging.aurora_migration_orders
  # target: lakebase://migrated-orders/app.orders
  # mode: merge; primary-key: id; trigger: continuous

# Phase 4 — cutover (day 5, < 5 min downtime)
# 4a. Freeze Aurora writes
psql "$AURORA_ADMIN_DSN" -c "REVOKE INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public FROM app_writer;"

# 4b. Drain to zero lag
until [ "$(check_backfill_lag_ms)" -lt 500 ]; do sleep 2; done

# 4c. Rewrite reverse-ETL — replace Fivetran / Hightouch with declared sync
databricks lakebase sync create \
    --name features-refresh \
    --source delta://analytics.gold.user_features \
    --target lakebase://migrated-orders/app.user_features \
    --mode merge --primary-key user_id --trigger on-source-commit

# 4d. Rewrite RBAC + masking in Unity Catalog
# (per policy, as covered in Section 3)

# 4e. Rolling app redeploy — DSN swap
kubectl set env deployment/orders-svc DB_HOST=migrated-orders.lakebase.us-east-1.databricks.com

# 4f. Monitor Lakebase for 24h; then decommission Aurora

Step-by-step trace.

Phase	Days	Downtime	Notes
Schema migration	1-2	0	pg_dump + extension review
Debezium bootstrap	3-4	0	initial snapshot + WAL tail
Reverse-ETL replacement	3-5	0	declared sync replaces Fivetran
RBAC + masking rewrite	3-5	0	manual per-policy work
Cutover	5	3-5 min	DSN swap + drain
24h soak	6	0	monitor Lakebase, keep Aurora warm
Aurora decommission	7	0	after soak passes

After deployment, the recommendation service and the ops app both talk to Lakebase over Postgres wire; the reverse-ETL pipeline that fed features from Delta to Aurora is replaced by a declared sync that reduces freshness from 30 minutes to under 60 seconds; Unity Catalog policies replace the separated Aurora IAM + Databricks workspace policies; per-PR branches replace the shared staging environment; the total cutover downtime was under 5 minutes.

Output:

Metric	Before	After
OLTP store	Aurora Postgres	Lakebase
Feature freshness	30 min	< 60 s
Reverse-ETL vendor	Fivetran	none (bundled sync)
Dev-environment cost	shared staging	O(1) per-PR branches
Governance planes	3 (Aurora IAM + UC + Fivetran)	1 (Unity Catalog)
Cutover downtime	(never done)	3-5 min

Why this works — concept by concept:

• Phased migration — schema first, backfill second, cutover last. Each phase can be validated independently; the total downtime is confined to the minutes when writes are frozen and the app is redeployed.
• Debezium CDC bridge — using Debezium as the RDS → Delta staging bridge rather than pg_dump preserves change-order semantics, captures deletes, and lets you validate the target's data volume against the source's live volume before cutover.
• Declared reverse-ETL replacement — the Fivetran → RDS reverse-ETL job becomes a single Databricks-managed sync. This delivers the freshness win independently of the OLTP-store migration; you often ship the sync replacement first, then migrate the OLTP.
• Unity Catalog RBAC rewrite — the manual work is real but bounded. Budget hours per role, do the translation once, and enjoy one governance plane afterward. Every senior migration plan calls this out as its own phase.
• Cost — phased approach means the whole team can review each artifact separately; downtime is minutes rather than hours; the operational surface shrinks (one governance plane, one sync mechanism, one dev-branch workflow) rather than being papered over. Net O(1) cutover time versus O(data) for a big-bang migration, at a strictly simpler post-migration operational load.

Design
Topic — design
Design problems on database migrations

Practice →

SQL
Topic — sql
SQL problems on schema translation and MERGE patterns

Practice →

---

Cheat sheet — Lakebase recipes

• Which lane is Lakebase for. Reverse-ETL replacement (Delta → Lakebase via declared MERGE sync, replaces Fivetran/Hightouch/Census); real-time feature serving (compute in Delta, MERGE-in to Lakebase, read via Postgres wire); operational dashboards (Delta aggregate → Lakebase, Grafana Postgres data source). The anti-pattern is very-high-QPS OLTP with sub-ms p99.9 SLOs — bid paths, sub-ms trading, 100k+ QPS sustained writes still belong on Aurora / DynamoDB / ScyllaDB. Every senior architect should be able to defend the four-lane map in an interview without pausing.
• Instance provisioning template. databricks lakebase instance create --name <inst> --region <region> --unity-catalog <uc.namespace> --compute-mode pinned --min-compute-cu 4 --max-compute-cu 32 --pitr-retention-hours 168. Pinned compute for production (no cold-start in request path), 4-32 compute units for autoscale headroom, 7-day PITR window for incident forensics. Never provision production with --compute-mode serverless unless the workload is genuinely burst-y and can tolerate 1-5s cold-starts on the first hit.
• Branch-per-environment DDL. main = production, pinned; staging = nightly-reset branch of main; pr-{n} = ephemeral per-PR branches with 15-minute idle timeout. Bash template: databricks lakebase branch create --instance <inst> --branch pr-${PR_NUMBER} --parent main --compute-mode serverless --idle-timeout-seconds 900. CI creates on PR open, deletes on PR close; total cost per PR is measured in cents.
• CDC-out (Lakebase → Delta) template. databricks lakebase sync create --name <sync> --source lakebase://<inst>/<schema>.<table> --target delta://<catalog>.<schema>.<table>_cdc --mode incremental --lag-target-ms 60000 --include-before-image true. Bronze CDC table is append-only with _change_type, _commit_ts, _lsn, and _before struct; silver-layer MERGE dedupes on row_number() OVER (PARTITION BY pk ORDER BY _lsn DESC).
• MERGE-in (Delta → Lakebase) reverse-ETL template. databricks lakebase sync create --name <sync> --source delta://<catalog>.<schema>.<table> --target lakebase://<inst>/<schema>.<table> --mode merge --primary-key <pk> --trigger on-source-commit. Trigger on source commit for freshness; use --trigger schedule --cron for time-bounded refresh; use --schema-compat-mode backward in production to fail loudly on incompatible schema changes.
• PgBouncer sidecar config. pool_mode = transaction, default_pool_size = 20, server_lifetime = 3600, server_idle_timeout = 600, max_client_conn = 200, query_wait_timeout = 30. Deploy as a sidecar container next to every app pod; app connects to 127.0.0.1:6432; PgBouncer opens TLS upstream to Lakebase. Audit app code for SET LOCAL, prepared statements, and cursors that cross transactions — transaction-mode pooling breaks all three.
• Unity Catalog masking policy template. CREATE FUNCTION my_org.security.mask_email(email STRING) RETURN CASE WHEN is_account_group_member('pii_readers') THEN email ELSE regexp_replace(email, '(.).*(@.*)', '\1***\2') END; then ALTER TABLE <table> ALTER COLUMN email SET MASK my_org.security.mask_email;. One definition covers both Delta scans and Lakebase reads; never define a parallel Postgres VIEW for masking on the Lakebase side.
• PITR forensic-branch runbook. databricks lakebase branch create --instance <inst> --branch forensic-YYYYMMDD-HHMM --parent main --parent-timestamp "YYYY-MM-DDTHH:MM:SSZ" --compute-mode serverless --read-only. Attach with psql / notebook; diff against production via dblink or cross-branch query; generate repair script from the diff; apply via guarded UPDATE during a change window; delete the forensic branch. Total incident-to-repair time drops from hours to minutes.
• Lakebase vs Unistore vs Neon decision matrix. Lakebase = Postgres-wire + Unity Catalog + branching + serverless + bundled with Databricks. Unistore = Snowflake row-store + Horizon + Snowflake wire + Snowflake-bundled. Neon = Postgres-wire + branching + serverless + standalone (no lakehouse integration). Pick Lakebase if you run Databricks and need OLTP; pick Unistore if you're a pure Snowflake shop; pick Neon if you want branching without a lakehouse commitment. Never pick the OLTP-in-lakehouse product from a vendor whose lakehouse you don't use.
• Migration cost per source. RDS / Aurora Postgres → Lakebase: ~2 engineer-weeks (schema + Debezium backfill + short cutover). Neon → Lakebase: ~2 days (native path + RBAC rewrite). Snowflake Unistore → Lakebase: ~3 engineer-weeks (schema translate + data migration through S3 + app code rewrite + Horizon-to-UC RBAC translation). Every migration is a bounded, plan-able project; the biggest risks are extension compatibility (RDS), branch-history preservation (Neon), and application-code footprint (Unistore).
• Extension compatibility check. Before any migration, list SELECT extname, extversion FROM pg_extension; on the source and cross-reference against Lakebase's supported-extension matrix. Common blockers: PostGIS versions, custom C extensions, plr / plpython3u, proprietary RDS extensions. Missing extensions must be resolved before schema-apply, not during cutover.
• Cost mental model for branches. Creating a branch is O(1) — bytes and milliseconds. Holding a branch costs the divergent-page bytes since fork, NOT the full source-table size. A 1TB source with a lightly-written dev branch might cost single-digit dollars per month; a heavily-written branch tens. Compare this to RDS multi-AZ + storage-per-clone at hundreds of dollars per month per dev environment.
• Failure-mode cheat sheet. Cold-start on scale-to-zero → 1-5s first hit; mitigate with pinned compute for production. Connection-pool churn → PgBouncer sidecar in transaction mode. Branch-lag on fresh forks → read-your-writes is defined-timing not instantaneous. Sync backpressure on large MERGE-in → sync engine throttles the source; alert on staleness > 2x refresh_interval. WAL retention on backfill → set --lag-target-ms conservatively; bigger backfills cost more storage-day.
• What to say in the first minute of a Lakebase interview. "Databricks Lakebase is a Postgres-compatible, serverless OLTP layer inside the lakehouse, built on the Neon acquisition and announced at Data + AI Summit 2025. It shares Unity Catalog governance with Delta, syncs bidirectionally, and replaces reverse-ETL for the three lanes it fits — segment / feature serving / ops dashboards — while leaving very-high-QPS bid paths on specialised OLTP. The migration mechanics differ per source: pg_dump plus Delta CDC for RDS, native path for Neon, schema translate plus RBAC rewrite for Snowflake Unistore." That's the entire senior framing in one paragraph.

Frequently asked questions

What is Databricks Lakebase in one sentence?

databricks lakebase is Databricks' Postgres-compatible, serverless OLTP layer built on the Neon fork acquired in mid-2025 and announced at Data + AI Summit 2025 — it lives inside the Databricks workspace as a peer to Delta tables under one Unity Catalog namespace, syncs bidirectionally with Delta so that operational reads (single-row lookups, joins on a primary key) and analytical reads (aggregate scans over hundreds of millions of rows) both happen against the same governed data, and eliminates the reverse-ETL glue jobs (Fivetran, Hightouch, Census) plus the specialised serving stores (Redis, DynamoDB in front of the warehouse) that have accreted around Delta-and-Snowflake deployments for the last decade. The mental model is "real Postgres inside the lakehouse" — same wire protocol, same MVCC, same index behaviour, plus the copy-on-write branching primitive Neon made famous and Unity Catalog governance that spans both OLTP and OLAP by construction. Every senior interviewer probes Lakebase because it changes the answer to "how does OLTP data reach the warehouse and how do computed features reach the operational app?" from "an Airflow DAG plus Fivetran plus a masking policy in each place" to "one declared bidirectional sync plus one Unity Catalog policy plane."

Lakebase vs Snowflake Unistore — which do I pick?

Pick databricks lakebase if your team already runs the Databricks lakehouse under Unity Catalog and needs a Postgres-wire OLTP layer with copy-on-write branching for dev / staging environments and one governance plane spanning OLTP and OLAP. Pick Snowflake Unistore Hybrid Tables if your team is a pure Snowflake shop with Horizon as the governance plane and you're willing to accept the Snowflake wire protocol (not Postgres) for OLTP access. The two products solve the same category — OLTP inside the lakehouse — but the wire protocol, governance plane, and branching-vs-clone story are all different. Lakebase's wire compatibility means every existing psycopg2 / JDBC / pgx client works with only a DSN swap; Unistore requires Snowflake connectors which are less ubiquitous. Lakebase's O(1) copy-on-write branching costs bytes and milliseconds; Unistore has no equivalent primitive and requires full clones. On the other hand, Snowflake Unistore has been generally available for a year longer than Lakebase, has a more mature ecosystem, and is the pragmatic choice for teams whose entire data stack is Snowflake. Never pick the OLTP-in-lakehouse product from a vendor whose lakehouse you don't run.

Is Lakebase the same as Neon?

Lakebase is built on the Neon fork Databricks acquired in mid-2025, but it is not the same product. Neon continues to exist as a standalone Postgres service (on AWS, Azure, GCP) for teams that want serverless Postgres + copy-on-write branching without committing to the Databricks workspace. Lakebase is the Databricks-workspace-integrated evolution of that engine: same underlying serverless-Postgres + shared-page-store architecture, same branching primitive, plus Unity Catalog integration, bidirectional Delta sync, and Databricks workspace management. The migration path from Neon to Lakebase is the cleanest of the three canonical sources (typically ~2 days) because the storage engine is compatible; the only significant translation work is RBAC / masking (Neon RBAC does NOT auto-translate to Unity Catalog policies). If you already run Neon standalone and are moving to Databricks, use the databricks lakebase import-neon native path; if you're deciding between them for a greenfield deployment, pick Neon if you don't need lakehouse integration and Lakebase if you do.

Can Lakebase replace my RDS Postgres?

Yes for most workloads, with the anti-pattern caveat. For the three winning lanes — reverse-ETL target, real-time feature serving, operational dashboards over warehouse data — Lakebase is a strict improvement over RDS: Postgres wire-compatible (zero application-code migration cost beyond the DSN swap), Unity Catalog governance parity with Delta, O(1) branch-per-environment workflow, bundled bidirectional sync with Delta. Migration cost is roughly 2 engineer-weeks for a moderate schema (30 tables, 500 GB): pg_dump --schema-only + Debezium CDC backfill from RDS into Delta + declared MERGE sync from Delta into Lakebase + a short (3-5 minute) cutover window. The exception is very-high-QPS OLTP with sub-millisecond p99.9 SLOs — ad-tech real-time bidding, sub-ms trading, 100k+ QPS sustained writes — where the serverless-Postgres cold-start tail and connection-pool churn are still measurable disadvantages against a locally-provisioned Aurora / DynamoDB / ScyllaDB deployment. For those workloads, keep the specialised OLTP and integrate to the lakehouse via traditional log-based CDC (Debezium against Aurora, DynamoDB Streams → Kinesis → Delta). The senior signal is knowing when Lakebase is NOT the right tool — every technology has a workload envelope.

How does Unity Catalog govern Lakebase?

Unity Catalog governs Lakebase as a first-class asset alongside Delta tables, with the same GRANT / REVOKE syntax, the same masking policies, the same row-level-security policies, the same lineage tracking, and the same audit log. The critical property is that a single policy definition — say, CREATE FUNCTION mask_email(email) RETURN CASE WHEN is_account_group_member('pii_readers') THEN email ELSE regexp_replace(email, '(.).*(@.*)', '\1***\2') END bound via ALTER TABLE customers ALTER COLUMN email SET MASK mask_email — applies identically to a SELECT email FROM customers issued from a Databricks notebook against Delta and to the same SELECT issued from a psycopg2 connection against Lakebase. This closes the entire class of "the policy is defined in Snowflake but we forgot to define it in Postgres too" governance drift bugs. Practically: define GRANTs and MASKs at the Unity Catalog level in the workspace UI or via SQL, then let the underlying engines (Delta scan planner, Lakebase Postgres query layer) enforce them at query time. Application code never needs to know about masking; the mask is transparent at the wire level, returning the redacted value directly. The load-bearing benefit is not just correctness — it's the audit-checkbox work: proving a policy applies to both worlds is one Unity Catalog artifact to point at, not two-that-hopefully-still-match.

When should I NOT use Lakebase?

Do NOT use databricks lakebase for very-high-QPS OLTP bid-path workloads — ad-tech real-time bidding at 100k+ QPS with sub-millisecond p99.9 SLOs, high-frequency trading pre-trade paths, chat message fan-out where sub-ms latency is a business requirement, or sub-ms leaderboard writes for competitive gaming. The reasons are the serverless-Postgres cold-start tail (1-5 seconds on the first connection after scale-to-zero), the connection-pool churn that transaction-mode PgBouncer can mitigate but not eliminate at extreme scale, and the network hops inherent in the shared-page-store architecture. A dedicated Aurora Postgres provisioned cluster, DynamoDB on-demand, or ScyllaDB self-hosted deployment will beat Lakebase by tens of milliseconds at the tail for these workloads at strictly lower cost per QPS. Also avoid Lakebase for workloads that require RDS-only Postgres extensions Lakebase doesn't support (check the compatibility matrix before committing), for teams that don't run the Databricks lakehouse (Unity Catalog integration is Lakebase's differentiator; without it, Neon standalone or vanilla Postgres wins on operational simplicity), and for workloads where the governance parity benefit is zero (a service that touches nothing else in the lakehouse doesn't gain from Unity Catalog policy convergence). The senior signal in a Lakebase interview is naming this anti-pattern unprompted — every technology has an envelope, and knowing yours defends every adoption decision.

Practice on PipeCode

• Drill the SQL practice library → for the Postgres-wire, JSONB, MERGE, and MVCC problems that Lakebase workloads live and die on.
• Rehearse on the design practice library → for the lakehouse, feature-serving, reverse-ETL replacement, and OLTP-vs-OLAP topology questions senior interviewers open with when Lakebase is on the table.
• Sharpen the streaming axis with the streaming practice library → for the bidirectional CDC-out and MERGE-in sync patterns that make Lakebase-plus-Delta a real bidirectional pipeline rather than a one-way ETL.
• Layer in joins and aggregation drills to cement the SQL-over-Postgres muscle memory Lakebase serving paths depend on — most Lakebase serving queries are indexed single-row joins or small aggregates.
• Practice the window-functions catalogue to master the row_number() OVER (PARTITION BY pk ORDER BY _lsn DESC) dedupe pattern that turns bronze CDC events into silver current-state tables — this is the load-bearing SQL primitive of every Lakebase → Delta pipeline.
• Stack the prerequisites against PipeCode's broader 450+ data-engineering catalogue to anchor the five-axis decision matrix (wire protocol, governance, analytical sync, cost model, QPS envelope) against real graded inputs.

Lock in databricks lakebase muscle memory

Docs explain products. PipeCode drills explain the decision — when Lakebase wins the reverse-ETL lane, when the branching primitive changes dev-environment economics, when the bidirectional Delta sync closes the training-serving parity gap, when the very-high-QPS bid path still belongs on a specialised OLTP. Pipecode.ai is Leetcode for Data Engineering — pattern-first practice tuned for the production trade-offs senior data engineers actually face across OLTP and OLAP in the modern lakehouse.

Practice SQL problems →
Practice streaming problems →