Data Observability Platforms Compared: Monte Carlo, Anomalo, Bigeye & Lightup

data observability platforms is the line item on every senior data engineer's 2026 platform roadmap that most teams underestimate until the first painful data downtime incident lands on a Tuesday morning and an executive dashboard renders a blank. The four vendors that dominate this market — Monte Carlo, Anomalo, Bigeye, and Lightup — look superficially similar in their marketing decks (all four claim "the five pillars," all four claim "ML-powered detection," all four claim "field-level lineage"), but they make very different bets on the detection model, the lineage depth, and the operating shape of the team that runs them.

This guide is the senior-DE platform-selection write-up you wished existed the first time a director asked "do we go with monte carlo data or anomalo for our warehouse?" or "is bigeye enough, or do we need lightup for the SLO framing?" It walks through what data observability actually means as a category (testing asserts; observability surfaces), what each of the four data observability tools is genuinely good at, how data quality monitoring differs from data downtime reduction, why the data slo framing changed the conversation in 2024–2026, and the 5-question decision tree senior platform engineers use to pick the right vendor (or stitch two together). Each section pairs a teaching block with a Solution-Tail interview answer — code, a step-by-step trace, an output table, then a concept-by-concept breakdown of why it works.

When you want hands-on reps immediately after reading, drill the ETL practice library →, rehearse on medium-difficulty ETL problems →, and sharpen the warehouse axis with the SQL practice library →.

---

On this page

• Why data observability is its own market in 2026
• Monte Carlo deep-dive
• Anomalo deep-dive
• Bigeye + Lightup
• Picking the platform — decision matrix
• Cheat sheet — observability picker
• Frequently asked questions
• Practice on PipeCode

---

1. Why data observability is its own market in 2026

Data observability is the runtime-monitoring discipline that data teams stole from SRE — testing asserts what you knew to check, observability surfaces what you didn't

The one-sentence invariant: data testing asserts the contract you already wrote (this column is non-null; row count > 0); data observability surfaces the contract you forgot to write (this column's distribution shifted 4 sigma; this pipeline is 3 hours late; this column's freshness curve broke its weekly seasonality). Once you internalise that distinction, the entire data observability market story falls out: testing is a build-time discipline owned by the analytics engineer; observability is a runtime discipline owned by the platform team, and it has its own tooling, its own metrics (MTTI, MTTR, MTBF), and its own on-call rota.

The five pillars — the canonical taxonomy every vendor agrees on.

• Freshness. When was this table last updated? Is the update on its expected cadence? Freshness breaks are the single most common observability incident — an upstream pipeline misses its run, and every dashboard reading from the table renders stale data without any null or schema signal to catch it.
• Volume. How many rows are in the table today vs the rolling baseline? Volume spikes (10x normal) and volume drops (50% lower than yesterday) are both signal — a spike often means a duplication bug upstream; a drop often means a filter regression or an ingestion failure.
• Distribution. What is the shape of each column? Mean, p50, p95, null rate, distinct count, top-K cardinality. A column whose null rate jumps from 0.1% to 8% is a silent contract break that no row-count check would surface.
• Schema. What columns exist, in what order, with what types? Schema drift is the second-most-common incident — an upstream team adds a column, drops a column, or changes a string to a varchar(64), and the downstream pipeline either crashes loudly or silently mis-parses.
• Lineage. What feeds this table, and what reads from it? Lineage is the dependency graph that lets you answer the impact-radius question — "if raw_orders is broken, which dashboards, ML features, and customer-facing surfaces are affected?" Without lineage, every incident is a manual hunt through dbt models, BI metadata, and tribal knowledge.

The four "must-answer" axes interviewers actually probe.

• Detection model. Auto-monitor (vendor decides what to watch — Monte Carlo), ML-first (no thresholds — Anomalo), threshold + autometric (Bigeye), or SLO + check schedule (Lightup). The detection model dictates how much config the platform team has to write — and how many false positives the on-call rotation will see.
• No-code vs code. Does the platform let an analyst click "add a freshness check" in the UI, or does every rule live in YAML / SQL / Python in the team's git repo? Vendor positioning here predicts who owns the rules — analysts vs platform engineers.
• Lineage depth. Table-level (this table feeds that table), column-level (this column feeds that column), or field-level (this value transformation feeds that value transformation, with the SQL diff). Field-level lineage is the gold standard but rare; column-level is the 2026 floor.
• Alerting. Slack, PagerDuty, ServiceNow, Microsoft Teams, custom webhook, Jira ticket, on-call schedule integration. Alerting hygiene (right channel, right severity, right runbook link) is the single biggest source of vendor regret in year 2.

The 2026 reality — what changed since 2022.

• Monte Carlo went from "data observability category creator" to a near-IPO incumbent with the deepest lineage product and the largest customer base. It is the safest default for big-team, catalog-adjacent deployments.
• Anomalo doubled down on the ML-first detection model — no thresholds to tune, just point it at a table and it learns the baseline. The "no-code, smart-default" pitch resonates with analytics-engineering teams that don't want a platform engineer on the rota.
• Bigeye sharpened its metric-store positioning — every check is a metric you can also export to Datadog / Grafana / Snowflake. Open APIs, custom-SQL monitors, and the "data quality is just metrics" framing.
• Lightup introduced the data slo vocabulary — every monitor is a service-level objective with an error budget, an MTTI target, and an MTTR target. The framing is SRE-native and resonates with platform teams that already run application SLOs.
• Hybrid stacks (Monte Carlo + Anomalo, Bigeye + Lightup) are now common at large enterprises — one vendor for lineage and incident management, another for ML detection or custom SQL monitors.

Why testing alone is not enough — the MTTR argument.

• Testing runs in CI / dbt-test / Great Expectations and asserts a fixed contract. If the contract was wrong (i.e. you didn't know to check this distribution shift), the test passes while the data is broken.
• Observability runs continuously against the live warehouse, with statistical baselines and ML anomaly detectors that catch the contracts you didn't write. Mean-time-to-incident (MTTI) drops from "the next dashboard refresh" to minutes.
• The 2026 industry benchmark is MTTI under 10 minutes for tier-1 tables and MTTR under 1 hour for the same. Without a real observability platform, both numbers slip into the multi-hour range and exec patience runs out.

What interviewers listen for.

• Do you say "testing asserts, observability surfaces" in the first sentence? — senior signal.
• Do you describe the five pillars unprompted, in order? — required answer.
• Do you mention MTTI and MTTR as the operating metrics, not the number of checks? — senior signal.
• Do you push back on "isn't dbt test enough?" with "dbt tests are a build-time contract; observability is the runtime safety net"? — senior signal.

Worked example — testing vs observability on the same column

Detailed explanation. A users table has an email_domain column. The analytics engineer wrote a dbt test asserting email_domain is non-null. Testing covers the contract she knew to write. Observability covers the contract she didn't — a sudden spike in gmail.com from 35% to 78% because a marketing campaign was retargeted, or a sudden appearance of a new top-K value example.com from a synthetic-data leak. Same column, two complementary disciplines.

Question. Show a dbt test on email_domain and the observability checks an observability platform would add. Highlight which checks each discipline owns.

Input.

column	row count	nulls	top-K (today)	top-K (rolling 7d)
email_domain	4.2M	0	gmail 78%, yahoo 9%, example.com 4%	gmail 35%, yahoo 22%, hotmail 12%

Code.

-- dbt test (testing — build-time contract on `users.email_domain`)
-- file: tests/users_email_domain.yml
version: 2
models:
  - name: users
    columns:
      - name: email_domain
        tests:
          - not_null
          - accepted_values:
              values: ['gmail.com', 'yahoo.com', 'hotmail.com', 'outlook.com', 'icloud.com']
              config:
                severity: warn

# Observability monitor (runtime — distribution drift + new value detection)
# file: observability/users_email_domain.yaml
monitor:
  table: prod.dim_users
  column: email_domain
  checks:
    - kind: distribution_drift
      metric: top_k_share
      k: 5
      sensitivity: medium
    - kind: new_categorical_value
      lookback: 7d
      min_share_to_alert: 0.01
    - kind: null_rate
      threshold: 0.5%
      compare_to: rolling_7d

Step-by-step explanation.

1. The dbt test asserts the contract the analytics engineer wrote in the past: email_domain is non-null and belongs to a closed set of five values. If gmail.com becomes 78%, the dbt test still passes — the contract was about presence, not distribution.
2. The observability monitor watches the same column from a different angle. The distribution_drift check fires when top_k_share shifts by more than the statistical baseline allows.
3. The new_categorical_value check catches example.com — a value not seen in the rolling 7-day window. dbt's accepted_values would have caught this too, but only because the engineer remembered to list every accepted value (rare in practice).
4. The null_rate check is the runtime version of dbt's not_null — it allows a small null tolerance and alerts on a rate change rather than a binary pass/fail. Useful for noisy upstream sources where nulls are sometimes acceptable.
5. The two disciplines compose: dbt tests block bad data from entering the warehouse on a daily build; observability monitors catch the contracts you didn't think to write, on a continuous schedule.

Output.

Concern	Owned by testing?	Owned by observability?
email_domain IS NOT NULL	yes	yes (as a rate check)
Accepted values list	yes (if you wrote it)	no (catalog drift)
Distribution drift	no	yes
New top-K value	no	yes
Freshness of source table	no	yes
Schema change in upstream	no	yes

Rule of thumb. Use testing for the contracts you can articulate at write-time. Use observability for the contracts that emerge at run-time. Teams that conflate the two end up with either too many flaky tests or too many silent failures — never the right balance.

Worked example — the five pillars on a single fact table

Detailed explanation. A fact_orders table is the most-read object in the warehouse — every revenue dashboard, every cohort report, every ML feature pulls from it. Designing observability for it means instrumenting all five pillars with the right cadence and the right escalation path. The exercise shows how the five pillars layer on a single table.

Question. Set up the five pillars (freshness, volume, distribution, schema, lineage) on fact_orders for tier-1 SLA. Show the checks, their cadence, and the alerting channel for each.

Input.

pillar	metric	baseline	tolerance
freshness	minutes since last write	≤ 60 min	warn @ 90, page @ 180
volume	row count delta vs 7d MA	±15%	warn @ ±30%, page @ ±60%
distribution	numerical column drift (KS test)	< 0.10	warn @ 0.15, page @ 0.30
schema	column add / drop / type change	none	page on any change
lineage	upstream tables reachable	7 / 7	warn @ 6, page @ 5

Code.

# Tier-1 observability spec for fact_orders
table: warehouse.prod.fact_orders
tier: 1
owners:
  - [e-mail]
  - [e-mail]

checks:
  freshness:
    schedule: every 10m
    metric: minutes_since_last_write
    warn_threshold: 90
    page_threshold: 180

  volume:
    schedule: every 30m
    metric: row_count_delta_pct
    baseline: rolling_7d_ma
    warn_threshold_pct: 30
    page_threshold_pct: 60

  distribution:
    schedule: hourly
    columns: [order_amount, ship_latency_hours]
    metric: ks_statistic
    baseline: rolling_30d
    warn_threshold: 0.15
    page_threshold: 0.30

  schema:
    schedule: every 5m
    metric: schema_fingerprint_diff
    page_on: [column_added, column_dropped, type_changed]

  lineage:
    schedule: every 15m
    upstream_tables_required: 7
    warn_threshold: 6
    page_threshold: 5

alerting:
  warn_channel: slack://#data-warn
  page_channel: pagerduty://data-platform-rota
  runbook: https://runbooks.example.com/fact-orders

Step-by-step explanation.

1. Freshness is the cheapest check — a single timestamp query on the warehouse — so it runs at the highest frequency (every 10 minutes). 60-minute baseline, 90-minute warn, 180-minute page; the 90 / 180 split gives the on-call rotation a chance to acknowledge before the page escalates.
2. Volume runs every 30 minutes against the rolling 7-day moving average. A ±30% deviation is a Slack warn; ±60% is a page. The thresholds are tuned to the natural daily seasonality — a Sunday morning lull is normal at ±20%, but ±60% is never normal.
3. Distribution checks two numerical columns (order_amount, ship_latency_hours) using a Kolmogorov-Smirnov statistic against the rolling 30-day baseline. The hourly cadence keeps the check cost bounded; the 30-day window smooths over weekly seasonality.
4. Schema drift is paged unconditionally — there is no "warn" tier for a column-add or type-change. A schema diff is a hard contract break, and the on-call engineer needs to acknowledge it before any downstream pipeline runs.
5. Lineage is checked every 15 minutes — the observability platform walks the dependency graph and verifies all 7 upstream tables are reachable and fresh. A missing upstream (6 of 7) is a warn; 5 of 7 is a page.

Output.

pillar	cadence	warn	page
freshness	10m	90 min	180 min
volume	30m	±30%	±60%
distribution	hourly	KS 0.15	KS 0.30
schema	5m	(none)	any change
lineage	15m	6/7 upstream	5/7 upstream

Rule of thumb. Tier-1 tables get all five pillars with tight thresholds and the lowest cadence you can afford. Tier-2 tables drop the distribution check (highest compute cost) and loosen the volume threshold. Tier-3 tables run schema + freshness only.

Worked example — MTTI and MTTR on a freshness incident

Detailed explanation. A common interview question — "your fact_orders table is 4 hours late. How does data observability change the incident timeline?" The answer is the cleanest demonstration of the MTTI / MTTR value proposition. Without observability, the freshness break is discovered when a dashboard renders stale; with observability, it is paged within minutes of the late checkpoint.

Question. Compare the incident timeline for a freshness break in fact_orders with and without a data observability platform. Show MTTI and MTTR for both scenarios.

Input.

event	time without observability	time with observability
Upstream pipeline misses 02:00 run	02:00	02:00
Expected write to fact_orders	02:30	02:30
Detection	next dashboard refresh	continuous freshness check
Acknowledgement	morning standup	on-call page
Root cause identified	mid-morning	within 30 min of page
Pipeline rerun + table caught up	mid-afternoon	within 90 min of page

Code.

# Pseudo-code for the observability runtime
def freshness_check(table: str, threshold_min: int) -> CheckResult:
    last_write_ts = warehouse.query(f"SELECT MAX(updated_at) FROM {table}")
    age_min = (now() - last_write_ts).total_seconds() / 60
    if age_min > threshold_min * 3:
        return CheckResult(severity="page", message=f"{table} is {age_min:.0f} min late")
    if age_min > threshold_min:
        return CheckResult(severity="warn", message=f"{table} is {age_min:.0f} min late")
    return CheckResult(severity="ok")

# Scheduler ticks every 10 minutes, evaluates the check, routes to Slack / PagerDuty

Step-by-step explanation.

1. Without observability, the freshness break is invisible until a downstream consumer notices — usually the next BI dashboard refresh or the morning standup. MTTI is dominated by "how often does someone look at this table?", which is typically 4–8 hours.
2. With observability, the freshness check fires within 10 minutes of the expected write deadline. The on-call engineer is paged immediately. MTTI drops to 10–30 minutes.
3. MTTR is similarly compressed because the page carries a structured incident: which table, what threshold was breached, what the historical baseline looked like, which upstream tables to check, and a runbook link. The on-call doesn't need to triage which dashboard reported stale data; the platform already points at the table.
4. The math at scale: a team with 50 tier-1 tables and 2 incidents per week sees MTTI compressed by ~6 hours per incident → 12 hours per week → 624 hours per year. That is more than one full-time engineer's worth of lost time.
5. The exec patience argument: the difference between "we caught it before the morning standup" and "the CEO's dashboard was stale at the all-hands" is the entire business case for a data observability platform.

Output.

metric	without observability	with observability
MTTI	4–8 h	10–30 min
MTTR	6–10 h	1–2 h
Incidents discovered by consumers	most	few
Incidents discovered by platform	few	most

Rule of thumb. The business case for data observability is MTTI compression, not check coverage. Vendors who ship 200 checks but no incident-management workflow leave teams worse off than vendors who ship 50 checks plus first-class Slack / PagerDuty / runbook integration.

Senior interview question on the observability category in 2026

A senior interviewer often opens with: "How would you justify spending $200K/year on a data observability platform when the team already has dbt tests and Great Expectations? What is the additional value, and how do you measure it?"

Solution Using a four-part business-case framework — coverage gap, MTTI, MTTR, and exec trust

Business case — data observability beyond dbt tests
====================================================

1. Coverage gap
   dbt tests cover the contracts the analytics engineer wrote.
   Observability covers the contracts they didn't write — distribution
   drift, freshness misses, schema changes, new categorical values.
   Audit: count failed observability monitors over 90 days; subtract
   the ones that any dbt test would have caught. The remainder is the
   net new coverage.

2. MTTI compression
   Before: MTTI = "when does the next consumer notice?" (4-8h)
   After:  MTTI = "scheduler tick + alert routing"   (10-30min)
   Audit: log every incident's first-detection timestamp. Compare to
   the timestamp the platform would have fired the check.

3. MTTR compression
   Before: MTTR = MTTI + triage + root-cause + fix (6-10h)
   After:  MTTR = page + structured incident + runbook (1-2h)
   The Slack / PagerDuty page carries: table, threshold, baseline,
   upstream tables, owners, runbook link.

4. Exec trust
   The qualitative win — execs see stale data once, lose trust for a
   quarter. Observability prevents the second incident.
   Audit: exec NPS survey on dashboard trust before/after rollout.

Net business case
=================
  Hours saved   = incidents * (old_MTTR - new_MTTR)
  Eng cost      = hours_saved * loaded_eng_rate
  Trust premium = qualitative, captured in exec NPS
  Compare       = (eng cost + trust premium) vs vendor cost

Step-by-step trace.

Step	Audit data	Calculation
Coverage gap	312 observability fires in 90d	248 not covered by dbt → 80% net new
MTTI before	avg 5.2h across 50 incidents	baseline
MTTI after	avg 22 min across 50 incidents	14.5x compression
MTTR before	avg 8.1h	baseline
MTTR after	avg 1.6h	5x compression
Hours saved / year	50 incidents/q × 4q × 6.5h	1300 eng-hours
Eng cost saved	1300 × $150/h	$195K

After the audit, the $200K vendor cost is approximately break-even on the eng-time line alone, before the trust premium. With the trust premium and the implicit revenue protection (revenue dashboards that don't render stale), the platform pays for itself in year one.

Output:

Dimension	Without observability	With observability
Coverage of "unknown unknowns"	0%	~80%
MTTI	4–8h	10–30 min
MTTR	6–10h	1–2h
Exec trust in dashboards	brittle	sustained
Annual eng-hours saved	—	1000–1500

Why this works — concept by concept:

• Coverage gap audit — counting observability fires that no dbt test would have caught is the cleanest quantitative argument. It puts a number on "what you didn't know to check."
• MTTI = scheduler tick + alert latency — once you frame MTTI as a property of the scheduler, the value of a sub-10-minute check cadence becomes obvious. Manual discovery is unbounded; scheduled discovery is bounded.
• MTTR = page + structured incident — the platform-supplied context (table, baseline, runbook) shortens triage. Without it, the on-call hunts; with it, they execute.
• Trust as a qualitative line item — execs do not buy "fewer false positives." They buy "the next quarterly review will not be derailed by a stale dashboard." Observability is the insurance policy for that.
• Cost — vendor cost is fixed; eng-hours saved scale with incident volume. At 50 incidents/quarter the platform pays for itself; below 10 it is harder to justify. Tier-1 table count is the proxy variable.

ETL
Topic — etl
ETL observability problems

Practice →

ETL Topic — etl · medium Medium ETL incident-response drills

Practice →

---

2. Monte Carlo deep-dive

monte carlo data is the category-defining incumbent — auto-monitors plus field-level lineage plus impact-radius queries are the bundle that justified the premium

The mental model in one line: Monte Carlo runs out-of-the-box monitors on four of the five pillars (freshness, volume, schema, distribution) the moment you connect a warehouse, layers field-level lineage on top so any incident can be traced to its impact radius, and treats incidents as a first-class object with status flags, ownership, and integrations into Slack / PagerDuty / Jira. Once you say "auto-monitor by default plus impact-radius by lineage," every monte carlo data interview question collapses to a deduction from that platform shape.

Auto-monitors — the "zero config" promise.

• Freshness auto-monitor. Monte Carlo's agent inspects warehouse metadata to learn the historical update cadence of each table. After ~7 days of observation, it sets an internal SLA on freshness without any configuration; freshness misses against that learned cadence become incidents.
• Volume auto-monitor. Same observation-and-learn pattern for row count over time. Daily seasonality, weekly seasonality, and trend are decomposed automatically; deviations beyond a configurable sensitivity become incidents.
• Schema auto-monitor. Continuous schema-fingerprint diff. Column add, column drop, type change, default change — all surfaced as schema-change incidents with the diff visible in the UI.
• Distribution auto-monitor (opt-in by table). Per-column statistical baselines on numerical and categorical columns; drift detection via standard statistical tests (KS, chi-squared) plus proprietary signals. Higher compute cost, so it is opt-in.

Field-level lineage — the differentiator that justified the premium.

• Column lineage — Monte Carlo parses every dbt model, every Looker / Tableau / Power BI metadata feed, every query log, and stitches them into a column-to-column dependency graph. A column in fact_orders knows which dashboards read it and which ML features depend on it.
• Field-level lineage — a deeper layer that tracks the transformation on each column (which CASE WHEN, which aggregation, which join condition produced the value). Field-level is the gold standard and was Monte Carlo's flagship feature in 2023–2024.
• Impact-radius queries — at incident time, the platform answers "if this column is broken, which downstream surfaces are affected?" in a single click. The query crawls the lineage graph and surfaces every dashboard / ML feature / customer-facing API that touches the broken column.

Incidents — first-class object, not a Slack message.

• Every fired monitor becomes an incident with a numeric severity, an owner, a status flag (new, acknowledged, in-progress, resolved, false_positive), and a thread of activity (Slack replies, root-cause notes, PR links).
• Incidents are routed to on-call rotations integrated with PagerDuty / Opsgenie / VictorOps. The runbook URL surfaces in the page.
• Incidents are tagged with impact (computed from lineage) — "this incident affects 12 dashboards, 4 ML features, 1 customer API" — so triage knows the blast radius before the on-call digs.

Key assets + field health — the dashboards.

• Key assets is a portfolio view of the tables that matter most. Each asset has a health score (rolled up from active incidents), an owner, and a freshness curve.
• Field health is the per-column equivalent — null rate, distinct count, top-K, mean / median for numerical columns — visualised on a timeline. Drift jumps off the page visually.

Pricing model + warehouse-native scanning.

• Per-table or per-asset pricing tier, with volume discounts for portfolios > 1000 tables. Enterprise contracts negotiate the unit.
• Warehouse-native scanning — Monte Carlo runs its checks inside your Snowflake / BigQuery / Databricks warehouse, using warehouse compute. No data ever leaves your environment; the platform sees only metadata and aggregated metrics.
• The "no data egress" architecture is the answer to every CISO question and is a material reason Monte Carlo wins large-enterprise deals.

Common interview probes on Monte Carlo.

• "What does Monte Carlo cover out of the box vs what needs configuration?" — freshness, volume, schema by default; distribution is opt-in per table; custom SQL monitors and field-health views are configured.
• "How does field-level lineage work?" — query-log parsing + dbt manifest + BI metadata, stitched into a column-to-column dependency graph.
• "What's an impact-radius query?" — lineage walk at incident time that surfaces every downstream surface affected by a broken column.
• "Where does Monte Carlo run?" — inside your warehouse; the data never egresses.

Worked example — auto-monitor detects a 2 AM schema drift

Detailed explanation. A backend team deploys a service change at 1:53 AM UTC. The change adds a loyalty_tier column to the orders table in Postgres. Fivetran picks it up on the next sync at 2:00 AM and propagates the new column into raw_orders in Snowflake. Monte Carlo's schema auto-monitor fires within 5 minutes.

Question. Trace the incident from the upstream schema change through Monte Carlo's auto-monitor to the on-call page. Show the incident object and the impact-radius query result.

Input.

time	event
01:53 UTC	backend deploys ALTER TABLE orders ADD COLUMN loyalty_tier VARCHAR
02:00 UTC	Fivetran sync runs
02:02 UTC	raw_orders in Snowflake has new column
02:05 UTC	Monte Carlo schema check ticks
02:05 UTC	schema fingerprint diff detected

Code.

# Pseudo-code for Monte Carlo's schema auto-monitor
def schema_fingerprint(table: str) -> str:
    cols = warehouse.query(f"""
        SELECT column_name, data_type
          FROM information_schema.columns
         WHERE table_schema || '.' || table_name = '{table}'
         ORDER BY ordinal_position
    """)
    return hashlib.sha256(json.dumps(cols).encode()).hexdigest()

def schema_check(table: str, previous: str) -> Incident | None:
    current = schema_fingerprint(table)
    if current == previous:
        return None
    diff = compute_schema_diff(table, previous_columns(table), current_columns(table))
    return Incident(
        kind="schema_change",
        table=table,
        severity="page",
        diff=diff,
        impact_radius=lineage_walk_downstream(table, diff.changed_columns),
    )

Step-by-step explanation.

1. The schema auto-monitor runs every 5 minutes against information_schema.columns. It computes a SHA-256 fingerprint over (column_name, data_type) tuples in ordinal-position order.
2. At 02:05 UTC, the fingerprint diff is non-empty — a new column loyalty_tier VARCHAR was added. The change is classified as column_added, severity page per the platform default.
3. The incident is created with the affected table, the diff, and an automatic impact-radius query that walks downstream lineage. The query finds 3 dbt models, 2 dashboards, and 1 ML feature reference the affected table.
4. The incident routes to the on-call channel in Slack and the PagerDuty rotation owning raw_orders. The on-call sees the diff, the impact list, and a runbook link — no triage hunt required.
5. The on-call acknowledges, opens a quick PR to add loyalty_tier to the dbt model and re-run, marks the incident resolved once the downstream models are caught up. Total wall-clock time from schema change to resolution: ~25 minutes vs the legacy ~6 hours of "wait for the morning standup."

Output.

field	value
incident.id	INC-04827
incident.kind	schema_change
incident.severity	page
incident.table	warehouse.raw.raw_orders
incident.diff	+ column loyalty_tier (VARCHAR)
incident.impact_radius.dbt_models	[stg_orders, fct_orders, mart_loyalty]
incident.impact_radius.dashboards	[Revenue Daily, CRO Weekly]
incident.impact_radius.ml_features	[loyalty_score_v2]
incident.status	new → acknowledged → resolved

Rule of thumb. Schema is the highest-signal auto-monitor — schema changes are objective (column added vs not), so the false-positive rate is near zero, and they almost always require downstream action. Set schema auto-monitor to page on every tier-1 table.

Worked example — field-level lineage and impact-radius query

Detailed explanation. A revenue dashboard renders an unexpected 40% drop. The on-call needs to know whether the drop is a real business event or a data incident. Without lineage, the hunt is manual through dbt model files. With Monte Carlo's field-level lineage, the dashboard's total_revenue field is a click away from its underlying column, which is a click away from every upstream transform that produced it.

Question. Use Monte Carlo's field-level lineage to walk from a dashboard metric total_revenue back to the raw warehouse column that produced it. Show the lineage chain and the impact-radius query.

Input.

layer	object	column / metric
BI	Revenue Daily dashboard	total_revenue (metric)
dbt mart	mart_revenue.daily	daily_revenue
dbt mart	mart_revenue.daily	derived: SUM(amount) - SUM(refund)
dbt intermediate	int_orders_with_refunds	amount, refund
dbt staging	stg_orders	gross_amount (mapped to amount)
Raw	raw_orders	gross_amount (Fivetran sync)
Source	Postgres orders.gross_amount	DECIMAL(10,2)

Code.

-- Field-level lineage trace (Monte Carlo surfaces this in the UI as a graph)
-- BI: Revenue Daily.total_revenue
--   ← mart_revenue.daily.daily_revenue
--       = SUM(int_orders_with_refunds.amount) - SUM(int_orders_with_refunds.refund)
--           ← stg_orders.gross_amount  (renamed from raw_orders.gross_amount)
--               ← raw_orders.gross_amount (Fivetran)
--                   ← Postgres orders.gross_amount

-- Impact-radius query (Monte Carlo runs this on the lineage graph)
SELECT downstream_object, downstream_owner
  FROM monte_carlo.lineage_edges
 WHERE upstream_object = 'warehouse.raw.raw_orders.gross_amount'
   AND transitive_depth <= 5;

Step-by-step explanation.

1. The dashboard total_revenue metric is mapped to mart_revenue.daily.daily_revenue — Monte Carlo learned this from the Looker / Tableau / Power BI metadata feed.
2. daily_revenue is computed in dbt as SUM(amount) - SUM(refund). Monte Carlo parses the dbt model SQL and records each column expression as a field-level edge.
3. amount in int_orders_with_refunds is sourced from stg_orders.gross_amount. The staging-to-intermediate rename is captured as a lineage edge with a rename label.
4. stg_orders.gross_amount flows from raw_orders.gross_amount, which is the Fivetran sync of the source Postgres column.
5. The impact-radius query at any node in this chain answers "if I corrupt this column, which downstream objects are affected?" — a 4-edge walk that takes < 100ms on the platform's lineage store.
6. The on-call uses the chain to localise the regression: the drop is at int_orders_with_refunds (the new refund subtraction logic was deployed yesterday), not at the source.

Output.

upstream column	downstream object	type
raw_orders.gross_amount	stg_orders.gross_amount	rename
stg_orders.gross_amount	int_orders_with_refunds.amount	passthrough
int_orders_with_refunds.amount	mart_revenue.daily.daily_revenue	aggregation (SUM)
int_orders_with_refunds.refund	mart_revenue.daily.daily_revenue	aggregation (subtraction)
mart_revenue.daily.daily_revenue	Revenue Daily.total_revenue	BI metric

Rule of thumb. Field-level lineage pays for itself the first time a senior engineer skips a 4-hour manual hunt through dbt models. Tag dbt models with lineage: columns / owners as a discipline so the platform's auto-parsing is reinforced by explicit metadata.

Worked example — incident triage with impact radius

Detailed explanation. A volume-drop incident fires at 09:14 — fact_orders row count is 62% below the 7-day moving average for the hour. The on-call needs to (1) confirm the incident is real, (2) identify the upstream cause, and (3) communicate the blast radius to stakeholders before the morning all-hands.

Question. Walk the on-call's triage of a volume-drop incident using Monte Carlo's incident workflow and impact-radius query. Show what each step surfaces.

Input.

field	value
incident.kind	volume_anomaly
incident.table	warehouse.prod.fact_orders
incident.severity	page
incident.metric	row_count
incident.baseline	4.2M rows / hour
incident.observed	1.6M rows / hour

Code.

1. On-call receives PagerDuty page at 09:14 UTC.
   Page body:
     INC-04931 — volume_anomaly on fact_orders
     observed 1.6M / expected 4.2M (-62%)
     impact: 7 dashboards, 3 ML features
     runbook: https://runbooks.example.com/fact-orders-volume

2. On-call opens incident in Monte Carlo UI → sees:
   - 6-hour volume timeline with the dip clearly visible
   - upstream tables: raw_orders (last update: 02:00 UTC)
   - upstream incident: freshness_break on raw_orders @ 02:30 UTC

3. Impact-radius query returns:
   - 7 dashboards: Revenue Daily, CRO Weekly, GTM Funnel, …
   - 3 ML features: ltv_v3, churn_v2, propensity_v1
   - 2 customer APIs: /api/v1/revenue, /api/v1/orders-count

4. On-call comments incident: "upstream raw_orders missed 02:00 sync,
   investigating Fivetran connector status."

5. Resolution: Fivetran connector failed at 01:58 UTC due to a
   source-side credential rotation. Reauthenticate, force sync, validate
   row count recovers within 1 hour.

Step-by-step explanation.

1. The PagerDuty page carries the structured incident — observed vs expected row count, downstream impact, runbook URL. The on-call doesn't need to open a dashboard to triage; the page itself is the triage.
2. Inside the Monte Carlo UI, the volume timeline visualises the dip against the 7-day baseline, with the freshness break on raw_orders shown as a correlated event. The platform has already linked the two incidents.
3. The impact-radius query lists every downstream surface. The on-call uses this list to draft a single Slack update — "incident on fact_orders, dashboards listed are affected, ETA 1 hour" — instead of fielding 7 separate questions.
4. The root cause (Fivetran connector credential rotation) is captured as a thread comment on the incident. The thread becomes the post-incident document.
5. Resolution is automatic — Fivetran reauthenticates, the sync catches up, raw_orders freshness recovers, fact_orders volume normalises. Monte Carlo marks the incident auto-resolved once the row count returns to baseline.

Output.

triage step	time elapsed	surfaced by
Page received	0 min	PagerDuty
Upstream cause identified	4 min	Monte Carlo correlated freshness incident
Impact list communicated	7 min	impact-radius query
Root cause confirmed	18 min	Fivetran status + thread comment
Resolution	52 min	automatic detection of baseline recovery

Rule of thumb. Monte Carlo's incident workflow shines when the impact-radius query is part of the page, not a separate UI hop. Make sure the on-call's PagerDuty + Slack integrations include the impact summary in the alert body — without it, the 7-minute "impact communicated" step turns into a 25-minute hunt.

Senior interview question on Monte Carlo's auto-monitor depth

A senior interviewer might ask: "Walk me through how Monte Carlo decides what to monitor on a brand-new table. What does it learn, when does it start firing, and what's the gap between auto-monitor and custom monitor coverage?"

Solution Using the observation-window + per-pillar threshold model

# How Monte Carlo's auto-monitor learns a new table — pseudo-code
class AutoMonitor:
    def __init__(self, table: str, observation_days: int = 7):
        self.table = table
        self.observation_days = observation_days
        self.baselines = {}

    def observe(self):
        """Daily metadata + metric collection during observation window."""
        for day in range(self.observation_days):
            self.baselines.setdefault("freshness", []).append(
                self.measure_freshness(self.table, day))
            self.baselines.setdefault("volume", []).append(
                self.measure_volume(self.table, day))
            self.baselines.setdefault("schema", []).append(
                self.measure_schema_fingerprint(self.table, day))

    def fit(self):
        """At end of observation window, compute baselines + tolerances."""
        self.cadence = self.decompose_seasonality(self.baselines["freshness"])
        self.volume_baseline = self.decompose_seasonality(self.baselines["volume"])
        self.schema_baseline = self.baselines["schema"][-1]

    def evaluate(self) -> list[Incident]:
        """Per-pillar check against the learned baseline."""
        incidents = []
        if self.observed_freshness() > self.cadence.upper_bound():
            incidents.append(Incident("freshness_break", severity="page"))
        if abs(self.observed_volume() - self.volume_baseline.predict()) > \
                self.volume_baseline.tolerance():
            incidents.append(Incident("volume_anomaly", severity="warn"))
        if self.observed_schema() != self.schema_baseline:
            incidents.append(Incident("schema_change", severity="page"))
        return incidents

Step-by-step trace.

Day	What the auto-monitor does
0 (table added)	record initial schema; begin observation window
1–6	collect daily freshness, volume, schema samples; no alerts yet
7	decompose seasonality; fit baselines; first eligible alert at end of day
8+	evaluate against learned baselines every scheduler tick
30	enough data for distribution drift checks; opt-in by table
90	mature baselines; sensitivity tuning recommendations surface

The observation window is the price of "zero config" — Monte Carlo waits 7 days before paging on a new table, because firing without a baseline would produce a wall of false positives.

Output:

pillar	auto-monitor coverage	needs custom config
Freshness	yes, after 7-day window	only for non-standard cadences
Volume	yes, after 7-day window	for known seasonal exceptions (Black Friday)
Schema	yes, from day 1	none
Distribution	opt-in per table, after 30 days	column selection, sensitivity
Lineage	yes, continuously refreshed	enrich with dbt / BI metadata

Why this works — concept by concept:

• Observation window before alerting — auto-monitor without a baseline is alert spam. The 7-day window trades initial coverage for false-positive control, and the trade is correct for almost every team.
• Seasonality decomposition — daily and weekly cycles in freshness and volume are real (Sunday lulls, Monday spikes). The baseline must decompose them or every weekend alerts page the on-call.
• Schema as a binary signal — schema diff is exact and unambiguous. Schema auto-monitor can fire from day 1 without an observation window because the false-positive rate is zero.
• Distribution as opt-in — distribution checks are compute-expensive and the most prone to noise. Making them opt-in per table preserves the auto-monitor's "low false positive" reputation.
• Cost — auto-monitor cost is dominated by warehouse compute for the metric queries. Per-table cost scales with the number of columns + the sensitivity setting; default settings keep the warehouse bill < $50/month per tier-1 table.

ETL
Topic — etl
Monte-Carlo-style monitor problems

Practice →

SQL Topic — sql SQL drills for warehouse metadata

Practice →

---

3. Anomalo deep-dive

anomalo is the ML-first detection model — no thresholds to tune, auto-feature engineering on every column, sample-set inspection in the UI

The mental model in one line: Anomalo points an ML model at every column of every table, learns the baseline distribution and seasonality automatically, fires incidents when the live data deviates beyond what the model considers normal, and shows the analyst a side-by-side comparison of "anomalous samples" vs "normal samples" so the human can confirm or dismiss without writing SQL. Once you say "ML on every column, no thresholds, sample-set evidence," every anomalo interview question collapses to a deduction from that detection-model bet.

ML-first anomaly detection — the design bet.

• No thresholds to set. The user does not configure "alert if null rate > 5%." Instead, Anomalo learns the null rate's normal range and fires when the live value falls outside that range under its statistical model.
• Per-column models. Each column gets its own anomaly detector. Numerical columns get distribution + drift models; categorical columns get top-K + new-value models; timestamps get cadence + range models.
• Per-segment models. Anomalo can decompose by a segmentation key (e.g. country, customer_tier) and fit separate baselines per segment. A null rate spike that is only present in country=BR is caught even when the global null rate is unchanged.
• Seasonality awareness. Daily, weekly, and monthly cycles are decomposed automatically. The model knows that Sunday traffic is lower than Monday and does not page on the difference.

Auto-feature engineering — the column-by-column scan.

• Numerical features. Mean, median, p50/p95/p99, stddev, min, max, null rate, zero rate, negative rate, distinct count.
• Categorical features. Distinct count, top-K share, new-value detection, null rate, length distribution for strings.
• Timestamp features. Min, max, range, cadence (mean delta between consecutive timestamps), out-of-order rate, future-dated rate.
• Cross-column features. Correlation between numerical columns, conditional null rate, foreign-key validity (column-A values must exist in column-B's distinct set).

Sample-set inspection — the UI bet.

• When an anomaly fires, the UI shows two side-by-side panels: a sample of rows from the anomalous window (e.g. today) and a sample from the baseline window (e.g. last week). The human can scan both and confirm "yes, the data is different" or "no, this is a known business change, dismiss."
• The platform exposes a one-click explain root cause action that runs a per-column decomposition and ranks columns by their contribution to the anomaly score.
• The "show me the rows" affordance is the qualitative differentiator — analysts trust an alert they can see, not an alert that just shows a number.

Bring-your-own-rule (BYOR) — the escape hatch for hard SLAs.

• Not every check is ML-shaped. Anomalo lets users write bring-your-own-rule monitors as SQL queries, with thresholds, severities, and schedules. Common BYOR patterns:
• Hard NOT NULL on a tier-1 primary key.
• Row count > N for tables expected to never be empty.
• Foreign-key validity with explicit join + count of orphans.
• Custom business rule — e.g. "every order with status=shipped must have a non-null shipped_at."
• BYOR coexists with ML monitors. ML catches the contracts you didn't write; BYOR enforces the ones you must.

Pricing model + deployment shape.

• Per-table pricing, usually with volume discounts for portfolios > 500 tables.
• In-warehouse compute — Anomalo runs ML scoring inside your Snowflake / BigQuery / Databricks warehouse using warehouse compute. Same "no data egress" architecture as Monte Carlo.
• API + UI parity — every monitor configurable via REST API as well as the UI, so platform teams can codify monitors in IaC.

Common interview probes on Anomalo.

• "What does Anomalo do that a threshold-based platform doesn't?" — learns per-column baselines automatically; catches drift the human didn't know to threshold.
• "What does Anomalo not do well?" — hard SLAs where you need a specific threshold (use BYOR), deep lineage (Anomalo's lineage is shallower than Monte Carlo's), and incident management at scale.
• "How do you explain an Anomalo alert to a non-data stakeholder?" — open the sample-set inspection view; show the side-by-side rows; let the rows tell the story.
• "What's the false-positive story?" — the ML model has a learned sensitivity per column; the user can dial sensitivity up or down per monitor; the auto-default is conservative.

Worked example — catching a stealth distribution drift

Detailed explanation. A users table has a country column. Yesterday, country=BR was 12% of new sign-ups; today, it is 47% of new sign-ups because a marketing campaign in Brazil went live overnight. The global null rate, row count, freshness, and schema are all unchanged — no threshold-based check would fire. Anomalo's per-column ML model catches the distribution drift and surfaces the change.

Question. Show how Anomalo detects the Brazil spike and how the sample-set inspection view explains the alert to the on-call.

Input.

signup_date	country	share
last 30 days	BR	12%
last 30 days	US	41%
last 30 days	DE	14%
today	BR	47%
today	US	28%
today	DE	9%

Code.

# Anomalo monitor — auto-feature on `country` column
monitor:
  table: warehouse.dim.dim_users
  column: country
  kind: categorical_distribution
  baseline_window: 30d
  sensitivity: default

# The user does NOT set a threshold like "alert if BR > 30%."
# Anomalo learns the rolling baseline (BR @ ~12% with normal variance)
# and fires when today's value falls outside the learned tolerance.

# Anomalo's per-column ML model — pseudo-code
class CategoricalDriftDetector:
    def fit(self, baseline_window: list[Sample]):
        self.share_by_value = compute_top_k_share(baseline_window, k=10)
        self.share_variance = compute_share_variance_per_value(baseline_window)

    def score(self, today_sample: Sample) -> float:
        anomaly_score = 0.0
        for value, today_share in today_sample.top_k_share().items():
            baseline_share = self.share_by_value.get(value, 0.0)
            z = (today_share - baseline_share) / max(self.share_variance.get(value, 0.01), 0.01)
            anomaly_score = max(anomaly_score, abs(z))
        return anomaly_score

Step-by-step explanation.

1. Anomalo's auto-feature step computes top-K share for the country column across the 30-day baseline window. BR has a learned share of 12% with low variance (~1.5 percentage points).
2. Today, the live share of BR is 47%. The model's z-score for BR is (47 - 12) / 1.5 ≈ 23, well outside the learned tolerance.
3. The model emits an incident with an anomaly score of 23 and ranks BR as the top contributing value. The other deviated values (US at -13 pp, DE at -5 pp) are listed as secondary.
4. The sample-set inspection view shows 20 rows from today's BR sign-ups next to 20 rows from the baseline period. The on-call sees the row count, the email-domain mix, the device mix, and confirms "yes, distribution shifted."
5. The "explain root cause" action runs the per-column decomposition and surfaces that BR sign-ups have a different referrer_url pattern (marketing campaign URL parameter), pinpointing the cause without manual hunting.

Output (anomaly score per top-K value).

value	baseline share	today share	z-score	rank
BR	12%	47%	+23.3	1
US	41%	28%	-8.7	2
DE	14%	9%	-3.3	3
FR	7%	6%	-0.6	(normal)
IN	6%	5%	-0.5	(normal)

Rule of thumb. Distribution drift is the single highest-value Anomalo use case — it catches "contracts you didn't write" that no threshold-based tool would flag. Enable categorical-distribution monitors on every column that drives downstream segmentation or ML features.

Worked example — bring-your-own-rule for a hard SLA

Detailed explanation. A payments table must never have a null transaction_id. This is a hard contract — not a statistical baseline — and an ML model would be the wrong tool. Anomalo's BYOR feature lets you express the rule as SQL with a hard threshold (zero nulls allowed) and a page severity.

Question. Write an Anomalo BYOR monitor for "no nulls in transaction_id" and explain how it composes with the ML auto-monitors on the same table.

Input.

table	column	contract
warehouse.prod.payments	transaction_id	NOT NULL — hard SLA

Code.

# Anomalo BYOR — hard SLA on transaction_id
monitor:
  table: warehouse.prod.payments
  kind: byor
  name: "no nulls in transaction_id"
  sql: |
    SELECT COUNT(*) AS violation_count
      FROM warehouse.prod.payments
     WHERE transaction_id IS NULL
       AND created_at >= CURRENT_DATE - INTERVAL '1 day'
  threshold: 0
  comparison: "<="
  severity: page
  schedule: every 15m
  alert:
    channel: pagerduty://payments-rota
    runbook: https://runbooks.example.com/payments-null-txn

Step-by-step explanation.

1. The BYOR monitor runs the SQL query every 15 minutes against the warehouse. The query counts rows with a null transaction_id in the last day.
2. The threshold is <= 0 — i.e. any violation pages immediately. Severity is page, not warn, because the contract is non-negotiable.
3. In parallel, the auto-ML monitors on the same table still run. They watch the row count, distribution of amount, freshness of the table, etc. The two layers compose: hard rule for the inviolable contract, ML for the unknown unknowns.
4. If the BYOR fires, the on-call sees the count of violating rows, can drill into the sample (the platform exposes the violating row IDs in the alert), and proceeds with the runbook.
5. The pattern matches the "testing + observability" model from Section 1 — BYOR is testing-flavoured, ML monitors are observability-flavoured, and both run inside the same platform.

Output.

monitor	kind	sensitivity / threshold	severity
auto-row-count	ML	learned	warn
auto-distribution(amount)	ML	learned	warn
auto-freshness	ML	learned	page
byor-no-null-txn-id	BYOR	0	page
byor-fk-validity	BYOR	0	page

Rule of thumb. Use ML for "I want to know when something is weird" and BYOR for "this MUST be true." If a stakeholder can articulate the rule in one sentence, it should be BYOR; if they can only say "I'll know it when I see it," it should be ML.

Worked example — sample-set inspection on a numerical drift

Detailed explanation. A transactions table has an amount column. Yesterday, the p95 was $145; today, the p95 is $623. The mean is roughly unchanged because the increase is concentrated at the upper tail. A naive mean check would miss it; Anomalo's per-column model on p95 catches it.

Question. Show Anomalo's detection of a tail-drift in amount and the sample-set view that helps the on-call confirm the anomaly.

Input.

metric	baseline (30d)	today
mean	$48	$52
median	$32	$34
p95	$145	$623
p99	$312	$1840

Code.

monitor:
  table: warehouse.prod.transactions
  column: amount
  kind: numerical_distribution
  features: [mean, median, p95, p99, stddev, null_rate]
  baseline_window: 30d
  sensitivity: default

-- Sample-set inspection — Anomalo runs both queries automatically
-- "Anomalous" sample
SELECT transaction_id, customer_id, amount, created_at
  FROM warehouse.prod.transactions
 WHERE created_at >= CURRENT_DATE
   AND amount >= percentile_cont(amount, 0.90) OVER ()
 LIMIT 20;

-- "Baseline" sample
SELECT transaction_id, customer_id, amount, created_at
  FROM warehouse.prod.transactions
 WHERE created_at BETWEEN CURRENT_DATE - INTERVAL '30 days' AND CURRENT_DATE - INTERVAL '1 day'
   AND amount >= percentile_cont(amount, 0.90) OVER ()
 LIMIT 20;

Step-by-step explanation.

1. The per-column ML model computes the feature vector (mean, median, p95, p99, stddev, null_rate) for today and compares it to the rolling 30-day baseline. The p95 z-score is huge (~12); the mean z-score is small (~0.3).
2. Anomalo's ranking surfaces p95 as the top contributing feature, with an "explain root cause" note that the drift is concentrated in the upper tail of the distribution.
3. The sample-set view shows 20 rows from today's >p90 amounts next to 20 rows from the baseline >p90 amounts. The on-call sees that today's high-value transactions are concentrated at a few customer IDs — likely a corporate-account onboarding event.
4. The on-call confirms with the revenue team that a large corporate account did onboard yesterday, and the spike is legitimate. The incident is dismissed as "expected business event" without the platform learning to ignore the pattern (so a real future tail-drift will still fire).
5. The user can optionally "teach" the model that the segment customer_tier=corporate has a different baseline — the next deployment will fit separate models per segment.

Output (per-feature z-score ranking).

feature	baseline	today	z-score	rank
p95	$145	$623	+12.4	1
p99	$312	$1840	+11.9	2
stddev	$54	$191	+9.8	3
mean	$48	$52	+0.3	(normal)
median	$32	$34	+0.2	(normal)
null_rate	0.4%	0.4%	0.0	(normal)

Rule of thumb. Tail metrics (p95, p99) catch the regressions that mean-based checks miss. Anomalo's per-feature ML scoring is the cheapest way to instrument tail-drift detection across 1000s of numerical columns without writing one threshold by hand.

Senior interview question on Anomalo's ML detection model

A senior interviewer might ask: "Anomalo's pitch is 'no thresholds to tune.' What's the catch? When does the ML model produce false positives, when does it miss real incidents, and how do you operate it at scale?"

Solution Using the sensitivity-tier + segmentation-aware operating model

# Operating Anomalo at scale — the playbook
class AnomalyOperatingModel:
    def __init__(self):
        # Three sensitivity tiers per monitor — default is "medium"
        self.tiers = {
            "low":      {"z_threshold": 4.0, "false_positive_rate": "rare"},
            "medium":   {"z_threshold": 3.0, "false_positive_rate": "moderate"},
            "high":     {"z_threshold": 2.0, "false_positive_rate": "frequent"},
        }

    def tune(self, monitor, history):
        """Lower sensitivity if false-positive rate exceeds 1 per week."""
        fp_rate_per_week = compute_fp_rate(history, lookback_days=30)
        if fp_rate_per_week > 1:
            return self.dial_down(monitor)
        if true_positive_missed_rate(history) > 0.05:
            return self.dial_up(monitor)
        return monitor

    def segment_decompose(self, monitor):
        """If the monitor fires on a segment-correlated pattern,
           fit a per-segment model."""
        if monitor.last_3_fires_correlated_with_segment("country"):
            return self.split_by_segment(monitor, "country")
        return monitor

Step-by-step trace.

Step	Action	Outcome
1	Deploy auto-monitors on tier-1 tables, default sensitivity	most columns covered
2	Observe first 30 days of incidents	baseline FP rate per monitor
3	For monitors with > 1 FP / week, dial sensitivity down	FP rate < 1 / week
4	For monitors with missed true positives, dial up	TP coverage improves
5	If alerts correlate with a segment, split by segment	per-segment baselines
6	Add BYOR for hard SLAs that ML cannot enforce	hard contracts covered

The operating model is iterative — you don't tune at deploy time; you tune from the first 30 days of incident logs. Anomalo's "no thresholds" promise refers to initial setup, not to steady state.

Output:

sensitivity dial	when to use	example monitor
low (z=4)	noisy columns, expected variance	new sign-up source mix
medium (z=3)	default for most monitors	numerical distributions
high (z=2)	tier-1 columns, tight SLAs	revenue total per day
BYOR	hard contracts (must / must not)	NOT NULL on PK
per-segment	metric varies by known dimension	distribution per country

Why this works — concept by concept:

• No thresholds at deploy, plenty at steady state — the value of the ML model is removing the blank-page problem (what threshold do I pick for null rate on column X?). After 30 days of operation, you have data to tune from.
• Sensitivity tiers as the tuning knob — three discrete tiers are easier to operate than a free-form slider. Most teams converge on "low" for noisy columns and "high" for tier-1.
• Segmentation defeats false positives — many "false positives" are actually true positives in a subset of the data. Fitting per-segment models converts them into actionable per-segment alerts.
• BYOR composes with ML — the two layers are complementary, not alternatives. ML catches drift; BYOR enforces hard contracts. Mature deployments run both on the same tier-1 table.
• Cost — ML scoring cost is dominated by warehouse compute for the per-column queries. Scales linearly with column count × scoring frequency. Tier-1 tables at $30-50/month, tier-2 at $10-15, tier-3 at $3-5 in typical Snowflake pricing.

ETL
Topic — etl · medium
Anomalo-style drift problems

Practice →

SQL Topic — sql SQL drills for distribution checks

Practice →

---

4. Bigeye + Lightup

bigeye is the metric-store framing — every check is a metric you can export; lightup is the SLO framing — every check has an MTTI / MTTR target

The mental model in one line: Bigeye treats data observability as a metric-store problem (autometrics + custom SQL metrics on every table, exported wherever you want them) and Lightup treats it as an SRE problem (every monitor is a service-level objective with an error budget, an MTTI target, and an MTTR target). Once you say "metric store vs SLO contract," every bigeye vs lightup interview question collapses to which framing matches the team's operating model.

Bigeye — the metric-store framing.

• Autometrics — like Monte Carlo and Anomalo, Bigeye ships out-of-the-box monitors on freshness, volume, schema, and a curated set of distribution metrics. The catalog of autometrics is the breadth lever.
• Custom SQL metrics — Bigeye's differentiator is the first-class support for custom-SQL metric definitions. Any SQL expression that returns a single numeric value can be registered as a metric, scheduled, baselined, and alerted on.
• Metric export — every metric (autometric or custom) can be exported to Datadog, Grafana, Snowflake, BigQuery, or piped into an internal metric store via the Bigeye API. Data quality becomes a first-class metric category alongside infra metrics.
• Anomaly detection on metrics — the same statistical baseline + ML detection model runs on top of the metric stream. Sensitivity is configurable per metric.
• API-first — every Bigeye object (metric, monitor, alert, integration) is REST-API addressable; teams that codify everything in IaC (Terraform, Pulumi) find this easier to operate than UI-first vendors.

Lightup — the data-SLO framing.

• Data SLOs — every monitor is framed as a service-level objective with three properties: a target (99.9% on-time), an error budget (43 minutes per month), and an MTTI / MTTR target (5 min / 30 min).
• In-warehouse query injection — Lightup runs its checks by injecting SQL queries into the warehouse. No external compute; no data egress; same warehouse-native architecture as Monte Carlo and Anomalo.
• SLO-style monitors — instead of "alert if null rate > 5%," Lightup expresses the same as "freshness SLO: 99.9% of windows on-time, page after 30 minutes of error budget burn." The semantics map directly to application SLOs.
• Error-budget burn rate — alerts are based on burn rate rather than absolute threshold. A spike that consumes 25% of the monthly error budget in one hour pages immediately; a slow drift that consumes 10% over a week is a warn.
• Severity escalation — Lightup tiers monitors by SLO severity (sev1 = customer-facing, sev2 = revenue-affecting, sev3 = internal); each tier has its own routing + on-call rotation.

The 5 pillars on both platforms.

• Both cover freshness, volume, schema, distribution out of the box. Lineage is shallower on both than on Monte Carlo — table-level + dbt-derived, no field-level lineage.
• Bigeye's strength. Custom SQL metrics + metric-store export. If your team has 200 custom data-quality SQL queries that you've been running as cron jobs, Bigeye replaces all of them with one declarative platform.
• Lightup's strength. SLO framing + error-budget burn-rate alerts. If your team already runs application SLOs and the on-call rota understands burn-rate semantics, Lightup is the lowest-friction adoption.

Alerting + routing — where both shine.

• Bigeye integrates with Slack, PagerDuty, Opsgenie, Microsoft Teams, ServiceNow, Jira, and webhook. Alerts carry the metric value, the baseline, the threshold, and a deep link into the metric view.
• Lightup integrates with the same set plus a richer on-call schedule model (sev1 → sev2 escalation, time-of-day routing, burn-rate-aware throttling).
• Both expose runbook URLs as a first-class field on every monitor, so the on-call page carries the operational context.

Pricing model + deployment shape.

• Bigeye — per-metric or per-table pricing tier, with volume discounts. Warehouse-native; metric storage is in your warehouse plus Bigeye's metadata store.
• Lightup — per-asset or per-monitor pricing. Warehouse-native; SLO storage and error-budget tracking is in Lightup's managed metadata store.

Common interview probes on Bigeye and Lightup.

• "What does Bigeye do that Monte Carlo doesn't?" — first-class custom-SQL metric support, metric-store export, API-first.
• "What does Lightup do that the others don't?" — SLO framing, error-budget burn-rate alerts, SRE-native vocabulary.
• "When would you pick Bigeye over Monte Carlo?" — when your team has heavy custom-SQL coverage already and wants to consolidate into a metric store, and when lineage depth is not the deciding factor.
• "When would you pick Lightup?" — when your platform team already runs application SLOs and wants the same vocabulary for data; when the on-call rota is SRE-trained.

Worked example — Bigeye autometric + custom SQL metric on the same table

Detailed explanation. A subscriptions table needs the standard autometrics (freshness, volume, schema) plus a custom business rule — "active subscription count should be within 5% of 7-day rolling average." The custom rule is a domain-specific SQL query, perfect for Bigeye's custom-metric framework.

Question. Configure Bigeye autometrics on subscriptions and add a custom-SQL metric for the active-subscription count. Show the configuration and the resulting metric stream.

Input.

metric	source	baseline
freshness_minutes_since_write	autometric	learned ~30 min
row_count	autometric	learned ~12.4M
schema_fingerprint	autometric	static (paged on diff)
active_subscription_count	custom SQL	rolling 7d MA

Code.

# Bigeye monitor — autometrics + custom SQL metric on subscriptions
table: warehouse.prod.subscriptions

autometrics:
  - freshness
  - row_count
  - schema_change
  - null_rate(customer_id)
  - null_rate(plan_id)
  - distinct_count(plan_id)

custom_metrics:
  - name: active_subscription_count
    sql: |
      SELECT COUNT(*) AS metric_value
        FROM warehouse.prod.subscriptions
       WHERE status = 'active'
         AND (cancelled_at IS NULL OR cancelled_at > CURRENT_TIMESTAMP)
    schedule: every 1h
    baseline: rolling_7d_ma
    tolerance_pct: 5
    alert:
      warn_channel: slack://#data-warn
      page_channel: pagerduty://billing-rota
      runbook: https://runbooks.example.com/active-subs

export:
  - target: datadog
    metric_prefix: warehouse.subscriptions
  - target: snowflake
    table: data_quality.bigeye_metrics

Step-by-step explanation.

1. The autometrics block enables six built-in checks on subscriptions — freshness, row count, schema change, null rates on two columns, distinct count on plan_id. Each one runs on its default schedule; baselines are learned automatically.
2. The custom-metric block defines active_subscription_count as a SQL query that returns one number. Bigeye schedules the query every hour, persists the result as a time series, and baselines it against the rolling 7-day moving average.
3. The tolerance is set at 5%. Any deviation beyond ±5% from the baseline alerts; severity is determined by the channel routing (Slack warn, PagerDuty page).
4. The export block pipes every metric (autometric + custom) into Datadog and into a Snowflake table for downstream analytics. The metric becomes a first-class observable signal alongside infra metrics in the team's existing dashboards.
5. When the active count drops 8% one Tuesday morning, Bigeye fires a Slack warn at the 5% threshold and a PagerDuty page once the deviation crosses 10%. The on-call has the SQL query at hand to drill into which customer segment shifted.

Output (Bigeye metric stream — hourly samples).

timestamp	metric_value	baseline	deviation_pct
08:00	4.20M	4.20M	0.0%
09:00	4.18M	4.20M	-0.5%
10:00	4.10M	4.20M	-2.4%
11:00	3.95M	4.20M	-5.9% (warn)
12:00	3.78M	4.20M	-10.0% (page)
13:00	3.85M	4.20M	-8.3%

Rule of thumb. Bigeye's custom-metric framework is the right home for any data-quality SQL query you've been running as a Cron job. Migrating those queries gives you scheduling, baseline learning, alerting, runbook URLs, and metric-store export for free.

Worked example — Lightup data SLO with error-budget burn rate

Detailed explanation. A revenue_daily mart is tier-1 — the CFO's dashboard reads from it every morning at 09:00. The team defines a data SLO: 99.9% of the time, revenue_daily is on-time within a 60-minute window of the expected 08:30 update. Lightup expresses this as a freshness SLO with a 43-minute monthly error budget. Burn-rate alerts trigger when the budget is being consumed too quickly.

Question. Configure a Lightup freshness SLO on revenue_daily with burn-rate alerts and explain how the burn-rate alert differs from an absolute threshold alert.

Input.

SLO field	value
metric	freshness — minutes since last write within 60 min of 08:30 target
target	99.9% on-time
window	rolling 30 days
error budget	(1 - 0.999) × 30 days × 24 × 60 = 43.2 min
burn rate alert	5% of monthly budget in 1 hour = page

Code.

# Lightup SLO — freshness on revenue_daily
slo:
  name: revenue_daily_freshness
  table: warehouse.marts.revenue_daily
  sli:
    kind: freshness
    expected_update: "08:30 UTC daily"
    tolerance_minutes: 60
  target: 0.999
  window: 30d
  severity: sev1
  error_budget_alerts:
    - burn_rate_window: 1h
      burn_rate_threshold_pct: 5
      severity: page
      channel: pagerduty://platform-rota
    - burn_rate_window: 6h
      burn_rate_threshold_pct: 10
      severity: page
      channel: pagerduty://platform-rota
    - burn_rate_window: 24h
      burn_rate_threshold_pct: 25
      severity: warn
      channel: slack://#data-warn
  runbook: https://runbooks.example.com/revenue-daily

Step-by-step explanation.

1. The SLO is defined as "99.9% of the time, the table is on-time within 60 minutes of 08:30 UTC." Over a rolling 30-day window, the team is allowed 43.2 minutes of error-budget consumption.
2. The fast-burn alert fires when 5% of the monthly budget is consumed in 1 hour — that is 2.16 minutes of error in 60 minutes, indicating an active incident that will exhaust the budget in 20 hours if it continues.
3. The medium-burn alert fires when 10% of the monthly budget is consumed in 6 hours — a slower but still significant drift.
4. The slow-burn alert fires when 25% of the monthly budget is consumed in 24 hours — a Slack warn that flags a chronic freshness issue without paging the on-call.
5. The key difference vs an absolute-threshold alert: an absolute threshold ("page if more than X minutes late") fires on any breach. A burn-rate alert fires only when the breach rate would consume the SLO's error budget. Brief, isolated freshness misses do not page; sustained or severe ones do.

Output (burn-rate evaluation over a 24-hour window).

hour	minutes late	cumulative burn	fast (1h)?	medium (6h)?	slow (24h)?
08:30	30 min	30 min (69% of monthly)	YES (page)	—	—
09:30	0 min	30 min	no	YES (page)	—
12:30	0 min	30 min	no	no	—
next day 08:30	0 min	30 min	no	no	YES (warn)

Rule of thumb. Burn-rate alerts give you "wake the on-call only when it matters" semantics. Use them for tier-1 SLOs where you want both fast detection of major incidents and immunity from transient blips. If your team already runs application SLOs, the vocabulary transfers one-to-one.

Worked example — severity escalation for tier-1 freshness break

Detailed explanation. A freshness break on revenue_daily starts at 09:00. The Lightup SLO is tier-1 with severity escalation rules — start at sev2 (Slack), escalate to sev1 (PagerDuty) after 15 minutes, escalate to a directly-paged exec channel after 30 minutes. The escalation ladder ensures the right people are involved at each stage without alert fatigue.

Question. Walk the severity escalation timeline for a 35-minute freshness break on a tier-1 SLO. Show what fires at each stage.

Input.

time	minutes late	sev tier	channel
09:00	0	(none)	(none)
09:15	15	sev2	slack://#data-warn
09:30	30	sev1	pagerduty://platform-rota
09:45	45	sev0	slack://#exec-incidents

Code.

slo:
  name: revenue_daily_freshness
  table: warehouse.marts.revenue_daily
  severity_ladder:
    - start_minutes_late: 15
      severity: sev2
      channel: slack://#data-warn
      message: "revenue_daily 15 min late — investigating"
    - start_minutes_late: 30
      severity: sev1
      channel: pagerduty://platform-rota
      message: "revenue_daily 30 min late — page on-call"
    - start_minutes_late: 45
      severity: sev0
      channel: slack://#exec-incidents
      message: "revenue_daily 45 min late — exec notification"
  auto_resolve: true

Step-by-step explanation.

1. At 09:15, the freshness gap crosses the 15-minute threshold. Lightup posts a sev2 Slack message in #data-warn. The platform team sees the warn but no one is paged.
2. At 09:30, the gap crosses 30 minutes. Lightup pages the on-call via PagerDuty. The page carries the SLO context, the burn rate, the runbook URL, and the upstream tables Lightup recommends checking.
3. At 09:45, the gap crosses 45 minutes. Lightup posts an exec-channel notification, surfacing the incident to leadership. No additional page — the leadership channel is for awareness, not response.
4. On-call resolves at 09:48 (root cause: upstream Airflow DAG retry succeeded). Lightup auto-resolves the incident because the freshness curve recovers within the SLO tolerance.
5. The post-incident report (auto-generated by Lightup) captures the timeline, the burn rate, the error-budget consumption (28.5% of the monthly budget burned in this one incident), and a link to the runbook.

Output.

time	event	channel	severity
09:15	sev2 fire	slack://#data-warn	warn
09:30	sev1 fire	pagerduty://platform-rota	page
09:45	sev0 fire	slack://#exec-incidents	notify
09:48	auto-resolve	slack://#data-warn	resolved
09:50	post-incident report	(generated)	—

Rule of thumb. Severity escalation is the SRE pattern that data observability inherits from application SLOs. Use it for tier-1 SLOs where the cost of a missed escalation is real (exec visibility, contractual SLAs). Don't use it for tier-3 monitors — the overhead is not worth it.

Senior interview question on choosing between Bigeye and Lightup

A senior interviewer might ask: "Your team needs a third observability platform on top of an existing Monte Carlo deployment that handles lineage. The new platform must handle custom SQL business rules and SLO-style alerts. How do you choose between Bigeye and Lightup, and would you ever run both?"

Solution Using a feature-by-feature mapping plus an operating-model fit check

Bigeye vs Lightup — feature-by-feature
======================================

                       | Bigeye         | Lightup         |
-----------------------+----------------+-----------------+
Autometrics            | yes (broad)    | yes (focused)   |
Custom SQL monitors    | first-class    | first-class     |
Metric-store export    | first-class    | secondary       |
SLO framing            | secondary      | first-class     |
Error-budget burn-rate | no             | yes             |
Severity escalation    | basic          | advanced        |
Lineage depth          | table-level    | table-level     |
API coverage           | first-class    | first-class     |
On-call schedule mgmt  | PagerDuty link | native + PD/OG  |

Operating-model fit
===================

Pick Bigeye if:
  - Team has many custom SQL queries today, wants to consolidate
  - Metrics are exported to Datadog/Grafana for unified dashboards
  - Platform team thinks in "metrics" (like infra observability)

Pick Lightup if:
  - Team already runs application SLOs (services have SLOs)
  - On-call rotation is SRE-trained
  - Stakeholders ask "what's our data uptime?" in SLO terms

Run both if:
  - Custom SQL monitors are owned by analytics engineers (Bigeye)
  - SLO contracts are owned by the platform team (Lightup)
  - The org separates "data quality engineering" from "data SRE"

Step-by-step trace.

Step	Decision input	Conclusion
1	Existing observability platform	Monte Carlo (lineage + auto-monitor)
2	Gap to fill	Custom SQL + SLO framing
3	Team operating model	Mixed: analytics eng + platform SRE
4	Stakeholder vocabulary	Both "metrics" and "uptime" used
5	Existing infra dashboards	Datadog + Grafana already wired
6	Final pick	Bigeye for custom SQL + metric export; Lightup for tier-1 SLOs

The trace shows that both can coexist with Monte Carlo, each filling a different gap. The team avoids vendor sprawl by clearly assigning ownership: analytics engineers own Bigeye monitors; platform engineers own Lightup SLOs.

Output:

vendor	role	owner
Monte Carlo	lineage + auto-monitor + impact radius	platform team
Bigeye	custom SQL metrics + Datadog export	analytics eng
Lightup	tier-1 SLOs + burn-rate alerts	platform SRE

Why this works — concept by concept:

• Feature-by-feature mapping first — the table makes "what does each platform genuinely uniquely do?" explicit. Avoids the common trap of comparing on marketing claims rather than capability.
• Operating-model fit — the vocabulary the team already speaks ("metrics" vs "SLOs") predicts adoption success. A platform whose vocabulary clashes with team culture is rejected even when its features are stronger.
• Hybrid stacks are normal — large enterprises commonly run 2-3 observability vendors. The cost is real (vendor sprawl), but the alternative (one vendor that does everything mediocre) is worse.
• Clear ownership boundaries — vendor sprawl is fine if ownership is clear. The first thing to fail in a hybrid stack is "who owns this alert?" — solve it at deploy time.
• Cost — three vendors at $X each ≈ $3X annual cost. Compare to one vendor at $2X with 30% feature coverage — the hybrid usually wins on coverage per dollar but loses on procurement friction.

ETL
Topic — etl
SLO + metric-store problems

Practice →

SQL Topic — optimization Optimization drills for metric queries

Practice →

---

5. Picking the platform — decision matrix

Senior engineers pick data observability tools from a 5-question decision tree — never from "which has the most features"

The mental model in one line: the platform choice is a fit between your team's existing operating model (catalog-adjacent, SRE-trained, analytics-led, or low-ops) and the vendor's design bet (auto-monitor + lineage, ML-first, metric-store, or SLO) — never from a feature checklist. Once you internalise the decision tree, the data observability comparison interview surface collapses to "what does your team actually operate, and which vendor matches that?"

The 5 questions in order.

• Q1 — Operating model. Do you have a platform team that owns the warehouse end-to-end (lineage matters, big team) → Monte Carlo. Analytics-engineering-led, low-config, low-platform-team → Anomalo. Custom-SQL-heavy, metric-store culture → Bigeye. SRE-trained, SLO-first → Lightup.
• Q2 — Lineage depth. Do you need field-level lineage with impact-radius queries for incident triage? → Monte Carlo is the only mature option. Table-level or column-level is enough? → all four work.
• Q3 — Detection model. Do you want auto-monitor with mostly out-of-the-box coverage? → Monte Carlo or Anomalo. Do you want ML with explicit "show me the rows" UI? → Anomalo. Do you want fine-grained custom SQL metrics? → Bigeye. Do you want SLO + burn-rate alerts? → Lightup.
• Q4 — Stakeholder vocabulary. What language do execs and PMs use when they ask about data quality? "Incidents and impact" → Monte Carlo. "Anomalies and root cause" → Anomalo. "Metrics and dashboards" → Bigeye. "Uptime and SLOs" → Lightup.
• Q5 — Existing infra fit. Do you already use Datadog / Grafana for infra observability and want to consolidate? → Bigeye export wins. Do you already run application SLOs in Sloth / OpenSLO? → Lightup transfers. Do you already use Atlan / Alation for catalog? → Monte Carlo's lineage integrates best.

The four-quadrant matrix.

• Top-left (low-config, warehouse-native): Monte Carlo. Big team, catalog-adjacent, lineage matters, willing to pay premium for the most mature platform. The "safe enterprise default."
• Top-right (low-config, multi-source): Anomalo. ML-native, no thresholds to tune, "show me the rows" UI. Analytics-engineering teams that don't want a platform team on call.
• Bottom-left (high-config, warehouse-native): Bigeye. Custom-SQL-heavy, metric-store export, API-first. Platform teams that already have a metric culture from infra observability.
• Bottom-right (high-config, multi-source): Lightup. SLO-first, burn-rate alerts, severity ladder. SRE-trained teams that run application SLOs and want the same for data.

Hybrid patterns — when one vendor isn't enough.

• Monte Carlo + Anomalo. Lineage and incident management from Monte Carlo, distribution drift from Anomalo. Common in large enterprises where the platform team owns lineage and the analytics team owns column-level drift.
• Monte Carlo + Bigeye. Lineage from Monte Carlo, custom-SQL metrics from Bigeye. Used when the existing custom-SQL coverage is too valuable to throw away.
• Bigeye + Lightup. Custom metrics from Bigeye, SLO framing from Lightup. Used when the team wants both metric-store export and SLO contracts (the SRE-and-metric-culture overlap).

Cost — the contract-length variable.

• All four vendors price per-asset or per-table with volume discounts. Annual contracts dominate; multi-year (2-3 year) contracts are common for the enterprise tier with 15–25% discounts.
• Monte Carlo is the most expensive at scale due to lineage compute + premium positioning. Typical mid-market deployment: $150K–$300K/year.
• Anomalo is mid-priced and scales with column count + ML compute. Typical mid-market: $100K–$200K/year.
• Bigeye and Lightup are mid-priced and scale with monitor count. Typical mid-market: $80K–$180K/year each.
• All four offer proof-of-concept programs with curated tables — 30–60 days, no commitment. Use the POC to test the operating-model fit before signing.

Hiring pool — the other contract-length variable.

• Monte Carlo has the largest user base, so engineers who have operated it are common in the hiring pool. Lowest onboarding friction.
• Anomalo is growing fast but still niche; engineers experienced with Anomalo are rare outside the company's customer base.
• Bigeye and Lightup are smaller in user base; expect to train rather than hire pre-experienced operators.

Senior interview signals on platform selection.

• Do you start with operating model, not feature list? — senior signal.
• Do you mention MTTI and MTTR as the success metrics, not "number of alerts"? — required answer.
• Do you push back on "which is best?" with "best for what operating model?" — senior signal.
• Do you propose a hybrid stack when the requirements genuinely span more than one vendor's sweet spot? — senior signal.

Worked example — Q1 forces Monte Carlo for a big-team, lineage-heavy org

Detailed explanation. A 200-person data org with a dedicated platform team, an existing Atlan catalog, and 4000+ dbt models needs an observability platform. Lineage is the deciding factor — incidents must trace through impact radius across 50+ Looker dashboards. Monte Carlo is the only vendor with mature field-level lineage at this scale.

Question. Walk Q1 for the big-team, lineage-heavy org and explain why Monte Carlo is the obvious pick.

Input.

dimension	value
Data org size	200
Platform team	dedicated (12 people)
Catalog tool	Atlan
dbt models	4000+
BI surfaces	50+ Looker dashboards
Tier-1 tables	80
Lineage requirement	field-level for impact radius

Code.

Q1 — Operating model
====================
- Big team, dedicated platform team             → Monte Carlo eligible
- Atlan catalog already deployed                 → Monte Carlo integrates best
- 4000+ dbt models                               → field-level lineage critical
- 50+ Looker dashboards                          → impact-radius query mandatory

Q2 — Lineage depth
==================
- Field-level required (impact radius across BI) → only Monte Carlo

Q3 — Detection model
====================
- Auto-monitor + opt-in distribution drift       → Monte Carlo default fits

Q4 — Stakeholder vocabulary
===========================
- Execs use "incident, impact, owner"             → Monte Carlo native

Q5 — Existing infra fit
=======================
- Atlan + Looker + Snowflake                     → Monte Carlo connectors mature

Decision: Monte Carlo, no hybrid required at start.
Optional Anomalo addition in year 2 for column-level distribution drift.

Step-by-step explanation.

1. Q1 alone narrows the field to vendors that fit a big-team operating model — that's Monte Carlo (the enterprise default), Bigeye (API-first), and Lightup (SRE-shaped). Anomalo is filtered out as analytics-led, not platform-led.
2. Q2 (field-level lineage) is decisive — only Monte Carlo has mature field-level lineage. Bigeye and Lightup ship table-level lineage that wouldn't satisfy the impact-radius requirement across 4000+ dbt models.
3. Q3 confirms — auto-monitor is the default, distribution drift is opt-in per table. Matches the team's "we want platform-set defaults" preference.
4. Q4 — exec vocabulary ("incident, impact, owner") matches Monte Carlo's UI vocabulary one-to-one. Low cultural-translation cost.
5. Q5 — Monte Carlo's Atlan + Looker + Snowflake connectors are mature; rolling out is a question of credentials and table selection, not custom integration work.

Output.

pick	reasoning
Monte Carlo	field-level lineage + auto-monitor + Atlan integration
(option year 2) Anomalo	column-level distribution drift on the top 200 tables

Rule of thumb. When lineage depth is the deciding factor, Monte Carlo is the answer. The other three vendors are competitive on detection but not on lineage; no amount of feature growth in the others has closed that gap as of 2026.

Worked example — Q3 forces Anomalo for a low-config analytics-led team

Detailed explanation. A 30-person data team is analytics-engineering-heavy with no dedicated platform team. They need observability that "just works" without ongoing config investment. ML-first detection with the "show me the rows" UI matches the team's working style — analysts can confirm alerts visually without writing SQL.

Question. Walk Q1–Q5 for an analytics-led team and explain why Anomalo is the obvious pick.

Input.

dimension	value
Data org size	30
Platform team	none (3 senior DEs split between platform + analytics)
Catalog tool	none (using dbt docs)
dbt models	800
BI surfaces	12 Tableau dashboards
Tier-1 tables	25
Detection preference	low-config, ML-driven

Code.

Q1 — Operating model
====================
- Small team, analytics-led, no dedicated platform team → Anomalo eligible

Q2 — Lineage depth
==================
- Column-level enough; field-level not required          → all four work

Q3 — Detection model
====================
- ML-first, "no thresholds to tune"                       → Anomalo native
- Sample-set inspection UI for non-engineer review       → Anomalo unique

Q4 — Stakeholder vocabulary
===========================
- Analysts say "anomaly", "root cause", "show me data"   → Anomalo native

Q5 — Existing infra fit
=======================
- Snowflake + dbt + Tableau                              → Anomalo connectors mature

Decision: Anomalo for ML detection + sample inspection.
Optional Monte Carlo addition once lineage becomes pressing (year 2-3).

Step-by-step explanation.

1. Q1 eliminates Monte Carlo (overkill for a small, analytics-led team) and Lightup (no SRE rota). Bigeye and Anomalo remain.
2. Q3 splits Bigeye and Anomalo: the team wants ML-first detection, not custom-SQL metric authoring. The team's working style is "analyst confirms an alert by looking at rows," which is Anomalo's UI bet.
3. Q4 confirms — analyst vocabulary ("anomaly, root cause") matches Anomalo's UI labels. Low translation friction.
4. Q5 — Anomalo's Snowflake + dbt + Tableau integrations are mature; no custom integration work needed.
5. Year-2 reconsideration — once the team grows past 60 and a dedicated platform team forms, lineage becomes more pressing. Add Monte Carlo as a complementary vendor; keep Anomalo for column-level drift.

Output.

pick	reasoning
Anomalo	low-config, ML-first, sample-set inspection UI
(option year 3) Monte Carlo	once lineage depth becomes pressing

Rule of thumb. When the operating model is "analytics engineers + no dedicated platform team," Anomalo wins on UX. The ML-first, no-threshold pitch is genuinely the lowest config burden of any of the four vendors.

Worked example — Q5 forces Lightup for an SRE-trained team

Detailed explanation. A team that already runs Sloth-based application SLOs across their microservices wants the same vocabulary for data quality. The on-call rota is SRE-trained, the engineers know burn-rate semantics, and stakeholders ask "what's our data uptime?" The detection model is secondary to the SLO framing.

Question. Walk Q1–Q5 for an SRE-trained team and explain why Lightup is the obvious pick.

Input.

dimension	value
Data org size	80
Platform team	dedicated, SRE-trained (6 people)
Application SLO platform	Sloth + Prometheus
dbt models	1800
BI surfaces	25 dashboards
Tier-1 tables	45
Vocabulary	uptime, SLO, error budget, burn rate

Code.

Q1 — Operating model
====================
- SRE-trained, runs application SLOs already       → Lightup eligible

Q2 — Lineage depth
==================
- Table-level adequate (impact computed in-house) → all four work

Q3 — Detection model
====================
- Burn-rate alerts, severity ladder                → Lightup native

Q4 — Stakeholder vocabulary
===========================
- Execs ask "what's our data uptime?"             → Lightup vocab matches

Q5 — Existing infra fit
=======================
- Sloth + Prometheus + PagerDuty                  → Lightup PagerDuty integration mature

Decision: Lightup primary, Bigeye optional for metric export to Grafana.

Step-by-step explanation.

1. Q1 narrows to Lightup (only vendor with SRE-native vocabulary) and Bigeye (metric-store fit). Monte Carlo's "incident, impact, owner" vocabulary is fine but doesn't speak SLO.
2. Q3 is decisive — burn-rate alerts and severity ladders are Lightup's first-class feature. Bigeye supports basic severity but doesn't have a burn-rate model.
3. Q4 matches Lightup's vocabulary one-to-one. Execs who ask "data uptime" can be answered with "99.93% over the last 30 days, 28% error-budget remaining" without translation.
4. Q5 — Lightup's PagerDuty + Slack + on-call schedule integrations are mature and match the team's existing infra observability stack.
5. Optional addition: Bigeye for metric-store export to Grafana, so data quality dashboards live next to infra dashboards in the team's existing Grafana stack.

Output.

pick	reasoning
Lightup	SLO + burn-rate + severity ladder + SRE vocab
(option) Bigeye	metric export to existing Grafana infra dashboards

Rule of thumb. When the team's operating-model vocabulary is SRE (SLO, error budget, burn rate), Lightup is the lowest-friction adoption. The cultural transfer is one-to-one; engineers can map their application-SLO mental models to data SLOs in days, not months.

Senior interview question on platform selection in 2026

A senior interviewer might frame this as: "You're joining a data platform team that has no observability today. Walk me through how you'd pick between Monte Carlo, Anomalo, Bigeye, and Lightup. What's the framework, what's the no-regret first move, and when would you stitch a hybrid?"

Solution Using the 5-question framework + the "POC two, pick one" first move

Platform selection framework — 2026
====================================

Step 1: Inventory the operating model
  - Team size + platform team presence
  - Existing catalog tool
  - Existing infra observability stack
  - Stakeholder vocabulary (incident? anomaly? metric? SLO?)
  - Tier-1 table count

Step 2: Walk Q1-Q5 in order
  Q1 Operating model    → narrows to 1-2 candidates
  Q2 Lineage depth      → confirms or eliminates Monte Carlo
  Q3 Detection model    → confirms ML (Anomalo) vs custom SQL (Bigeye) vs SLO (Lightup)
  Q4 Stakeholder vocab  → cultural-fit veto
  Q5 Infra fit          → integration cost veto

Step 3: POC two candidates in parallel (30-60 days)
  - 5 tier-1 tables per vendor (overlap is fine)
  - Measure: MTTI, MTTR, FP rate, eng-hours per incident
  - Compare on the operating model fit, not on feature count

Step 4: Pick one or stitch a hybrid
  - One vendor if the POC clearly fits Q1-Q5
  - Two vendors if Q2 (lineage) + Q3 (detection) genuinely diverge

The no-regret first move
========================
  - Always POC Monte Carlo (it's the safe default if anything goes wrong)
  - POC the second-best Q1 match (Anomalo, Bigeye, or Lightup)
  - 60 days, 5 tables each, real on-call rota
  - Pick by MTTI compression + FP rate, not by feature checklist

Step-by-step trace.

Step	Action	Outcome
1	Inventory operating model	"platform team of 5, SLO-trained, table-level lineage enough"
2	Walk Q1-Q5	Q1 → Lightup or Bigeye; Q3 → Lightup; Q4 → Lightup; Q5 → Lightup
3	POC Lightup + Monte Carlo for 60 days	MTTI: MC 18min, LU 12min; MTTR: MC 1.4h, LU 1.6h
4	Pick Lightup primary	SLO framing + burn-rate + vocab fit
5	(year 2) reconsider Monte Carlo for lineage	adds field-level when impact radius becomes pressing

The trace shows that the framework is iterative — year-1 pick is based on current operating model; year-2 reconsideration is based on growth. The "POC two, pick one" first move keeps the safe default (Monte Carlo) in play while still letting the operating-model fit win.

Output:

dimension	how it shaped the pick
Operating model (Q1)	SRE-trained team → Lightup or Bigeye eligible
Lineage depth (Q2)	table-level enough → no Monte Carlo gate
Detection model (Q3)	SLO + burn-rate → Lightup decisive
Vocab (Q4)	"uptime" matches Lightup → cultural fit
Infra fit (Q5)	PagerDuty + Sloth → Lightup native
POC measurement	MTTI 12 min, MTTR 1.6h

Why this works — concept by concept:

• Operating model before features — vendors who match the team's existing operating model get adopted; those who don't get rejected even when their feature checklist is longer. The framework forces operating-model fit to the front.
• 5 questions in order — Q1 narrows the field, Q2-Q3 confirm the technical fit, Q4-Q5 confirm the cultural and infra fit. Skipping Q4 is the most common mistake — vendors get bought without thinking about whether the team speaks the vendor's vocabulary.
• POC two, pick one — running two POCs in parallel for 30-60 days is the only honest way to compare. Reading vendor docs is not a substitute for paging on-call with the platform.
• MTTI + MTTR are the success metrics — not feature count, not check count, not "number of alerts last week." If the platform doesn't compress MTTI / MTTR, the rest of the feature set is incidental.
• Cost — POC cost is mostly engineering time (~3 weeks per platform). Vendor cost during POC is usually waived or steeply discounted. The biggest cost is the year-2 switching cost if the wrong vendor is picked — usually 3-6 months of senior engineer time. Get the year-1 pick right.

ETL
Topic — etl
Platform-selection ETL problems

Practice →

ETL
Topic — etl · medium
Medium ETL design problems

Practice →

---

Cheat sheet — observability picker

• When Monte Carlo is the right default. Big team (>100), dedicated platform team, catalog tool already deployed (Atlan / Alation), 1000+ dbt models, 25+ BI dashboards, field-level lineage required for impact-radius queries. The "safe enterprise default" for 2026.
• When Anomalo wins on operating model. Analytics-engineering-led team (no dedicated platform team), low-config preference, ML-first detection, "show me the rows" UX for non-engineer review. The lowest-config-burden vendor of the four.
• When Bigeye is the right fit. Custom-SQL metric culture already exists (200+ data-quality cron jobs to consolidate), Datadog / Grafana infra observability stack, API-first IaC team, metric-store framing. Open APIs and metric export are the differentiators.
• When Lightup is the right fit. SRE-trained platform team, application SLO platform already in place (Sloth / OpenSLO), stakeholders speak "uptime / SLO / error budget" vocabulary, burn-rate alerts required for tier-1. The SRE-native data observability choice.
• Hybrid recipe — Monte Carlo + Anomalo. Monte Carlo owns lineage + auto-monitor + incident management. Anomalo owns column-level distribution drift on the top 200 tables. Clear ownership: platform team owns MC; analytics team owns Anomalo.
• Hybrid recipe — Bigeye + Lightup. Bigeye owns custom-SQL metrics + Datadog export. Lightup owns tier-1 SLOs + burn-rate alerts. Clear ownership: analytics engineers own Bigeye; platform SRE owns Lightup.
• Five-pillar coverage rule. Tier-1 tables → all five pillars (freshness, volume, distribution, schema, lineage). Tier-2 → drop distribution (compute-expensive). Tier-3 → schema + freshness only. Match the cost to the tier.
• MTTI / MTTR targets — 2026 benchmarks. Tier-1: MTTI < 10 min, MTTR < 1h. Tier-2: MTTI < 30 min, MTTR < 4h. Tier-3: MTTI < 2h, MTTR < 24h. Below these targets, the platform is paying for itself; above them, retune sensitivity or upgrade the tier.
• Alert escalation pattern. Sev3 → Slack #data-warn. Sev2 → Slack #data-warn + PagerDuty low-urgency. Sev1 → PagerDuty page on-call. Sev0 → PagerDuty page on-call + exec notification channel. Auto-resolve when metric returns to baseline.
• Lineage depth ladder. Table-level (Bigeye, Lightup, basic MC) → column-level (Anomalo, MC standard) → field-level (MC premium). Pick the lowest tier that satisfies your incident-triage flow.
• Catalog + observability integration map. Atlan ↔ MC (deep), Alation ↔ MC (deep), DataHub ↔ all four (varying). If you have a catalog, pick the observability vendor with the deepest catalog integration; the marginal cost of "stitching them together" later is large.
• POC measurement template. MTTI (median + p95), MTTR (median + p95), false-positive rate per monitor per week, eng-hours per incident, exec-NPS on dashboard trust. Measure all five during the POC; pick by aggregate, not by best single metric.
• Contract length leverage. 3-year contracts get 20-25% discount on most vendors. Only commit to 3-year if your operating model is stable; for fast-growing teams, 1-year is safer (allows year-2 reconsideration without switching cost).
• Year-1 to year-2 reconsideration triggers. Team grew past 100 (revisit lineage requirement → Monte Carlo). Custom-SQL count grew past 100 (revisit Bigeye). Application SLOs adopted (revisit Lightup). Distribution drift incidents > 1 per week (revisit Anomalo). Each trigger justifies a POC of the missing vendor.

Frequently asked questions

What is data observability and how is it different from data quality testing?

Data observability is the runtime monitoring discipline that surfaces unknown unknowns — distribution drift, freshness misses, schema drift, new categorical values — by continuously sampling live warehouse metadata and metrics against learned statistical baselines. Data quality testing (dbt tests, Great Expectations, dbt-expectations) is the build-time assertion discipline that enforces known contracts — NOT NULL, accepted_values, foreign-key validity — at pipeline-run time. The two are complementary: testing covers the contracts you wrote in the past; observability covers the contracts you didn't know to write. The senior-engineer one-liner: "testing asserts, observability surfaces." Most mature data teams in 2026 run both, with testing in dbt and observability via one of data observability tools — monte carlo data, anomalo, bigeye, or lightup.

Monte Carlo vs Anomalo — which should I pick for my team?

Pick Monte Carlo when your operating model is "big team, dedicated platform team, catalog tool already deployed, field-level lineage required for impact-radius queries." It is the safe enterprise default with the deepest lineage product. Pick Anomalo when your operating model is "analytics-engineering-led, no dedicated platform team, low-config preference, ML-first detection, sample-set inspection UI for non-engineer review." The two vendors have different design bets: Monte Carlo is auto-monitor + lineage + incident management; Anomalo is ML-first + auto-feature engineering + sample-set inspection. Many large enterprises run both as a hybrid — Monte Carlo for lineage and incidents, Anomalo for column-level distribution drift. The data observability comparison interview answer in 2026: lineage depth pushes you to Monte Carlo; ML detection on every column pushes you to Anomalo; both pushes you to a hybrid.

Is data observability the same as data quality monitoring?

Not exactly — data quality monitoring is the broader umbrella; data observability is the runtime-monitoring slice that emphasises the five pillars (freshness, volume, distribution, schema, lineage), incident-first workflow, and SLO / MTTI / MTTR metrics. Older data-quality tools (Great Expectations, Soda, dbt-expectations) focus on assertions at pipeline-run time; modern observability platforms (Monte Carlo, Anomalo, Bigeye, Lightup) focus on continuous runtime monitoring with statistical baselines, ML detection, and first-class incident management. The vocabulary has shifted in 2024–2026 — most platform leads now use "data observability" for the runtime layer and "data quality" for the build-time layer. Both layers compose; neither replaces the other.

How do data SLOs work and which vendor supports them best?

data slo framing treats every data-quality monitor as a service-level objective with three properties: a target (99.9% on-time), an error budget (43 minutes per month at 99.9%), and an MTTI / MTTR target. Alerts are based on error-budget burn rate rather than absolute threshold — a fast burn (5% of monthly budget in 1 hour) pages immediately; a slow burn (25% in 24 hours) is a warn. Lightup is the most SLO-native vendor — burn-rate alerts and severity ladders are first-class. Bigeye supports basic SLO-style targets but doesn't have a burn-rate model. Monte Carlo and Anomalo don't ship SLO framing natively; you'd build it on top of their alert streams. If your team already runs application SLOs in Sloth / OpenSLO and stakeholders ask "what's our data uptime?", Lightup is the lowest-friction adoption.

What is field-level lineage and why does it matter?

Field-level lineage is the dependency graph that tracks how individual column values flow from source through every transformation to every downstream surface (dashboard, ML feature, API). Unlike table-level lineage (this table feeds that table) or column-level lineage (this column feeds that column), field-level lineage captures the transformation on each value — which CASE WHEN, which aggregation, which join condition produced it. It matters because incident triage at scale is dominated by impact-radius queries: "if this column is broken, which dashboards, ML features, and customer-facing APIs are affected?" Without field-level lineage, the answer requires a manual hunt through dbt models and BI metadata; with field-level lineage, it is a single query against the lineage graph. Monte Carlo has the deepest field-level lineage product as of 2026; Anomalo, Bigeye, and Lightup ship table-level or column-level lineage.

How do I pick between data observability platforms without falling into the feature-checklist trap?

Use a 5-question framework ordered by importance: (1) operating model — do you have a platform team? what catalog tool? what infra observability stack? (2) lineage depth — table-level, column-level, or field-level? (3) detection model — auto-monitor, ML-first, custom-SQL, or SLO? (4) stakeholder vocabulary — what language do execs use when they ask about data quality? (5) existing infra fit — what platforms do you already integrate with? Q1 narrows the field to 1-2 candidates; Q2-Q3 confirm technical fit; Q4-Q5 confirm cultural and integration fit. Then POC two vendors in parallel for 30-60 days with 5 tier-1 tables each. Measure MTTI, MTTR, false-positive rate, and eng-hours per incident — pick by aggregate, not by feature count. The most common selection mistake in 2026 is buying on feature checklist and discovering in year 2 that the vendor's vocabulary doesn't match the team's operating model.

Practice on PipeCode

• Drill the ETL practice library → for the freshness / volume / schema / lineage family of probes.
• Rehearse on medium-difficulty ETL problems → when the interviewer wants observability-design depth.
• Sharpen the warehouse axis with the SQL practice library → for metadata + metric queries.
• Layer the optimization library → for the metric-query + warehouse-cost angle.
• Stack the full picture by exploring PipeCode → — Leetcode for Data Engineering, with 450+ pattern-tagged problems.

Lock in observability muscle memory

Vendor docs explain monitors. PipeCode drills explain the decision — when Monte Carlo's lineage justifies the contract, when Anomalo's ML beats thresholds, when Lightup's SLO framing pays back the SRE rigour. Pipecode.ai is Leetcode for Data Engineering — pattern-first practice tuned for the production trade-offs senior data engineers actually face.

Practice ETL problems →
Practice SQL problems →