DEV Community: Blaine Elliott

How Do You Know Your Data Quality Agent Is Right?

Blaine Elliott — Tue, 01 Sep 2026 02:33:03 +0000

Agent verification is the practice of confirming that an AI system's finding traces back to records you can open yourself, instead of accepting prose that merely sounds correct. Every data quality vendor now ships an agent. Almost none of them tell you how to check its work.

That gap is the whole problem. An agent that says "fct_revenue looks stale because an upstream sync failed" is either doing real correlation or generating a sentence with the right shape. From the outside, those two things are identical. They read the same, they arrive in the same Slack channel, and one of them is worthless.

I build one of these agents, so treat what follows as interested but specific. Everything below describes machinery I can point at.

Why is "the agent said so" not an answer?

The failure mode is not that the model lies. It's that the model is fluent about things it half-knows, and fluency is the exact signal humans use to judge competence.

Here's the concrete version. You ask why a table went stale. Behind the scenes the system pulls rows from a few tables: schema change events, alert history, freshness records, metric snapshots, lineage edges. Then it does one of two things. It stuffs those raw rows into a context window and asks a model to write an explanation, or it composes an answer from a structure it built before the model was ever called.

The first approach produces an answer that name-drops events without linking to them. The model saw twelve rows, mentioned four, invented a causal story connecting them, and dropped the eight that didn't fit the narrative. You can't tell which four were real. Neither can the person who forwards the explanation to a stakeholder.

Data engineers already ran this experiment with dbt tests. A test that fires with severity warn produces a line in a log that nobody reads, and everyone treats a green build as a passing build. The lesson was that an unverifiable signal decays into no signal. Agents are the same shape of problem at higher fluency.

So the question worth asking a vendor isn't "how accurate is your agent." Accuracy claims are unfalsifiable without your data. The question is: what can I click on?

What does a verifiable finding actually look like?

The structure we settled on is a capsule: a typed object with a fixed shape that every investigation returns, where each piece of evidence carries the identifier of the row it came from.

{
  "investigation_id": "5f2c1a7e-...",
  "question": "Why is fct_revenue stale?",
  "root_cause_hypothesis": "Schema change on upstream table stg_orders: column_type_changed",
  "confidence": 0.75,
  "triggers": [
    {
      "event_type": "schema_change",
      "confidence": 0.75,
      "timestamp": "2026-08-27T19:04:00Z",
      "summary": "Schema change on upstream table stg_orders: column_type_changed",
      "source": "schema_change",
      "source_id": "9b41d0c2-...",
      "details": { "column_name": "order_total", "is_breaking": true }
    },
    {
      "event_type": "freshness_cascade",
      "confidence": 0.85,
      "summary": "Upstream table raw_orders is also stale (11h since last update)",
      "source": "lineage",
      "source_id": "c7a9e310-..."
    }
  ],
  "consequences": [ "..." ],
  "suggested_fix": "Review the type change on order_total before rerunning the model",
  "open_questions": []
}

The important field is source_id. It's a public UUID for a row that exists in the database, and it resolves through one uniform lookup no matter which of the five sources it came from: intelligence records, schema change events, alerts, metrics, or lineage nodes. In the web agent, each evidence row renders as a link. You click the schema change and land on the schema changes page for that asset, looking at the event itself.

That's the entire trick, and it's less clever than it sounds. The capsule is a typed projection of a row the investigation service already wrote. It doesn't re-derive anything. It doesn't ask a model to summarize. It takes structured output that already existed and gives it a shape you can navigate. We wrote about the mechanics of this in citations for your data incidents.

The consequence matters more than the mechanism. If a claim has no source_id, it didn't come from a row. There's no ambiguity to argue about. Either the evidence points at something, or the answer is prose.

How is that confidence number produced?

Here is where most vendor documentation goes vague, so let me be specific about ours.

Confidence on a piece of evidence is a fixed prior attached to the type of correlation that found it. Not a model's self-assessment, not a learned score. A constant in the code, chosen by a human, per correlation type.

What the correlator found	Window	Confidence	Reasoning
Schema change on this table	48 hours	0.90	Direct, recent, on the object in question
Freshness cascade from an upstream table	current status	0.85	The upstream is measurably stale right now
Schema change on an upstream table	48 hours	0.75	Real, but two hops of lineage away
Code change touching this asset	7 days	file-match score	Scored by how well the changed files map to the asset
Metric anomaly before the alert	24 hours	0.60	Correlated in time, weakest causal link

The overall confidence on the capsule is the confidence of the single strongest event. Nothing more elaborate. The events sort by confidence descending, the top one becomes the root cause hypothesis, and its confidence becomes the capsule's confidence.

We considered having a second model grade the first model's answer, and rejected it. LLM self-grading is documented as overconfident, it costs a call per investigation, and a number that goes up when the model feels good about itself isn't a measurement. Rules are deterministic, free, and I can explain them to a customer in a table, which is what I just did.

What is the confidence number actually measuring?

Read that table again and you'll notice something most vendors would rather you didn't.

Because the capsule's confidence is the top event's fixed prior, a capsule whose strongest signal is a same-table schema change always reads 90%. A capsule whose strongest signal is a metric anomaly always reads 60%. The number doesn't move with how much evidence piled up, and it doesn't move with how likely the hypothesis is to be correct.

It's a category label wearing a percentage sign. It tells you what kind of evidence was found, not the probability that the answer is right.

That's a real limitation and I'd rather say it than have you discover it. The number is still useful, because "this is a direct schema change on your table" and "this is a metric that wobbled in the same 24 hours" are genuinely different qualities of evidence and you should treat them differently. But if you read 90% as "nine times out of ten this is the cause," you're reading something that isn't there.

The UI bands it deliberately, which is an admission of the same thing:

Displayed	Range	What it means
Low	under 40%	Weak or no correlation found; go look yourself
Medium	40% to 70%	Circumstantial, usually timing-based
High	70% and up	Direct evidence on the object or its immediate upstream

Three buckets is about the resolution the underlying signal supports. Any tool showing you 87.3% confidence on a root cause is showing you false precision, and you should ask what produced the digits.

Can you see which detector made the call?

Same principle one layer down. When an anomaly detector says a value is out of range, you should be able to see which detector, what range, and what it expected.

Our metric detection returns the verdict alongside the machinery: whether it used Prophet-based forecasting or fell back to a simple standard deviation threshold, the expected value, the expected range, the anomaly score, and which seasonality patterns it detected. The fallback triggers below 14 historical data points, because forecasting a seasonal model on 9 observations produces confident nonsense.

Sensitivity is a confidence interval, and it maps to a z-score by the normal quantile function rather than by a lookup table someone tuned:

Sensitivity	z-score	Effect
0.80	1.28	Narrow band, more alerts, more noise
0.95	1.96	Default
0.99	2.58	Wide band, fewer alerts, more misses

None of that is proprietary and none of it should be. If a tool won't tell you whether a finding came from a forecast model or a two-line standard deviation check, you can't calibrate how much to trust it, and you'll end up trusting all of it equally. That's how alert fatigue starts. The general shape of these methods is covered in the anomaly detection guide.

The honest framing: a threshold fallback on 11 data points is a rough check, and knowing it's rough is what lets you ignore it correctly.

What happens when the agent is wrong?

It will be wrong. Black Friday, a backfill, a marketing campaign, a schema migration you scheduled yourself. The volume triples and the detector does exactly what it should do, which is flag it, which is exactly what you don't want on the fourth identical alert.

The test isn't whether the agent is wrong. It's whether your disagreement survives.

The mechanism we ship is snapshot exclusion. Mark a data point as expected and it's excluded from the baseline, with the reason, the user, and the timestamp stored alongside it. The next detection run filters those points out before the model sees them.

# Mark a known spike as expected so it stops poisoning the baseline
curl -X PATCH \
  "$ARMOR_API/api/v1/assets/$ASSET_ID/metrics/$METRIC_UUID/snapshots/$SNAPSHOT_ID/exclude?excluded=true&reason=Black+Friday+traffic" \
  -H "Authorization: Bearer $ARMOR_API_KEY"

Two properties make this a real feedback loop rather than a mute button. The exclusion is reversible, so passing excluded=false puts the point back. And the excluded points stay visible on the metric history chart, marked as excluded, so a year from now somebody can see that the November spike was real and deliberately set aside.

Compare that to the common alternative, which is snoozing an alert. Snoozing hides the symptom and leaves the bad point in the baseline, where it widens the expected range and quietly makes the detector worse at its job forever. That's the difference between correcting a system and silencing it.

The question to ask a vendor: when I tell your agent it's wrong, does anything change in how it computes the next answer, or did I just close a notification?

The four-question audit

Here's the framework, and it's short because it needs to survive a vendor call.

Does every claim cite a record I can open? Not a link to a dashboard. A link to the specific row that produced the claim. Ask for a finding with a citation that 404s and see what the UI does.
Do I know which method produced the number? Forecast or threshold, which window, how much history. A tool that hides this is asking for uncalibrated trust.
Do I know what the confidence number measures? Ask what the number would be for two different findings that are both certainly true. If the answer is "different numbers," it's measuring evidence type, not probability. That's fine, as long as they say so.
Does my disagreement change the next answer? Mark something as expected, then check whether the baseline moved.

Score each one. Two points if the tool does it and documents it, one point if it does it and you had to ask, zero if the answer is a deflection.

	Dashboard-era tools	LLM-wrapper tools	Agent-native
Citations on claims	N/A, no claims to cite	Usually absent	The design constraint
Method disclosure	Often good	Hidden behind the model	Returned with the verdict
Confidence semantics	No confidence shown	Model self-report	Fixed priors, published
Feedback changes output	Threshold editing	Chat only	Baseline exclusion

The middle column is where most of the market sits right now, and it's the dangerous one. Dashboard-era tools don't make claims, so they can't hallucinate. A model summarizing your warehouse can, and it does it in complete sentences. We drew the same distinction from a different angle in what separates a real agent from a wrapper and compared specific vendor approaches in data quality agents compared.

Notice that three of the four questions have nothing to do with model quality. A better model doesn't fix an unciteable answer. The verifiability is an architectural property, decided before the model is called, and no amount of prompt engineering adds it later.

How do you run the check yourself?

Don't take a vendor demo's word for it. The demo is on their data, where they know the answers.

The version worth doing is a replay on an incident you already understand. Pick a failure from last quarter where you know the root cause cold. Point the agent at the affected table and ask why it broke. Then grade the answer against what actually happened.

from anomalyarmor import Client

client = Client(api_key="...")

capsule = client.investigations.explain(
    asset_id="fct_revenue",
    question="Why did this table go stale on August 27?",
)

print(capsule.root_cause_hypothesis, capsule.confidence)

for evidence in capsule.triggers:
    # Every row should name a source and an id you can open
    print(f"{evidence.confidence:.2f}  {evidence.source}  {evidence.source_id}")
    print(f"      {evidence.summary}")

Three outcomes and all of them are informative. The agent names the cause you know is right, and every trigger resolves to a real row: that's the good case, and now you know what its evidence looks like when it's correct. The agent names something plausible but wrong, and the triggers still resolve: the correlation was real, the ranking was off, and you've learned that the top hypothesis deserves scrutiny. The agent writes something confident with no citations underneath: you have your answer about the tool.

Run it on five incidents. You'll learn more than any benchmark table will tell you, because the failure modes are specific to your warehouse's shape. A warehouse with 20 tables and no lineage produces thin capsules regardless of how good the agent is, and that's a fact about your setup rather than the software.

Budget an hour for this in any evaluation. If your loaded engineering cost is $80 to $150 an hour, an hour of replay against known incidents is the cheapest procurement diligence available, and it's more predictive than a 30-day trial where nothing breaks.

What this still doesn't prove

Four things this approach doesn't give you, stated plainly, because a post about verification that oversells its own verification would be funny in the wrong way.

A citation proves provenance, not causation. Every trigger in a capsule can resolve to a real row and the hypothesis can still be wrong. The schema change was real, it happened 40 minutes before the failure, and it had nothing to do with it. Citations rule out fabrication. They don't rule out coincidence, and time-correlated evidence is exactly where coincidence lives.

Capsules are computed on demand, not stored. We deliberately don't persist them, which keeps a migration, a retention policy, and a privacy review off the table. The cost is that you can't diff how an explanation changed between Tuesday and Thursday, and you can't build regression tests on answer quality over time. That's a real feature we don't have.

Older investigations degrade. Evidence rows written before the citation fields existed project into capsules with an empty source, so they render as summaries without links. The shape is backward compatible on purpose, which means the guarantee is "new findings cite" rather than "all findings cite." Check the date on anything you're auditing.

A resolvable identifier is a contract, not a guarantee. The registry that maps each evidence source to its lookup exists so citations resolve uniformly. It isn't a substitute for you clicking one. If a cited row was deleted after the investigation ran, the link 404s, and the UI says so rather than pretending otherwise.

The reason to be this specific about limits is that the alternative is the vendor position, which is that the agent is accurate because the vendor says so. That claim is unfalsifiable and everybody knows it, which is why nobody believes it and why the honest version is a competitive advantage.

The agents are going to get better. The verification surface is what determines whether you can tell.

Frequently asked questions

How do I verify an AI data quality finding?

Click through to the underlying record. A verifiable finding carries an identifier for the row that produced each claim, so you can open the schema change event, alert, or metric snapshot and confirm it exists and says what the agent said it says.

What is an evidence capsule?

A typed object returned by an investigation containing the question, a root cause hypothesis, triggering evidence, downstream consequences, a confidence value, and a suggested fix. Each piece of evidence names its source and the UUID of the row it came from.

Should I trust an LLM's confidence score?

No, if the model generated it by grading itself. Self-graded confidence is well documented as overconfident and it moves with the model's fluency rather than with the evidence. Ask how the number is computed before you use it to prioritize anything.

What does a 90% confidence root cause actually mean?

In our implementation, that the strongest correlated event was a schema change on the table itself within the last 48 hours. It's a fixed prior for that category of evidence, not a probability that the hypothesis is correct.

Why use fixed confidence priors instead of a learned score?

They're deterministic, cost nothing to compute, and can be published in a table a customer can argue with. A learned score would need labeled incident outcomes we don't have at volume, and an unexplainable number is worse than a coarse one.

How does an agent avoid hallucinating a data incident?

By composing the answer from structured rows before the model is involved. The correlators query real tables, the capsule is a projection of what they found, and the model's job is limited to writing prose over a structure it can't add rows to.

Can I tell which anomaly detection method fired?

You should be able to. Ours returns the detection method alongside the verdict, either a Prophet-based forecast or a standard deviation fallback, plus the expected value and range. The fallback runs below 14 historical data points.

What happens if I mark an anomaly as expected?

The snapshot is excluded from the baseline with the reason and user recorded, and the next detection run filters it out before computing expectations. It's reversible, and excluded points stay visible on the history chart.

Is excluding a data point the same as snoozing an alert?

No. Snoozing hides the notification and leaves the bad point in the baseline, where it widens the expected range permanently. Excluding removes it from the calculation, which is the correction you actually wanted.

How do I evaluate a data quality agent before buying?

Replay incidents you already understand. Pick five past failures with known causes, ask the agent why each table broke, and grade both the hypothesis and whether every cited record resolves. An hour of this beats a quiet 30-day trial.

Does more context in the prompt make the agent more reliable?

Not in a way you can verify. A larger context window means the model saw more rows, not that it cited the right ones. The reliability comes from the structure you hand it, not the volume.

What can citations not tell me?

Whether the correlation is causal. Every cited row can be real and the hypothesis can still be coincidence, which is most likely when the strongest evidence is something that merely happened nearby in time.

Do investigations get stored so I can compare them over time?

Not in our current implementation. Capsules are computed on demand from rows that persist in their own tables, so there's no history of how an explanation changed. Persisting them is a known follow-up, not a shipped feature.

How Do You Find the Blast Radius of a Broken Table?

Blaine Elliott — Tue, 25 Aug 2026 04:41:40 +0000

Blast radius is the set of downstream tables, models, and dashboards that become unreliable when one upstream table breaks. Lineage is how you compute it, and computing it is the difference between an incident you can triage and forty alerts you have to read.

Most writing about data lineage sells it as documentation. A pretty graph, a governance artifact, something you show an auditor. That framing is why lineage projects stall: nobody maintains documentation that does not pay for itself on a Tuesday afternoon.

Lineage pays for itself during an incident. That is the only argument for it I find convincing, and it is the one nobody makes.

What actually happens when an upstream table breaks?

Here is the shape of the problem, and if you have been on call for a data platform you already know it.

A Fivetran sync fails at 19:04. Nobody notices, because the sync failing is not the thing that pages you. At 02:00 the transformation job runs against stale source data and succeeds, because stale data is still data. At 06:30 the freshness monitors fire. By the time you open Slack there are thirty or forty alerts: raw_orders is stale, stg_orders is stale, fct_revenue is stale, the executive dashboard is stale, the finance extract is stale, three ML feature tables are stale.

Every one of those alerts is true. Not one of them tells you that a single Fivetran connector is the reason.

So you start reading. You open the lineage graph if you have one, or you grep the dbt project if you don't. You work backwards from the most alarming alert to whatever sits upstream of it. Fifteen minutes later you know what you could have known immediately: one connector broke, and everything else is a symptom.

That fifteen minutes is the tax. It gets paid on every multi-table incident, by whoever is on call, usually before coffee.

What does the triage tax actually cost?

Put numbers on the fifteen minutes, because "lineage saves time" is the kind of
claim every vendor makes and nobody quantifies.

Assume a loaded engineering cost of $80 to $150 an hour. Assume a multi-table
incident takes 15 minutes of pure root-cause hunting before any fixing starts,
which matches what I see on warehouses in the low hundreds of tables. Assume
those happen twice a month, which is conservative for a team shipping schema
changes weekly.

	Without lineage	With lineage and health
Alerts to read	30 to 40	1 incident
Time to identify root cause	~15 min	under a minute
Who finds out numbers are bad	whoever opens the dashboard	you tell them first
Cost per incident at $115/hr	~$29	~$2
Cost per year at 24 incidents	~$690	~$48

Six hundred dollars a year is not the argument. The argument is the fourth row.

The money is rounding error against an engineer's salary, and any vendor waving
that number at you is padding a business case. What actually matters is that
without a blast radius you find out your numbers were wrong when a stakeholder
tells you, and with one you tell the stakeholder first. That difference does not
show up in a spreadsheet and it is the entire reason to bother.

Two caveats on the arithmetic. The 15-minute figure is my estimate from watching
teams triage, not a benchmark, and it scales with how many tables you have and
how well you know them. If you have forty tables and wrote all of them, lineage
buys you very little. The value climbs steeply somewhere past the point where one
person can hold the graph in their head.

Why is table-level lineage enough for triage?

There is a persistent belief that lineage is only useful at column granularity. Column-level lineage is genuinely better for impact analysis before a change, when you want to know whether renaming one field breaks anything.

For incident response, table-level lineage does the job. When raw_orders stops updating, every column in it is stale. You do not need to know which columns feed which downstream fields, because the answer is all of them. The question during an incident is not "which fields are affected," it is "what do I have to tell people, and which pipeline do I fix first."

That distinction matters because table-level lineage is dramatically cheaper to obtain. You probably already have it and have not turned it on.

How do you get lineage without a migration?

If you use dbt, your lineage already exists. It is sitting in the manifest file dbt generates every time it compiles.

dbt parse             # fastest, writes target/manifest.json without compiling
# any of these also write it
dbt compile
dbt run

manifest.json contains every model, source, seed, and snapshot in your project, plus the parent and child relationships between them. That is a dependency graph. Upload it and you have lineage.

In AnomalyArmor that means going to the Lineage tab for a database asset and uploading the file. Nodes come back as tables, edges as dbt dependencies, and each node carries the database, schema, and table name so it lines up with what you are actually monitoring.

A few things worth knowing before you do it:

You do not need to change your dbt project. No new macros, no package, no config block. The manifest is a build artifact you already produce.
Sources count. Anything declared in a sources: block shows up as a source node, so the graph extends past the models you wrote to the raw tables they read from.
Re-upload after structural changes. The manifest is a snapshot. Add models and the graph does not know until you upload again. Wiring this into CI after dbt parse is the obvious move.

Wiring the upload into CI is the part that keeps the graph honest:

# .github/workflows/dbt.yml
- name: Parse dbt project
  run: dbt parse --target prod

- name: Upload lineage
  run: |
    curl -sf -X POST \
      "$ARMOR_API/api/v1/assets/$ASSET_ID/lineage/upload" \
      -H "Authorization: Bearer $ARMOR_API_KEY" \
      -F "file=@target/manifest.json"

Run it on merges to main rather than on every pull request. The graph should
reflect what is deployed, not what someone is proposing.

If you do not use dbt, you can build the graph manually, which is worth it for a critical path and not worth it for a warehouse.

What does cascading staleness do?

This is the part that turns a graph into a triage tool.

A lineage graph on its own is structure. It tells you fct_revenue depends on stg_orders. Useful, static, and something you could have read out of the dbt project yourself.

The useful version carries health. Each node shows its own state: healthy, warning, critical, or at risk from upstream. When a node goes stale or critical, every node downstream of it is marked at risk automatically.

That single behavior collapses the forty-alert problem. Instead of reading thirty individual freshness failures, you look at the graph and see one red node with an orange fan spreading out from it. The red node is the thing to fix. The orange nodes are the blast radius, which is what you tell people.

The distinction between "at risk" and "critical" is doing real work here. An at-risk node has not failed a check of its own. It is downstream of something that did, so its data is suspect. Treating those as separate states is what stops the blast radius from looking like forty independent fires.

Node state	What it means	What you do
Healthy	Its own checks pass, nothing upstream is broken	Nothing
Warning	A check is degraded but not failing	Watch it
Critical	Its own check failed	This is probably the root cause
At risk from upstream	Its checks pass, but something upstream is critical	Communicate, do not debug

That last row is the one that saves time. Engineers waste triage minutes debugging tables that are working correctly and reporting bad numbers because their input is bad.

How do alerts stop being a flood?

Lineage plus health gets you a readable graph. The next step is not sending forty messages in the first place.

When a problem cascades through lineage, the downstream alerts collapse into a single incident with a named root cause and a visible blast radius. Slack gets one message rather than forty. Later alerts thread underneath it instead of arriving as new notifications. When every grouped alert clears, the incident auto-resolves on its own.

This is on by default for accounts that have lineage, which is the other reason to upload the manifest. Without a graph there is no way to know that fct_revenue failing and raw_orders failing are the same event.

Full details are in the incident correlation docs.

Manifest lineage or query-log lineage?

There are two ways to build a lineage graph and they fail differently, so it is
worth knowing which one you are buying.

Manifest-based lineage reads your transformation tool's own dependency
declarations. In dbt that is manifest.json. It is exact for everything dbt
manages, because dbt already resolved those dependencies to build your models in
the right order. It is completely blind to anything dbt does not manage.

Query-log lineage parses the SQL your warehouse actually executed and infers
dependencies from it. It catches everything that touched the warehouse regardless
of what tool wrote it, including the analyst's scheduled query and the Python job
nobody documented. It also inherits every ambiguity in SQL parsing: dynamic SQL,
SELECT * through views, and temp table chains all degrade it.

	Manifest	Query log
Accuracy inside dbt	Exact	Inferred
Sees non-dbt writes	No	Yes
Setup cost	Upload a file	Warehouse log access and permissions
Fails by	Omitting what dbt does not manage	Mis-parsing unusual SQL
Freshness	As of last upload	Continuous

We do manifest-based lineage. I am telling you that plainly because the honest
consequence is the one above: if a meaningful share of your tables are written by
something other than dbt, our graph will understate your blast radius, and a
graph that understates blast radius is worse than knowing you do not have one.

The practical answer for most teams is that dbt manages the transformation layer
and the gaps are at the edges: ingestion above it and reverse ETL below it.
Ingestion usually appears anyway, because dbt sources declare the raw tables.
Reverse ETL usually does not, so if a broken table feeds a sync back into
Salesforce, that consequence will not be in the graph and you need to know it
from memory.

Pick based on where your pipelines actually live rather than on which sounds more
sophisticated.

How should you triage with lineage?

Here is the sequence I would actually follow, and it fits on an index card.

1. Find the deepest critical node. Not the loudest alert, the one furthest upstream that is critical rather than at risk. That is your root cause candidate. In a graph with health, this is a visual step rather than an investigation.

2. Confirm it is a cause and not a coincidence. Two unrelated things do break at once occasionally. Check whether the at-risk set actually descends from that node. If half your alerts sit outside its downstream cone, you have two incidents.

3. Read the blast radius before you fix anything. The people who need to know are the owners of the at-risk leaves: the dashboard consumers, the feature store, the finance extract. Telling them at 06:35 that numbers are suspect is worth more than fixing the pipeline ten minutes sooner and telling them at 07:15.

4. Fix the root, then watch the fan clear. If the graph is doing its job, resolving the critical node should walk the at-risk states back to healthy as each downstream check re-runs. If a node stays at risk after its upstream recovers, that node has its own problem and you have found a second incident.

5. Record the time to detection. The gap between the Fivetran failure at 19:04 and the first alert at 06:30 is over eleven hours of undetected staleness. That number, not the fix time, is what you should be trying to shrink. We wrote about measuring it properly in data downtime.

The step most teams skip is 3. It is also the only one that changes how the incident feels to everyone who is not you.

What does lineage not tell you?

I would rather be useful than complete, so here are the limits.

It does not tell you why. The graph says raw_orders is critical. It does not say the Fivetran connector's credentials expired. Lineage narrows the search to one node; something else has to explain that node. Our investigations produce a cited capsule for exactly that gap, where each claim links back to the alert or record behind it.

It only knows what the manifest knows. A pipeline that writes to a table outside dbt is invisible. Reverse ETL, a Python job, a stored procedure, an analyst's scheduled query: none of it is in the graph unless you add it. Lineage graphs are always partial, and treating a partial graph as complete is how you conclude the blast radius is smaller than it is.

It is table-level. If you need to know whether renaming one column breaks a specific downstream field, this will not answer it. It will tell you which tables to go look at.

A stale manifest lies confidently. This is the failure mode I would watch for. A graph built from a three-month-old manifest will happily show you a blast radius that omits every model added since. It looks correct because graphs always look correct. Upload from CI.

Where do schema changes fit?

Freshness is the easy case, because a stale table is obviously stale. Schema changes are the ones that cascade quietly.

A column changing type does not stop a pipeline. Downstream models keep building, dashboards keep rendering, and the numbers are wrong until somebody notices. Lineage plus schema monitoring is what turns that into an alert with a blast radius attached instead of a discovery three weeks later. We covered the detection side in monitoring schema changes, and the dbt-specific version of quiet failure in silent dbt test failures.

Frequently asked questions

What is blast radius in data engineering?

The set of downstream tables, models, and dashboards that become unreliable when one upstream table breaks. It is computed by walking the lineage graph forward from the failed node.

Do I need column-level lineage for incident response?

No. When a table stops updating, every column in it is stale, so table-level granularity answers the triage question. Column-level lineage is more useful for impact analysis before making a change.

How do I get lineage from dbt?

Run dbt parse to produce target/manifest.json, then upload that file on the Lineage tab of your asset. dbt compile and dbt run write it too; dbt parse is fastest because it skips compilation. The manifest already contains every model, source, seed, and the dependencies between them.

Do I have to change my dbt project?

No. The manifest is a build artifact dbt produces during a normal compile. There is no package to install and no config to add.

How often should I re-upload the manifest?

After any structural change, which in practice means wiring it into CI after dbt parse. A stale manifest produces a graph that looks correct while omitting recently added models.

What if I do not use dbt?

You can define lineage manually. It is worth doing for a critical path of a dozen tables and not worth doing for an entire warehouse.

What does "at risk" mean on a node?

Its own checks pass, but something upstream of it is critical, so its data is suspect. It is a signal to communicate rather than to debug, because the node itself is working correctly.

Does the blast radius include dashboards?

It includes whatever is in the graph. Tables and models come from the manifest. BI dashboards appear if they are represented as nodes, otherwise the graph ends at the table that feeds them.

How does lineage reduce alert volume?

Downstream alerts that share a root cause collapse into one incident with a named cause and a visible blast radius, so a cascading failure produces a single Slack message instead of one per affected table.

Does the incident close itself?

Yes. Once every alert grouped under the incident clears, the incident auto-resolves.

What if two unrelated things break at once?

Check whether the at-risk set actually descends from your candidate root cause. Alerts outside that node's downstream cone belong to a separate incident.

Can lineage tell me the cause of a failure?

No. It narrows the search to a node. Explaining that node needs something else: logs, a schema event history, or an investigation that cites the underlying records.

Is a partial lineage graph still useful?

Yes, as long as you know it is partial. The risk is concluding the blast radius is smaller than it really is because a non-dbt pipeline is missing from the graph.

What should I measure to know this is working?

Time to detection, not time to resolution. Shrinking the gap between when a source actually broke and when the first alert fired is where the real hours are.

What about tables written outside dbt?

They will not appear in a manifest-based graph. Ingestion usually shows up anyway
because dbt sources declare the raw tables, but reverse ETL and ad-hoc jobs will
not, so a broken table feeding a Salesforce sync is a consequence you have to
remember rather than read off the graph.

What Is the Best Great Expectations Alternative in 2026?

Blaine Elliott — Mon, 03 Aug 2026 17:42:27 +0000

A Great Expectations alternative is a data quality tool that replaces hand-authored expectation suites with monitors derived from the data itself. The distinction matters because Great Expectations is free to install and costly to keep, and the cost lives in engineering time rather than a line item.

I want to be straight about something before going further. Great Expectations is good software. The project popularized the idea that data deserves assertions the way code deserves tests, and the ODCS-style thinking that a lot of us now take for granted owes it a debt. If you want your data assertions reviewed in a pull request, versioned next to the models they guard, and executed in CI, GX is the right tool and nothing here changes that.

The problem shows up later, and it always shows up the same way.

Why do teams look for a Great Expectations alternative?

Nobody churns off GX in week one. Week one is great. You install it, point it at a table, write a handful of expectations, and watch them pass. The demo works.

The trouble starts around month six, when you have a hundred tables instead of one.

Do the arithmetic on a mid-size warehouse. A hundred monitored tables averaging eight columns is eight hundred columns. If you write even one expectation per column, which is the bare minimum anyone would call coverage, you are authoring and then maintaining eight hundred assertions. That number is not the hard part. The hard part is that it is not a number at all, it is a rate. Every new table adds to it. Every schema change invalidates some of it. Every threshold you set by hand drifts out of date as the data changes underneath it.

Four specific things go wrong, and they compound:

Suite maintenance outruns suite authoring. Writing expect_column_values_to_be_between("amount", 0, 10000) takes a minute. Discovering eight months later that legitimate order values now exceed ten thousand, tracking down which suite that lives in, and updating it takes an afternoon and usually starts with a false alert at 2am.

Coverage stalls where attention stopped. The tables someone cared about in Q1 have expectations. The tables that appeared in Q3 have none. There is no mechanism that notices this, because the mechanism is a person remembering.

Thresholds are guesses that age. A hand-set bound encodes what you believed about the data on the day you wrote it. Data changes. The bound does not.

The runner is your problem. Checkpoints need scheduling. Data docs need hosting. Something has to alert when a suite fails, and by default that something is an exit code.

None of this is a defect in GX. It is what a library does. A library gives you the primitive and leaves the operations to you. The question is whether you want to run that operation.

What does eight hundred expectations cost in engineering time?

Free software with a large time cost is still a cost, so put a number on it.

Assume a loaded engineering cost of $80 to $150 per hour, which is the usual range for a mid-level data engineer in the US once benefits and overhead are counted. Now assume authoring an expectation takes five minutes on average. Some take thirty seconds, some take an hour because you had to go ask what the column means.

Eight hundred expectations at five minutes each is about 67 hours, so roughly $5,400 to $10,000 of engineering time to reach baseline coverage once.

The one-time number is the small one. Maintenance is the real line item. In practice teams spend somewhere between two and eight hours a month per hundred tables on suite upkeep: chasing false alerts from thresholds that drifted, updating suites after schema changes, and adding coverage for new tables. Call it four hours a month at 100 tables. That is 48 hours a year, another $3,800 to $7,200 annually, and it recurs.

Compare that against a published per-table price. A hundred tables at $5 per table per month is $6,000 a year, and nobody on your team spends an afternoon on threshold drift. The comparison people make is "free versus $6,000." The comparison worth making is "$6,000 versus roughly $9,000 of engineering time that could have gone to the pipeline instead."

Two honest caveats on that math. The five-minute figure is my estimate from watching teams do this, not a benchmark, and your maintenance hours depend heavily on how volatile your schemas are. If your warehouse barely changes, GX maintenance is close to free and you should stay. If you ship schema changes weekly, the number is worse than I've written.

What is GX Cloud, and does it solve this?

Great Expectations the company ships a hosted product, GX Cloud, with Developer, Team, and Enterprise tiers. It handles the runner, the scheduling, and the UI, which removes the operational half of the problem.

It does not remove the authoring half. You still describe what "correct" means, expectation by expectation. Hosted execution is a real convenience and it is not the same thing as not writing the suite.

One practical note for anyone evaluating: GX Cloud publishes no pricing on any tier. You will be talking to a salesperson to find out what it costs. That is a common pattern in this category and worth naming, since the reason many people run open-source GX in the first place is that they did not want that conversation. We take the opposite position and publish a per-table number, for the same reason we published our full cost breakdown of the category: if a vendor will not tell you the price before a call, the price is a negotiation and you are the variable.

What should a Great Expectations alternative actually do differently?

Here is the test I would apply, and it has nothing to do with feature checklists. Ask how many assertions you have to write before the tool finds its first real problem.

For GX, the answer is however many you author. For a monitoring platform that profiles your data, the answer should be zero, because the tool has already looked at the table and knows what normal looks like.

That single question sorts the category faster than any comparison grid:

Capability	Great Expectations (OSS)	GX Cloud	AnomalyArmor
Assertions before first finding	However many you write	However many you write	0
Threshold source	Hand-set	Hand-set	Learned from history
New table appears	Unmonitored until authored	Unmonitored until authored	Profiled and monitored
Schema change	Suite breaks or silently passes	Suite breaks or silently passes	Detected and reported
Runner and scheduling	You host it	Hosted	Hosted
Assertions in version control	Yes, native	Partly	Yes, via ODCS contracts
Published price	Free	None published	$5 per table per month

That version-control row is the one people assume they have to give up, and they don't. More on that below, because it is the thing that actually blocks migrations.

How do you migrate off Great Expectations without losing the suites?

This is the real objection, and it is a good one. You have spent a year encoding domain knowledge into expectation suites. "Rip it out and let the AI figure it out" is not a migration plan, it is a request to throw away work.

So we wrote the converter. armor migrate-from great-expectations reads a GX project on disk and emits an ODCS YAML contract:

armor migrate-from great-expectations ./great_expectations/ \
  | armor contract apply --asset <asset-uuid> -f -

The adapter is file-based. It reads great_expectations.yml and the expectation-suite JSON directly, so you do not need the great_expectations package installed to run it. It auto-detects legacy projects and v1 projects using Fluent Datasources.

Two expectations land on native ODCS fields, which means they round-trip through any ODCS-compatible tool rather than being locked to us:

Great Expectations	ODCS field
`expect_column_values_to_not_be_null`	`schema[].properties[].required = true`
`expect_column_values_to_be_unique`	`schema[].properties[].unique = true`

Five more map to validity rules under customProperties.anomalyarmor.validity:

Great Expectations	Rule type	Translation
`expect_column_values_to_be_in_set`	`allowed_values`	`value_set` to `values`
`expect_column_values_to_match_regex`	`regex_match`	`regex` to `pattern`
`expect_column_values_to_be_between`	`range_bounds`	`min_value` / `max_value` to `min` / `max`
`expect_column_value_lengths_to_be_between`	`length_bounds`	to `min_length` / `max_length`
`expect_column_values_to_match_strftime_format`	`format`	`%Y-%m-%d` to `iso_date`

What the converter cannot translate

I would rather tell you this up front than have you discover it mid-migration. These do not convert, and the command reports them as warnings in the summary instead of dropping them silently:

expect_column_pair_* and expect_multicolumn_*, because cross-column logic has no clean single-property home
expect_column_kl_divergence_* and other statistical distribution checks
Custom expectations, including dbt_expectations.* and any user-defined subclass
expect_column_values_to_be_of_type, because type-system translation is genuinely its own problem
Table-scoped expectations like expect_table_row_count_to_be_between

For those, write a custom SQL check after the migration rather than forcing them through a shape that does not fit. A converter that pretended to handle multicolumn logic would be worse than one that tells you it can't.

If your suites live in a database or S3 rather than on disk, export them first:

import great_expectations as gx

context = gx.get_context()
for name in context.list_expectation_suite_names():
    suite = context.get_expectation_suite(name)
    with open(f"expectations/{name}.json", "w") as f:
        f.write(suite.to_json_dict())

The output is an open standard, not our proprietary format. If you leave us in a year, the contract comes with you. That is deliberate, and it is the same reasoning behind open-sourcing our query layer.

Do you lose assertions-as-code by moving off GX?

No, and this is the misconception worth killing.

The choice is usually framed as explicit assertions in git versus a black box that decides for you. That framing is wrong because the two are not mutually exclusive. ODCS contracts are YAML. They live in your repo, they diff in a pull request, and they apply through a CLI you can run in CI.

What changes is what you have to write by hand. Automatic monitoring covers the broad, boring surface: freshness, volume, null rates, distribution shifts, schema changes. Contracts cover the specific domain rules that a profiler could never infer, like "status is one of these five values" or "this identifier matches this regex."

You should be writing the second kind. You should not be hand-writing the first kind for eight hundred columns.

Where does dbt fit?

If your assertions live in dbt tests rather than GX, most of this still applies, with one extra wrinkle: dbt tests fail in ways that don't look like failures. A test with severity: warn fails and still exits zero. A test that only runs in CI never protects production. We covered the five patterns in detail in silent dbt test failures, and the short version is that a green dbt build is weaker evidence than most teams treat it as.

The GX-versus-dbt-tests question is mostly a preference about where assertions live. Neither one profiles your data or notices a table nobody wrote a test for.

How does this compare to the commercial platforms?

If you are evaluating GX alternatives you are probably also looking at the managed observability vendors. Short version, with the caveat that I run one of these companies and you should discount accordingly:

Monte Carlo is the enterprise incumbent, priced accordingly, and sells to a data platform team with a budget line. See the detailed comparison.
Metaplane, now part of Datadog, is the natural fit if you are already deep in Datadog. Comparison here.
Bigeye sits in the enterprise tier with custom quotes. Comparison here.
Soda splits open-source and commercial, with SodaCL as a contract language, which puts it closer to GX philosophically than to the auto-profiling tools.

The honest summary: for a team coming off open-source GX, the deciding factor is rarely the detection algorithms. Those converge. It is whether you can get a price without a sales call and whether the tool covers the tables you forgot about.

How do you run a fair bake-off?

Vendor evaluations in this category usually get decided by demo quality, which measures the vendor's sales engineer rather than the tool. Here is a four-week test that measures the tool instead. Run it against your own warehouse, because a demo dataset is chosen to make detection look easy.

Week 1: baseline in silence. Connect the candidate tool to a real schema and change nothing else. Keep GX running. Do not tune anything, do not create rules. You are measuring what the tool finds with zero configuration, which is the whole claim under test.

Week 2: count the disagreements. Build three lists. Things GX caught that the tool missed. Things the tool caught that GX missed. Things both caught. That third list is the least interesting and it is where most evaluations stop.

Week 3: judge the second list honestly. For every finding the tool surfaced that your suites missed, ask whether it was real and whether you would have wanted to know. A finding that is technically correct and operationally irrelevant is noise, and noise is what kills these tools six months in. Count precision, not volume. A tool that surfaces 40 findings of which 6 matter is worse than one that surfaces 8 of which 6 matter.

Week 4: break something on purpose. Rename a column in a staging table. Delay a load by six hours. Insert a batch of nulls into a column that has never had them. Time how long each system takes to tell you, and note which one tells you what broke downstream rather than only that something broke.

The number that decides it is not detection rate. It is findings per week that changed what someone did. Track that for four weeks and the decision usually makes itself, in whichever direction.

One thing to hold constant: run both against production data, not a sample. Profiling-based tools learn from history, so a tool pointed at two weeks of backfilled test data will underperform what it does against a real table with real seasonality.

Choosing: a short decision rule

Stay on Great Expectations if your assertions are the point. Regulated reporting, contractual data guarantees, and anything where you need to show an auditor the specific rule and its commit history. GX is built for that and does it well.

Move if coverage is the point. If you cannot name every table in your warehouse from memory, and if new tables appear faster than someone writes suites for them, you are running a race you lose slowly.

Most teams need both, which is why the migration path exists rather than a replacement pitch.

Frequently asked questions

Is Great Expectations still maintained in 2026?

Yes. The project is active and GX Cloud is the company's commercial offering. Choosing an alternative is a decision about operating model, not about abandonment risk.

What does Great Expectations cost?

The open-source library is free. The cost is engineering time to author and maintain suites, plus hosting the runner and data docs. GX Cloud has Developer, Team, and Enterprise tiers, and publishes no pricing for any of them.

How many expectations does a typical warehouse need?

At one expectation per column, a hundred tables averaging eight columns is eight hundred. Real suites usually run several per column on important tables, so the practical number is higher. The count grows with your warehouse whether or not anyone is writing them.

Can I convert my expectation suites automatically?

Partly. armor migrate-from great-expectations converts seven common expectation types into ODCS contract fields and validity rules. Cross-column, statistical, custom, type, and table-scoped expectations are reported as warnings and need manual handling.

Do I need Great Expectations installed to run the migration?

No. The adapter reads great_expectations.yml and the suite JSON files from disk. If your suites live in a non-default store, export them to expectations/ first.

Does the migration support GX v1 projects?

Yes. The adapter auto-detects legacy projects and v1 projects using Fluent Datasources, and pulls per-column type hints from the fluent_datasources block into physicalType.

Will I lose my assertions in version control?

No. The output is ODCS YAML, an open standard. It lives in your repo, diffs in review, and applies through the CLI in CI.

What happens to custom expectations like dbt_expectations?

They surface as warnings in the migration summary rather than being silently dropped. Rewrite them as custom SQL checks after migrating.

Is ODCS a proprietary format?

No. The Open Data Contract Standard is a public specification. Contracts written against it work with other ODCS tooling, which is the point of using it instead of inventing our own YAML.

What is the difference between data testing and data monitoring?

Testing asserts what you already know should be true. Monitoring detects what you did not think to assert. Suites are testing, profiling is monitoring, and most teams discover they needed the second one during an incident.

Does automatic monitoring replace domain rules?

No. Profiling cannot infer that status must be one of five values or that an identifier follows a particular regex. Write those as contracts. Do not hand-write freshness and null-rate checks for every column.

How long does a migration take?

The conversion itself is one command. The real work is triaging the warnings for expectations that did not map, which scales with how many custom and cross-column expectations you wrote.

Can I run both during a transition?

Yes, and you should. Keep GX running in CI while monitoring runs against production, compare what each catches for a few weeks, and decide with evidence instead of a migration-day cutover.

What about schema changes breaking my suites?

That is the failure mode worth planning for. A renamed column breaks a suite loudly, which is fine. A new column arrives with no expectation at all, which is the silent case. We wrote up how to monitor schema changes separately.

Data Quality Agents Compared: Monte Carlo, Acceldata, and the Agent-Native Approach

Blaine Elliott — Sun, 02 Aug 2026 05:06:46 +0000

Every data observability vendor shipped a "data quality agent" in the last year, and they are not the same thing. Monte Carlo added Observability Agents to its platform in April. Acceldata sells a Data Quality Agent inside its agentic data management suite. Datadog is folding Metaplane into its stack after the 2025 acquisition. On paper they all promise the same thing: an agent that monitors, investigates, and resolves data issues. In practice they split into two camps, and the split matters more than any feature checklist.

The short version: the incumbents added an agent to a dashboard you already pay a lot for. The agent-native approach makes the agent the product and drops the dashboard. Which one you want depends on whether you already run one of these platforms and how much you care about verifying what the agent tells you.

The two camps

Agent-as-a-feature. Monte Carlo, Acceldata, and Metaplane are mature observability platforms. Their agents are a new capability layered on top of the existing product: the same warehouse of alerts, dashboards, and lineage, now with an LLM that can summarize an incident or suggest a root cause. You still log into their console. The agent is a helper inside a tool you were already using.

Agent-native. The agent is the whole product. There's no separate dashboard to live in. The agent watches the warehouse, investigates issues, and answers you where you work, usually through your existing AI assistant or a chat surface. AnomalyArmor is built this way. So, increasingly, are newer entrants like DQLabs' Prizm.

Neither camp is automatically right. If your org has standardized on Monte Carlo and has three engineers who live in its console, its agent is a reasonable add-on. If you're a smaller team that doesn't want to run and pay for a heavy platform just to get an agent, the native approach is a better fit.

How the agent approaches compare

	Monte Carlo Agents	Acceldata	Metaplane / Datadog	Agent-native (AnomalyArmor)
What the agent is	Feature on the platform	Feature in the suite	Feature, folding into Datadog	The entire product
Where you work	Their console	Their console	Datadog	Your AI assistant, or a chat surface
Investigation output	Prose summary + RCA	Automated find/fix	Summaries	Cited evidence, every claim links to a record
Setup	Platform onboarding	Platform onboarding	Datadog integration	Connect warehouse, agent profiles it
Pricing shape	Enterprise, seat + volume	Enterprise	Aligning to Datadog	$5 per table
Best for	Existing MC customers	Large data estates	Datadog shops	Teams who want the agent, not a platform

A few honest caveats about this table. The incumbent platforms are genuinely more mature on breadth: years of connectors, enterprise controls, and scale that a newer tool hasn't matched. If you need to monitor thousands of tables across a dozen sources with fine-grained RBAC today, that maturity is worth something. The agent-native tools trade that breadth for a simpler surface and, in the good ones, better verifiability.

The question that actually separates them

Feature tables blur together. The question that doesn't: when the agent tells you something, can you check it?

This is the thing to press every vendor on. An agent that produces a confident paragraph explaining your incident is only useful if you can verify the paragraph. Otherwise you've replaced "I don't know what broke" with "a model told me what broke and I can't tell if it's right," which is not an upgrade. The failure mode of agentic analysis isn't that it's dumb. It's that it's plausible and wrong, and plausible-and-wrong is expensive when you act on it during an incident.

Our answer is that every claim an AnomalyArmor investigation makes is a citation. When it says a freshness check failed at 9:04 against a schema change from the night before, both facts are links to the actual records, and you click through to confirm. We wrote up how that works in Citations for Your Data Incidents. Some of the incumbents are moving toward this; most still hand you prose. Ask for a live investigation in the demo and see whether you can click into the evidence, or whether you're being asked to trust the narration.

Where cost comes in

The agent-as-a-feature tools carry the pricing of the platform underneath them. That's fine at enterprise scale and painful for a mid-sized team that just wants an agent watching its warehouse. Monte Carlo and Acceldata are enterprise contracts. Metaplane's pricing is expected to drift toward Datadog's now that it's part of that stack.

AnomalyArmor is $5 per table, which is roughly half of what Metaplane charged before the acquisition. The reason we can do that is the same reason we're agent-native: there's no heavy platform to fund, so the agent doesn't have to carry a platform's price tag. We went deeper on the math in How Much Does Data Observability Cost in 2026. If budget is the thing keeping you from adding monitoring at all, that gap is the whole point.

How to choose

You already run Monte Carlo or Acceldata and it's working. Use their agent. You've paid for the platform; the agent is a sensible extension of it.
You're standardized on Datadog. Metaplane inside Datadog will be the path of least resistance, and consolidation has real value.
You want an agent without buying a platform, and you care about verifying its output. That's the agent-native lane. Start with a tool where the agent is the product and every finding is checkable.
You're a smaller or cost-sensitive team. The per-table pricing of a native tool will almost always beat an enterprise platform contract.

There's no universally correct answer here, and anyone who tells you their tool wins every scenario is selling. Pick based on what you already run and how much verifiability matters to you.

FAQ

Are these agents actually autonomous, or just chatbots?

It varies, and it's worth testing. The stronger ones do real correlation and root-cause work across sources. The weaker ones summarize alerts you could already see. The test is whether the agent tells you something you didn't already know from the alert, and whether it shows the evidence.

Can I run a data quality agent without replacing my current stack?

Usually yes. Most agents, native or bolted-on, work off warehouse metadata and sit alongside your existing dbt tests and pipelines. You don't have to rip anything out to try one.

Is agent-native less capable than the big platforms?

On raw breadth of connectors and enterprise controls, the incumbents are ahead, and it's fair to say so. On verifiability and time-to-value for a normal-sized team, the native tools are often better. Match the choice to your scale.

What about false positives?

Every agent risks over-flagging, regardless of camp. Evaluate whether it learns per-table baselines and whether you can tune sensitivity. That matters more than which logo is on it.

AnomalyArmor is an agent-native data quality tool for Snowflake and Databricks, priced at $5 per table and currently in private beta. If you want an agent whose findings you can actually verify, reach out and we'll get you access.

What Is a Data Quality Agent?

Blaine Elliott — Mon, 13 Jul 2026 17:00:34 +0000

A data quality agent is software that continuously monitors your data, detects problems, investigates the likely cause, and explains what happened in plain language, without a human writing rules or watching a dashboard. It's the agentic version of data quality monitoring. Instead of maintaining a static set of tests and staring at charts, you let an agent watch the warehouse, decide what's worth flagging, and do the first pass of the investigation for you.

That's the short definition. The longer answer matters, because "agent" is getting stapled onto every data tool this year, and most of what's branded that way is a chatbot bolted to an old dashboard.

What a data quality agent actually does

Strip away the marketing and an agent runs a loop. It observes the warehouse, detects something off, investigates it, explains the finding with evidence, and then alerts or acts. Then it does it again, continuously, without being asked.

Observe. It watches schema, freshness, row counts, distributions, lineage, and job history across your tables. Not a table you pointed it at once. All of them, on an ongoing basis.
Detect. It learns what normal looks like for each table and flags deviations: a column that changed type, a load that's twelve hours late, a null rate that tripled overnight.
Investigate. This is the part that separates an agent from an alert. When something breaks, it correlates across sources to find the likely cause: the upstream schema change, the failed sync, the lineage path to the affected dashboard.
Explain. It writes up what it found in language a human can read, ideally with links back to the exact records so you can check its work.
Alert or act. It tells the right person, or, in narrow cases you've approved, takes an action.

The rule-based tools most teams run today stop at step two. They detect and they alert. Steps three and four, the investigation and the explanation, land on a data engineer at 7am. The agent's whole point is to do that pre-work before you wake up.

Data quality agent vs traditional data quality monitoring

The difference isn't cosmetic. It changes who does the work.

	Rule-based monitoring	Data quality agent
Setup	You write and maintain tests per table	It learns each table's normal behavior
Coverage	Only what you remembered to write a rule for	The whole warehouse, including tables you forgot
When something breaks	You get an alert	You get an alert, a likely cause, and the evidence
Investigation	Human, manual, from scratch each time	Done by the agent as a first pass
Output	A red check and a threshold	A plain-language explanation you can verify
Failure mode	Silent gaps where no rule exists	Over-flagging if the agent isn't tuned

Rule-based testing isn't wrong. dbt tests and Great Expectations catch real problems, and they're precise about the things you thought to encode. The trouble is everything you didn't think to encode. Nobody writes a test for the column that doesn't exist yet, or the freshness SLA on the table a teammate added last quarter. Coverage tracks your memory, and your memory has holes. An agent's pitch is that it covers the holes.

Why the category showed up in 2026

Two years ago "data quality agent" wasn't a phrase anyone used. Now the whole market is shipping one. Monte Carlo launched its first Observability Agents in April, split into a monitoring agent and a troubleshooting agent that does root-cause analysis. Acceldata sells a Data Quality Agent that promises to find, fix, and prevent issues. Actian shipped a suite of observability agents for detection, diagnosis, and routing. When the incumbents all move at once, the category is real.

The reason is that the underlying models finally got good enough to do the investigation step. Correlating a freshness failure with an upstream schema change and a broken lineage edge used to require a human who knew the warehouse. A model with the right context can now do the obvious 80% of that reasoning, which is exactly the part that eats a data engineer's morning. The bottleneck was never detection. It was the tedious middle work between an alert firing and a human understanding it.

What separates a real agent from a wrapper

Here's the position: most "agents" being sold are a language model draped over a product that already existed, and you can tell the difference by asking three questions.

Can it show its evidence? An agent that hands you a confident paragraph with no links is worse than a dumb alert, because now you have to trust prose you can't check. A real agent cites the row. When it says "the upstream sync failed at 19:04," you should be able to click through to that exact record. We built our investigation output, EvidenceCapsule, specifically so every claim points back to a verifiable record, because an investigation you can't audit isn't worth much.

Does it maintain memory, or re-derive everything each time? A good agent builds a persistent, structured picture of your warehouse and keeps it current, so it isn't relearning your schema on every question. We treat that knowledge base as a living document the agent maintains rather than a pile of chunks it re-embeds each time.

Does it live where you work, or in one more tab? The agents bolted onto legacy platforms still make you go to their dashboard. The more useful shape is one that answers from inside the AI assistant you already use, so the investigation happens where you're already working.

If a tool fails those three, it's a wrapper. It'll demo well and frustrate you by week two.

Data quality agent FAQ

Is a data quality agent the same as data observability?

Not quite. Data observability is the broader practice of knowing the state of your data across freshness, schema, volume, quality, and lineage. A data quality agent is a way of delivering that: an autonomous system that does the watching and the first-pass investigation instead of a dashboard you monitor. The agent is the how, observability is the what.

Does it replace dbt tests?

No, and be suspicious of anyone who says it does. Explicit tests are great for the specific invariants you know matter, like a primary key that must be unique. An agent covers the long tail you never wrote tests for and does the investigation when something fails. Run both. They solve different halves of the problem.

Will it flood me with false positives?

That's the real risk, and it's the honest downside of the category. An agent that flags everything is as useless as no monitoring. The thing to evaluate is whether it learns each table's baseline and whether you can tune its sensitivity, not whether the demo looked slick.

Do I need to write rules to start?

With a well-built agent, no. It should profile your tables, learn their normal behavior, and start watching without you authoring a test suite first. Writing rules is optional refinement, not a prerequisite.

What data does it need?

Metadata, mostly: schema, load times, row counts, distributions, and lineage. A good agent works off the information your warehouse already exposes, so it doesn't need to copy your actual rows to do its job.

AnomalyArmor is a data quality agent for Snowflake and Databricks teams, currently in private beta. If you want it watching your warehouse instead of another dashboard you have to remember to check, reach out and we'll get you access.

OKF vs RAG for Data Warehouse Context: Why We Chose a Living Knowledge Base

Blaine Elliott — Tue, 07 Jul 2026 16:50:21 +0000

If you want an AI agent to answer questions about your data warehouse, you have two ways to give it context. You can embed your warehouse metadata into a vector database and retrieve chunks at query time, which is RAG. Or you can maintain a curated knowledge base the agent reads directly, which is what Google's new Open Knowledge Format (OKF) standardizes. For stable, structured knowledge about how a warehouse actually works, the second approach wins, and it's the one AnomalyArmor has used since late 2025.

That's not a prediction about where the industry is heading. It's a description of a bet we already made and shipped, months before Google gave the pattern a name.

What is the Open Knowledge Format (OKF)?

OKF is a spec that Google Cloud published on June 12, 2026. Sam McVeety and Amir Hormati wrote it, it's Apache 2.0, and it's currently a v0.1 draft. The format itself is deliberately boring: a folder of Markdown files, each with YAML frontmatter and a required type field, where every file represents one "concept" your organization knows about. APIs, metrics, runbooks, system behavior. An agent reads those files directly. No embedding step, no vector search, no re-ranking.

It's a formalization of what Andrej Karpathy called the "LLM Wiki" pattern: instead of pointing an agent at raw documents and asking it to re-derive the same facts on every query, you build a living wiki that the agent (and humans) read and update over time. The wiki gets more useful as it accumulates knowledge. A pile of PDFs behind a vector index does not.

OKF vs RAG: the actual difference

The two aren't competitors so much as answers to different questions. RAG is built for "I know the answer is somewhere in this large corpus, go find it." OKF is built for "here is what we already know, stated plainly, go read it."

	RAG	OKF (and OKF-style KBs)
Source shape	Large, changing document corpus	Curated, structured, stable knowledge
Access pattern	Embed, retrieve, re-rank at query time	Direct read of the relevant file
Latency	Embedding + vector search + re-rank	File read
Hallucination surface	Model reconstructs an answer from retrieved chunks	Model reads a fact that's already written down
Best at	Unstructured search across many documents	Known facts about how your systems work
Failure mode	Retrieves the wrong chunk, or a stale one	KB goes stale if nothing keeps it fresh

The last row is the whole game. RAG's hard problem is retrieval quality. A knowledge base's hard problem is staleness. Pick the approach whose hard problem you can actually solve for your data.

For a data warehouse, staleness is the solvable one, because we already know the moment a fact changes.

How AnomalyArmor's knowledge base works

AnomalyArmor builds a markdown knowledge base for every monitored table. Not one global wiki, one KB per asset, assembled from nine domain generators that each write a section:

Schema drift: column changes over the last 90 days, bucketed by time
Freshness: update patterns, SLA violations, the schedule we've learned for the table
Lineage: upstream and downstream dependencies, dbt-backed
Data quality: unified metrics and validity checks
Alert rules: what's configured, what fired, how it resolved
Tags, job history, git changes, and an LLM-written summary of business and technical context

Each generator produces plain markdown. Stitched together, that's a readable document describing everything the system knows about one table. When you ask "why is orders stale," the agent reads that KB, not a set of vector-retrieved fragments it has to reassemble.

Here's the part that makes it a "living" knowledge base rather than a stale one. The KB regenerates lazily, and only when it's actually out of date. Every domain's KB carries a timestamp. When the underlying feature data changes (a new schema diff lands, a freshness check fires, lineage shifts), that domain is marked stale. The next time anything reads the KB, the stale domain regenerates from current data and the fresh domains are served as-is. A table whose schema hasn't moved in a month doesn't get its schema section rebuilt on every question. A table that just broke does.

That's the difference between our version and a hand-maintained wiki. Nobody writes these files. The warehouse's own monitoring signals write them, and staleness detection keeps them honest. It's OKF's shape (curated markdown, read directly, growing more useful over time) fed by a monitoring pipeline instead of a human editor.

Why we didn't use RAG for this

We tried the obvious thing first, because everyone does. Dump the schema, the recent changes, the lineage edges, and the alerts into a vector store, retrieve the relevant bits per question, hand them to the model. Three problems showed up fast.

The chunks lie about structure. A foreign key relationship, a lineage edge, a "this column changed type three days ago" fact: these are relationships, and vector retrieval flattens them into text fragments that lose the connections. The model then guesses at structure it should have been handed directly.

Retrieval adds a failure mode you can't see. When the agent gives a wrong answer, you can't tell whether the model reasoned badly or the retriever fetched the wrong chunk. That's two suspects for every bug. Reading a KB directly removes one of them.

We already know when facts change, so retrieval is wasted work. RAG earns its cost when you can't predict what's relevant. But we run the monitoring. We know the instant a schema drifts or a table goes stale. Paying the embed-and-search tax on every query, to rediscover facts we already detected, is effort spent solving a problem we don't have.

None of that means RAG is bad. It means RAG is the wrong tool for structured knowledge you already track.

Where RAG still fits

If your context lives in a big, messy, unpredictable pile (support tickets, Slack history, a decade of Confluence, arbitrary PDFs) RAG is the right call and a KB is the wrong one. You can't curate what you can't predict, and you shouldn't try to hand-shape a corpus that changes faster than you can read it. OKF's own authors are explicit that the two are complementary. OKF handles the known knowns. RAG handles "it's in there somewhere."

Our own system isn't RAG-free either. There's still a projection step that budgets tokens across domains before a call, because even a curated KB can outgrow a context window. The point isn't purity. The point is that the knowledge layer is a living KB, not a vector index, and that choice is why an AnomalyArmor investigation can cite the exact row behind a claim instead of gesturing at a retrieved fragment. That citation layer, EvidenceCapsule, only works because the knowledge underneath it is structured and addressable in the first place.

The takeaway

Before you reach for a vector database to give an AI agent context about your data, ask one question: do you already know when the underlying facts change? If yes, you don't need retrieval, you need a knowledge base that updates itself the moment they do. Google just standardized that pattern as OKF. For data-warehouse context specifically, it's the right shape, and it has been since before it had a name.

OKF vs RAG FAQ

Is OKF a replacement for RAG?

No, and its authors don't claim it is. OKF is for stable, curated, structured knowledge that an agent reads directly. RAG is for searching large, changing, unstructured corpora. Most real systems use both: a knowledge base for what you know, retrieval for what you have to go find.

Does AnomalyArmor implement the OKF spec literally?

No. We built a variation of the same pattern in late 2025, before OKF existed. Our knowledge base is per-asset markdown, generated from monitoring signals and refreshed on staleness, rather than the exact file layout OKF's v0.1 draft specifies. Same idea, different plumbing, and ours predates the spec.

What makes a knowledge base "living" instead of stale?

Something has to detect when a fact changes and rewrite the relevant part. In a human wiki, that's an editor. In AnomalyArmor, it's staleness detection wired to the monitoring pipeline: when a schema diff, freshness event, or lineage change lands, the matching KB section is marked stale and regenerated on next read. Nothing rebuilds until the underlying data actually moves.

Why is direct read better than retrieval for warehouse metadata?

Because warehouse structure is relational, and retrieval flattens relationships into disconnected text chunks. Reading a curated KB hands the model the structure directly. It also removes retrieval as a hidden failure mode, so when an answer is wrong you have one suspect instead of two.

Does this only work with your own data?

The knowledge base is built from the metadata AnomalyArmor already collects from your warehouse and integrations. It needs no extra instrumentation beyond the existing connection.

AnomalyArmor is in private beta. If you want to see a self-updating knowledge base answer questions about your own warehouse, reach out and we'll get you access.

Citations for Your Data Incidents: Introducing EvidenceCapsule

Blaine Elliott — Mon, 29 Jun 2026 17:00:40 +0000

When an AI agent answers "why is orders stale this morning?" you get one of two things. Either the agent dumps every row it touched into the response (token-wasteful, hard to read, no audit trail), or it writes a confident prose answer that name-drops events without telling you which row in which table it came from. The second mode is faster to read and structurally untrustworthy. You cannot click through to verify a single claim.

EvidenceCapsule is the typed investigation output we shipped to fix that. Every piece of evidence the agent surfaces during an investigation is a citation: a typed row that points back to the actual record (alert, schema change, metric, lineage edge, intelligence note) in your AnomalyArmor instance, with a clickable deep link in the UI and a stable JSON shape for the SDK and the Claude Code skill.

The problem this is solving

Investigations in a data warehouse pull from a half-dozen sources at once. A real "why is this stale" answer has to consider:

The freshness check that fired and when
Schema changes on the upstream table or its dependencies in the last few days
Alerts on related assets that may be the actual root cause
Recent metric anomalies that point at the same window
Lineage edges showing which downstream consumers depend on this asset

An agent that does this well needs all of those rows in context. An agent that does this trustably needs to tell you, claim by claim, which row supported which conclusion. Without that, the response is structurally indistinguishable from a hallucination, even when it is technically correct.

This is the same problem as a research paper without citations. The argument might be right, but you cannot check the argument because you cannot find the underlying source.

What an EvidenceCapsule looks like

Every investigation returns a capsule with five fields plus the question that was asked:

Field	What it contains	Example
`question`	The natural-language question that triggered the investigation	"Why is the orders table 14 hours stale?"
`root_cause`	A concise hypothesis grounded in the cited evidence	"Upstream Fivetran sync for `raw.orders` failed at 19:04 yesterday and has not recovered"
`triggers`	Typed evidence rows that point at the precipitating events	Schema change on `raw.orders.customer_id`, alert fired at 9:04
`consequences`	Typed evidence rows for what is downstream-affected	Three dashboards consuming `fct_orders` show stale data
`confidence`	A numeric confidence (0.0 to 1.0) aggregated from the underlying correlators	0.82
`open_questions`	Things the agent could not resolve and worth a human looking at	"Is the Fivetran credential expired or did the upstream API change?"

Each evidence row in triggers and consequences is a typed object with a summary, a source (alert, schema change, metric, lineage edge, intelligence note), a UUID pointing at the actual record, the time it was observed, and the structured fields the correlator pulled. In the UI, the summary becomes a clickable link that takes you directly to the alert detail, the schema diff, the metric chart, or the asset page.

A real capsule, returned by the SDK or rendered in the agent UI, looks like this:

question: "Why is the orders table 14 hours stale?"
root_cause: >
  Upstream Fivetran sync for raw.orders failed at 19:04 yesterday
  and has not recovered. Downstream fct_orders built at 06:00 today
  against stale source data.
confidence: 0.82

triggers:
  - source: alert
    source_id: 8c40f984-1a7c-4576-bb8b-eb442ce14965
    summary: "Freshness check failed on raw.orders (14h since last update, SLA 6h)"
    observed_at: 2026-05-24T09:04:11Z
    fields:
      asset: raw.orders
      expected_freshness_hours: 6
      actual_freshness_hours: 14
  - source: schema_change
    source_id: c1a908ef-3e32-45b7-af3d-74c9ab2a9605
    summary: "Column customer_id type changed from BIGINT to VARCHAR on raw.orders"
    observed_at: 2026-05-23T14:31:08Z
    fields:
      asset: raw.orders
      column: customer_id
      change_type: type_changed
      old_type: BIGINT
      new_type: VARCHAR

consequences:
  - source: lineage
    source_id: 73dbca36-a05f-4d71-97ff-60b1dd054b5c
    summary: "exec_revenue dashboard depends on fct_orders, last refreshed 07:15 on stale data"
    observed_at: 2026-05-24T07:15:00Z
    fields:
      consumer: exec_revenue
      consumer_type: dashboard
      depth: 2
  - source: lineage
    source_id: 6a1286f7-ea8c-a400-018e-9d00f3f3b20f
    summary: "finance_daily dashboard depends on fct_orders, last refreshed 07:45 on stale data"
    observed_at: 2026-05-24T07:45:00Z
    fields:
      consumer: finance_daily
      consumer_type: dashboard
      depth: 2

open_questions:
  - "Is the Fivetran credential expired or did the upstream API change?"
  - "Should fct_orders be paused until raw.orders recovers?"

Every source_id resolves to a real record in the AnomalyArmor instance. The UI renders each evidence row as a citation chip linking to the corresponding detail page. The SDK exposes the same shape as typed Pydantic models you can iterate over directly.

You read the prose answer. If a claim looks suspicious, you click the link next to it. Either the record supports the claim or it does not. There is no third option, which is the whole point.

Why this changes how trust works with AI investigations

The standard objection to LLM-driven analysis in production data systems is "I cannot tell when it is wrong." That objection is correct as a general statement, and the usual response (better prompts, more guardrails, careful evals) is real but incomplete. The structural fix is not making the model better at not hallucinating; it is removing the path by which a hallucination can hide.

A capsule with citations cannot hide a hallucinated claim, because the claim is either backed by a row that exists or it is not. The link either resolves to a real record or it 404s. A reviewer (a data engineer, an on-call rotation, a manager reading a postmortem) can spot-check three citations in 30 seconds and develop calibrated trust about whether the whole capsule is reliable.

That is a different shape of trust than "I read the answer and it sounded right." It is closer to how engineers already trust SQL: not because the database always returns the right answer, but because they can read the query and the table and verify.

How this works across the surfaces you actually use it from

EvidenceCapsule is the same shape whether you are in the web UI, asking the SDK from a notebook, or running the Claude Code skill from your terminal. The mental model does not change between contexts.

In the web agent. Ask "why is orders stale" in the chat. The response is the agent's prose answer plus a rendered capsule below it. Each evidence row shows up as a citation chip: schema-change icon, "Column customer_id type changed on raw.orders (2026-05-23 14:31)", clickable. The agent's prose narrates the case; the capsule gives you the receipts.

In the SDK. A new client.investigations resource has two methods. get(id) returns the capsule for an existing investigation. explain(asset_id, question) runs an ad-hoc investigation against a single asset and returns the capsule directly. Both return typed EvidenceCapsule and Evidence models you can iterate over, log, or feed into your own downstream processing.

from anomalyarmor import AnomalyArmor

client = AnomalyArmor(api_key=...)
capsule = client.investigations.explain(
    asset_id="abc-123",
    question="Why is this table 14 hours stale?",
)
print(capsule.root_cause)
for evidence in capsule.triggers:
    print(f"  - {evidence.summary} ({evidence.source}: {evidence.source_id})")

In the Claude Code skill. The armor-investigate skill used to be a five-step recipe: list investigations, get one, pull alerts, pull schema changes, stitch a prose answer. Each step was an SDK call the model had to remember to make in the right order. With EvidenceCapsule, the skill is one typed call. The model asks the SDK for an explanation, the SDK returns a capsule, the model renders it. The collapse from five stitched calls to one is the highest-leverage change in this whole feature.

Why not just give the LLM more context and hope for the best?

A reasonable question, since "stuff more rows into the prompt" is the default reflex. Three concrete reasons.

Cost. Investigation context can easily be hundreds of rows once you include alerts, schema events, metrics, and lineage. Putting all of that in the prompt every time pays the token cost on every invocation. Building the capsule on the backend before the LLM call gives the model the minimal projection it needs and keeps token usage predictable.

Determinism. The capsule shape is the same every time. A prose answer's structure depends on the model's mood, the temperature setting, and the exact phrasing of the question. Downstream consumers (logging, eval harnesses, incident-ticket exports, the frontend renderer) cannot reliably parse free-form prose. They can reliably parse a typed schema.

Verifiability. This is the one that matters most. A flat blob of rows in the prompt does not constrain the model's output to those rows. The model can still confabulate around the data it was given. A typed capsule with source_id fields forces the output to be grounded in specific records that either exist or do not. The verifiability is structural, not behavioral.

What this unlocks next

Capsule-as-output is the substrate for several things we could not build before. Capsule diffing across runs (did the same investigation produce a different answer this week than last week?). Persistent capsules for incident postmortems (a stable record of what the agent saw and concluded, not just what it said). Eval harnesses that compare agent capsules against a ground-truth set on the same data. Export-to-Jira or export-to-Linear flows that turn an investigation into an actionable ticket with all the evidence pre-attached.

None of those are shipping today. They become buildable because the shape exists.

How to try it

EvidenceCapsule shipped to production. If you have an existing AnomalyArmor instance:

Web agent: ask any investigation question in the chat. The capsule renders below the prose answer.
SDK: upgrade to the latest Python SDK and call client.investigations.explain(asset_id, question).
Claude Code skill: update armor-investigate to the latest version and use it the same way you always have. The output shape is now structured JSON.

For background on why we are building observability tooling that lives inside your AI assistant rather than in a separate dashboard, see a data observability tool that works from inside your AI assistant. For how AI-native workflows change incident response in practice, see how AI-native data observability changes incident response.

EvidenceCapsule FAQ

Is this just RAG?

No. RAG (retrieval-augmented generation) is a pattern for retrieving relevant context and stuffing it into the prompt. EvidenceCapsule is a typed output schema with citations to the retrieved records. They are complementary: the agent retrieves relevant rows the same way it always did, then produces a capsule that lets you verify each claim against those rows. RAG is the retrieval side. EvidenceCapsule is the auditability side.

Does the LLM still write the prose answer?

Yes. The capsule is a structured projection. The prose narration on top is still generated by the model, grounded in the capsule. The difference is that any claim in the prose is checkable against a citation in the capsule.

What sources are supported in v1?

Alerts, schema changes, metrics, lineage edges, and intelligence notes. These are the structured-row sources investigations already touch. Raw log clustering and unstructured text ingestion are explicitly out of scope; if your evidence is in structured tables, it is supported.

Does this work with my own data?

EvidenceCapsule operates on the metadata AnomalyArmor already collects from your warehouse and integrations. It does not require any new instrumentation on your side beyond the existing AnomalyArmor connection.

Is the capsule format stable?

Yes, the schema is versioned and additive. The existing get_investigation tool returns the legacy JSON shape plus the new evidence_capsule field, so clients that do not opt in keep working unchanged.

Can I export a capsule to an incident ticket?

Not as a built-in flow yet. The capsule's typed structure makes that build straightforward, and it is on the short list of things we expect to ship as a follow-up. If you want to do it manually today, the SDK gives you all the fields.

How is confidence computed?

Confidence is aggregated from the underlying correlators (each evidence row carries its own confidence; the capsule's overall confidence is the aggregated result). It is not a second LLM call grading the first one. Rule-based aggregation is more predictable and avoids the "model grades itself" failure mode.

Why do you call it a "capsule"?

The naming comes from the broader trend of structured investigation outputs in observability tooling. The shape (root cause + triggers + consequences + confidence + open questions, each backed by typed evidence) is the load-bearing part, not the word.

If you want to see EvidenceCapsule in action on your own warehouse, AnomalyArmor is in private beta. Reach out and we will get you access.

How AI-Native Data Observability Changes Incident Response

Blaine Elliott — Mon, 22 Jun 2026 18:44:39 +0000

AI-native data observability changes incident response by replacing dashboard navigation with conversational queries grounded in your warehouse metadata. Instead of clicking through a lineage graph to find which dashboards depend on a broken table, you ask the assistant, get an answer with citations to the actual metadata, and act on it. The change is not the alert itself; alerts have always fired. The change is the minutes between the alert firing and the engineer knowing exactly what to do.

I build AnomalyArmor, which is one of the tools that works this way. The argument below is about the workflow shape, not the product specifically. The same workflow exists with any tool that exposes its metadata to an AI assistant through an MCP server or similar surface. The reason it matters is mean time to resolution, not vendor differentiation.

What a traditional incident response looks like

Take a real failure mode: a Fivetran sync to raw.orders runs late, the downstream fct_orders model builds against stale data, and the executive dashboard shows yesterday's revenue as today's. A freshness monitor fires at 9:04am.

Here is the workflow most teams run today.

9:04 — Alert fires in Slack. "Freshness check on raw.orders: last update 14 hours ago, expected < 6."
9:06 — Engineer opens the observability tool dashboard. Searches for raw.orders. Opens the table view.
9:08 — Clicks into lineage. Visual graph shows 12 downstream models and 4 dashboards. Engineer reads each one to figure out which matter.
9:12 — Switches to Fivetran. Confirms the sync is in a retrying state. Reads logs.
9:15 — Switches to dbt Cloud. Checks whether the model run that consumed stale data already completed. It did.
9:18 — Switches back to observability tool. Cross-references which dashboards consume fct_orders directly versus through a downstream mart.
9:22 — Pings #data-platform. "Hey, exec dashboard is on stale data, Fivetran is recovering, ETA?"
9:30 — Posts in #exec-team. "Heads up, dashboard is stale until ~10am, here is what is affected."

Twenty-six minutes from alert to communication. Most of that time is context-switching between tools to assemble a picture the metadata already contains. None of that time is spent fixing the actual problem.

What an AI-native incident response looks like

Same incident. Same alert. Different workflow because the assistant has access to the same metadata the dashboard does, plus the ability to reason across it.

9:04 — Alert fires in Slack. Same alert.
9:05 — Engineer opens Claude Code / Cursor / their assistant. Types: "raw.orders is 14h stale, which dashboards consume it and what is the blast radius."
9:05 — Assistant responds. "raw.orders feeds fct_orders (built 6:00 today on stale data). fct_orders feeds 3 dashboards: exec_revenue (refreshed 7:15), sales_pipeline (refreshed 7:30), finance_daily (refreshed 7:45). All three are showing yesterday's data. No other downstream consumers." Each claim cites the metadata source.
9:06 — Engineer asks one more. "What is the Fivetran sync status and last successful run."
9:06 — Assistant responds. "Fivetran sync raw_orders is in 'recovering' state since 8:51, retry 2 of 5, last successful sync at 19:04 yesterday. Expected next success window 9:30 to 10:00 based on retry cadence."
9:07 — Engineer posts in #exec-team. "Heads up: exec_revenue, sales_pipeline, and finance_daily are stale until ~10am. Fivetran recovering, no action needed on your side."

Three minutes from alert to communication. The dashboards were not opened. The lineage graph was not clicked through. The cross-tool stitching happened in one query because the assistant has access to the metadata that exists in the warehouse, the observability tool, and the data sync platform.

The fix did not get faster. Fivetran still needs to recover at its own pace. What got faster is everything that is not the fix: classification, impact analysis, stakeholder communication, decision making.

Why the workflow shape changes the math, not just the tool

The reason this matters is not that AI assistants are clever. It is that the bottleneck in most incident response is not "I do not know what broke" or "I do not know how to fix it." It is "I do not yet know what depends on the thing that broke." That is a metadata-traversal problem, and metadata traversal is what AI assistants do well when you give them the right context.

Phase	Traditional time	AI-native time	Why the difference
Detect	0 to 5 min	0 to 5 min	Same alert
Classify (is this real?)	2 to 5 min	30 sec	Assistant reads alert payload + recent state
Impact analysis (what depends?)	5 to 15 min	30 sec	Assistant traverses lineage in one query
Cross-tool context (sync, dbt, BI)	5 to 10 min	30 sec	Assistant queries all three sources
Stakeholder communication	2 to 5 min	1 min	Engineer has the answer, just writes the message
Total mean time to communication	15 to 35 min	2 to 8 min	Metadata traversal happens in parallel, not serially

The "AI" is not doing the engineer's job. It is removing the dashboard-clicking from the engineer's job so that the engineer can spend their attention on the parts of the response that actually require judgment: do we roll back, do we wait, do we communicate broadly or narrowly, do we change the SLA going forward.

The technical shape that makes this work

The capability that enables the AI-native workflow is the MCP server (or equivalent agent-callable interface) that exposes the observability tool's metadata to the assistant. Without it, the assistant has training-data-level knowledge of "what dbt is" but no access to your specific schemas, freshness state, lineage graph, or recent alerts. With it, the assistant can query the same metadata the dashboard renders, and reason across it in natural language.

A working MCP surface for data observability needs to expose, at minimum:

Asset inventory. Tables, schemas, dependencies, owners.
Current state. Freshness, last successful sync, recent schema changes, active alerts.
Lineage. Which downstream models, dashboards, and consumers depend on a given table or column.
Historical context. What changed in the last hour, day, week. What alerts fired and how they resolved.
Cross-tool metadata where it exists. dbt run history, Fivetran/Airbyte sync state, BI tool consumption.

Each of these is a discrete tool call the assistant can compose into an answer. The engineer asks one natural-language question; the assistant runs three or four metadata queries and synthesizes a response. The work that used to be "open four tabs, read each one, reconcile in your head" becomes one round trip.

What AI-native observability does not do

Be honest about the limits, because the failure mode of AI-native workflows is overclaiming.

It does not replace the human decision. The assistant tells you that exec_revenue and finance_daily are stale. You still decide whether to publish a stale dashboard, roll back to a snapshot, or let it ride to the next refresh. That decision involves business context the assistant does not have and should not be asked to have.

It does not catch what the underlying detection misses. If your freshness monitors are misconfigured or your schema checks are turned off, the AI-native workflow surfaces the absence of signal cleanly, but it does not invent signal. The underlying monitors still have to be good.

It does not work without metadata access. The whole capability hinges on the assistant being able to query the tool's metadata. A tool that does not expose an MCP server or equivalent does not participate in this workflow at all, regardless of how good its dashboard is.

It does not eliminate dashboards. The AI-native flow handles the "during an incident" path well. Long-term trend analysis, cross-team review, exec-level reporting, and visual lineage exploration still benefit from a UI. Both surfaces matter; they cover different work.

Where this changes hiring and team shape

A secondary effect worth naming. When incident response moves from "navigate four tools" to "ask one question," the implicit seniority floor for handling an incident drops. A new data engineer in week two cannot effectively triangulate a dashboard breakage across Snowflake, dbt Cloud, Fivetran, and the observability tool. They can ask the assistant a clear question and read a sourced answer.

The result is not that you need fewer senior engineers. It is that incident response stops monopolizing senior attention for routine cases, which frees senior engineers to spend time on the work where their judgment actually matters: schema design, SLA negotiation, postmortems that change the system, not just patch it.

How to evaluate whether a tool genuinely supports this

Many tools market "AI features" that do not change the incident-response workflow at all. Three questions separate the substantive integrations from the marketing layer.

Does the tool expose an MCP server or equivalent agent-callable interface? If the AI is only available inside the vendor's own dashboard chat box, the workflow has not changed; you have moved the dashboard click into a chat input. The point is to use the assistant you already work in, not to add another UI.
Are the answers grounded in your metadata, with citations to the source? "The lineage graph says X" is grounded. "Based on common patterns, X is likely" is not. Hallucinated answers in incident response are worse than no answers.
Can the assistant query historical state, not just current? "What changed in the last 24 hours" is the question incident response runs on. A tool that only exposes current snapshots forces you back to the dashboard for history, which puts you back in the traditional workflow.

Tools that answer yes to all three change the workflow. Tools that answer yes to one or two are partially there. Tools that answer no to all three have a feature called "AI" that does not affect how you handle an incident.

A short experiment to run before committing

You do not have to switch tools to test whether this workflow would help your team. Run a one-week experiment.

For five business days, log every incident response. Time-stamp: alert fired, engineer started, impact identified, stakeholders notified, root cause known, fixed.
After each incident, write down which steps were "looking at metadata to figure out the shape of the problem" versus "actually doing the fix."
Total the "metadata-traversal" time across the week. That is the time an AI-native workflow targets.

Most teams who run this exercise find that 50 to 70 percent of their incident-response time is metadata traversal. If your number is in that range, the workflow change is worth evaluating concretely. If your number is below 25 percent (either because you have very few incidents or because your team has built strong tooling already), the gain is smaller and the priority lower.

For what AI-native observability looks like end to end, see a data observability tool that works from inside your AI assistant. For why this matters relative to dbt-only workflows, see how to catch silent dbt test failures.

AI-native incident response FAQ

Is this the same as ChatGPT for data engineering?

No. A general assistant has no access to your warehouse, your lineage, your alert history, or your dbt project. It can talk about data engineering in the abstract. AI-native observability gives the assistant grounded access to your specific metadata so the answers are about your warehouse, not generic patterns.

What is an MCP server in this context?

MCP (Model Context Protocol) is a standard for exposing tools and data to AI assistants. An observability tool with an MCP server exposes its metadata (tables, lineage, freshness, alerts) as callable tools the assistant can use. The assistant decides which to call based on the question, runs them, and synthesizes the response. The engineer never has to know which tool was called.

Does this work with Claude, Cursor, ChatGPT, or my own agent?

Any assistant or agent that speaks MCP can use any MCP-exposing tool. Claude Code, Cursor, and the OpenAI Agents SDK are the most common surfaces right now. The point of using a standard is that the observability tool does not have to integrate with each assistant separately.

Does this change the alert itself?

No. Alerts still fire from the same underlying monitors (freshness, schema, volume, distribution, custom rules). What changes is what happens between the alert firing and the human deciding what to do. That window is where the workflow shape matters.

What if my team already has good runbooks?

Runbooks are static; metadata is dynamic. A runbook tells you the kind of thing to check when freshness fails. AI-native observability runs the checks against the actual current state and tells you what is true right now. The two work together: the runbook is the strategy, the AI-native flow is the execution.

How is this different from the chat box already in my observability tool?

A vendor-specific chat box keeps you inside the vendor's UI, which means you have to context-switch into that UI during an incident. An MCP server lets the assistant you already use (the same one you write SQL with, edit code in, and run terminal commands from) talk to the observability tool. The difference is whether you change tools to ask the question.

Is the assistant going to hallucinate when I am trying to resolve an incident?

Hallucination risk drops sharply when the assistant has grounded access to real data and is asked to cite sources for its claims. The remaining risk is real but manageable: treat the assistant's answer as a faster version of what a junior engineer would tell you, and verify the same way you would verify a junior engineer (does the cited source actually say what they said it says).

Does this only work in well-instrumented warehouses?

It works best in well-instrumented warehouses, the same way dashboards work best when the underlying metrics exist. If your tool has no lineage, the assistant cannot traverse lineage. The capability of AI-native observability is bounded by the metadata that exists under it.

The bottom line

The promise of AI-native observability is not smarter alerts; it is shorter time from "something broke" to "we know what to do." The change happens in the metadata-traversal window, which is where most incident-response time actually goes. The shape of the workflow shifts from "open four tabs" to "ask one question," and the engineer's attention moves from clicking through tools to making the decisions only a human can make.

This is not the only thing AnomalyArmor does, but it is the thing that has changed how I think about what an observability tool is for. AnomalyArmor is in private beta. If you want to see what the assistant-side workflow looks like on your own warehouse, reach out and we will get you access.

How Much Does Data Observability Cost in 2026?

Blaine Elliott — Mon, 15 Jun 2026 22:22:40 +0000

Data observability costs between $0 and roughly $60,000 per year for a mid-sized warehouse in 2026, depending entirely on the pricing model: open-source tools have no license fee but cost engineering time to run, transparent per-table tools run $5 to $10 per monitored table per month, and enterprise platforms are custom-quoted and typically land in the five-figure annual range. The list price is only part of the number. The total cost includes implementation, ongoing maintenance, alert triage, and the switching cost you pay if you pick wrong. This guide breaks down each model with real numbers and gives you a formula to estimate your own total before any sales call.

I build AnomalyArmor, a per-table-priced data quality monitoring tool, so treat this as a biased source and verify every number against each vendor's own pricing page and your own quote. The pricing models and the cost structure below are vendor-independent. The point is to let you budget accurately, not to sell you anything.

What does data observability cost in 2026?

Here is the range by pricing model, for a representative mid-market warehouse of around 100 monitored tables. These are list prices and typical ranges, not quotes.

Pricing model	Example tools	Typical annual cost (100 monitored tables)	What drives the price
Open-source, self-hosted	Soda Core, Elementary, Great Expectations	$0 license + engineering time	Setup and maintenance hours, infra
Transparent per-table	AnomalyArmor ($5/table/mo), Metaplane by Datadog ($10/table/mo)	$6,000 to $12,000	Number of monitored tables
Consumption / volume	Platform-billed tools	Variable, often $15,000+	Rows scanned, compute, monitor runs
Custom enterprise	Monte Carlo, Bigeye	Five figures, custom-quoted	Table count, sources, monitor depth, seats

The spread is wide because "data observability" covers everything from a Python library you run yourself to a full enterprise incident-management platform with formal SLA workflows. The right number for you depends on warehouse size, how much engineering time you can spend, and whether you need enterprise procurement features or just reliable detection.

What are the pricing models for data observability tools?

There are five pricing models in the market, and the differences between them matter more than the headline numbers. A tool can be cheap on list price and expensive in total, or free on license and costly in engineering time.

Model	How you are billed	Predictable?	Best fit
Per monitored table	Flat rate per table with an active monitor	Yes, scales linearly with tables	Teams that want to budget by warehouse size
Per seat / per user	Flat rate per user with platform access	Partly, until the team grows	Small teams, large warehouses
Consumption / volume	Rows scanned, compute used, or monitor runs	No, varies with data volume	Teams comfortable with usage-based bills
Custom enterprise	Negotiated bundle across multiple axes	Only after the quote	Enterprises with procurement and SLA needs
Open-source self-hosted	No license; you run it	License yes, total no	Teams with spare engineering capacity

The two questions that separate a predictable bill from a surprising one: does the price scale on an axis you control (tables) or one you do not (data volume), and can you see the number before a sales conversation?

How does per-table pricing work?

Per-table pricing charges a flat monthly rate for each table that has an active monitor. It is the most predictable model because the cost axis is something you decide: you choose which tables to monitor, so you control the bill directly.

Two published examples make the comparison concrete. Metaplane by Datadog lists its Pro plan at $10 per monitored table per month. AnomalyArmor lists at $5 per monitored table per month. Both bill on tables with monitors running, so the comparison is direct.

Monitored tables	At $5/table/mo	At $10/table/mo	Annual difference
25	$1,500/yr	$3,000/yr	$1,500
50	$3,000/yr	$6,000/yr	$3,000
100	$6,000/yr	$12,000/yr	$6,000
250	$15,000/yr	$30,000/yr	$15,000
500	$30,000/yr	$60,000/yr	$30,000

The critical detail most teams miss: you pay for monitored tables, not tables in the warehouse. A warehouse with 4,000 tables does not cost 4,000 times the per-table rate, because you do not monitor every staging and intermediate object. Most teams monitor 50 to 300 tables that actually feed dashboards, models, or downstream consumers. Estimate that number before you read any pricing page, because it is the only input that matters.

You can get a rough count straight from your warehouse. This works on Snowflake and adapts to Databricks with information_schema equivalents:

-- Snowflake: tables touched by downstream consumers in the last 30 days
-- is a far better proxy for "what to monitor" than total table count
SELECT count(DISTINCT table_name) AS candidate_tables
FROM snowflake.account_usage.access_history,
     LATERAL FLATTEN(input => base_objects_accessed) bo
WHERE bo.value:"objectName"::string IS NOT NULL
  AND query_start_time >= dateadd('day', -30, current_timestamp());

The number that comes back is closer to your real monitoring scope than the raw table count. Note a nuance worth budgeting for: per-table pricing is list pricing, and vendors do discount at scale. Metaplane offers volume and multi-year discounts that land below the $10 list rate at higher table counts. Your negotiated number is the one that matters above roughly 250 tables. A published flat rate, by contrast, is the same for everyone and visible before any conversation.

Why is enterprise data observability pricing opaque?

Enterprise observability platforms like Monte Carlo and Bigeye do not publish list pricing. You learn the number through a sales conversation that scopes table count, source count, monitor depth, and seats. For an enterprise procurement team with a quarter-long evaluation cycle, that is routine. For a mid-sized data team trying to compare three tools in a week, it is a friction tax.

Public references and third-party marketplace data put both Monte Carlo and Bigeye deployments in the five-figure annual range for mid-to-large warehouses, scaling with the axes above. The lack of a published number is itself a meaningful cost: you cannot budget, compare, or get internal approval without first spending the time to extract a quote.

The category has also consolidated, which changes how buyers weigh pricing stability. Metaplane was acquired by Datadog in April 2025 and is now "Metaplane by Datadog." Monte Carlo restructured in 2026, cutting roughly 30% of staff. Both events made vendor independence and written pricing-change notice periods first-class buying criteria rather than afterthoughts. If you sign an enterprise contract, the notice period for a pricing or packaging change is now a term worth negotiating explicitly.

How much does open-source data observability cost?

Open-source data observability has no license fee. Soda Core, Elementary, and Great Expectations are free to download and run. The cost is engineering time: you host the tool, configure the checks, maintain it through upgrades, and build the alerting and scheduling around it.

That cost is real and recurring. A reasonable estimate is one-quarter to one-half of an engineer's time during setup, dropping to a few hours a week for maintenance once stable. At a loaded engineering cost of $80 to $150 per hour, even four hours a week of maintenance is $16,000 to $31,000 per year. Open-source is genuinely free on license and frequently the most expensive option in total cost once you price the engineering time honestly. It is the right call when you have spare capacity, want full control, and have an engineer who will own it. It is the wrong call when that engineer's time is worth more spent elsewhere.

What is the total cost of data observability?

List price is the number vendors quote. Total cost is the number you actually pay. The gap between them is where budgets break. Use this framework to estimate the real annual cost of any option, regardless of pricing model.

Total Cost of Data Observability (annual) = License + Implementation + Maintenance + Triage + Switching risk

License: the quoted or list subscription cost (or $0 for open-source).
Implementation: the one-time setup cost, amortized. Connecting sources, configuring monitors, importing existing tests. Estimate the hours and multiply by loaded engineering cost.
Maintenance: recurring engineering time to keep it running. Near zero for managed tools, substantial for self-hosted.
Triage: the cost of responding to alerts, including false positives. A noisy tool that fires 40 alerts a week where 35 are noise has a high triage cost even at a low license price.
Switching risk: the expected cost of having to migrate if the tool, vendor, or pricing changes. Higher for opaque enterprise contracts and acquired products; lower for transparent, standalone tools.

A worked formula for a managed per-table tool at 100 tables and $5/table/month:

License:        100 tables x $5 x 12        = $6,000/yr
Implementation: 16 hours x $120, amortized  = ~$1,920 (one-time)
Maintenance:    ~1 hour/week x $120 x 52    = $6,240/yr
Triage:         depends on alert quality    = variable
Switching risk: low (transparent pricing)   = ~$0 modeled
                                              -----------
First-year total (ex-triage):                ~$14,160
Steady-state annual (ex-triage):             ~$12,240

Run the same formula for an open-source tool and the license drops to $0 while maintenance climbs to $16,000 or more, often flipping the ranking. Run it for an enterprise tool and the license climbs into five figures while implementation grows with the services-led onboarding. The framework is the point: compare totals, not list prices.

What hidden costs should you budget for?

Five costs rarely appear in a vendor quote but always appear in your actual spend.

Hidden cost	Where it hides	How to estimate
Per-source surcharges	Some tools charge per connected warehouse or source on top of per-table	Count your sources, ask if each adds a fee
Seat expansion	Per-user models get expensive as the team grows	Project headcount over the contract term
Onboarding services	Enterprise tools bundle paid implementation	Ask if onboarding is included or extra
Alert triage time	Noisy detection burns engineering hours weekly	Track false-positive rate during a trial
Renewal step-ups	Acquired-product pricing often holds year one, rises after	Get the multi-year rate in writing

The two that catch teams most often are alert triage and renewal step-ups. A tool's license price tells you nothing about how much engineering time you will spend dismissing false positives, which is why a parallel run that measures alert quality on your real data is worth more than any spec sheet. And acquired products frequently hold pricing for the first term and step up afterward, so the renewal rate matters more than the introductory one.

Worked example: what does monitoring a 100-table warehouse cost?

Take a concrete mid-market scenario: a Snowflake or Databricks warehouse with 100 tables worth monitoring (the dashboard-feeding, model-feeding, consumer-facing tables, not the full object count). Here is the realistic first-year total cost across the main options, using the framework above and rounding triage out as variable.

Option	License	Implementation	Maintenance	First-year total (ex-triage)
Open-source self-hosted	$0	~$3,000	~$20,000	~$23,000
Per-table at $5/table/mo	$6,000	~$1,900	~$6,200	~$14,100
Per-table at $10/table/mo	$12,000	~$1,900	~$6,200	~$20,100
Custom enterprise	~$25,000+	included/services	low	~$25,000+

Two takeaways. First, open-source is not the cheapest option here once engineering time is priced in; it only wins when you have genuinely spare capacity. Second, the per-table license difference of $6,000 ($5 versus $10) compounds every year while the implementation cost is paid once, so the multi-year gap is larger than the first-year table suggests. Model three years, not one.

How do you reduce data observability cost?

Five levers actually move the number, in rough order of impact.

Count monitored tables, not warehouse tables. The single biggest lever on a per-table or consumption bill. Most teams discover that 20 to 40 percent of monitored tables are low-value staging objects monitored by default, not by decision. Drop them and the bill drops proportionally.
Match the pricing model to your shape. A small team on a large warehouse is cheaper on per-seat or per-table than on consumption. A large team on a small warehouse may be the opposite. Pick the axis that scales slowest for you.
Negotiate at renewal, not mid-term. Renewal is the moment of maximum leverage, especially for enterprise and acquired-product contracts where the pricing direction becomes a concrete number.
Measure alert quality before committing. Run a trial and track the false-positive rate. A tool that is cheap on license but noisy on alerts has a high total cost in triage time.
Avoid per-source and seat surcharges. Confirm whether the quoted rate is all-in or whether connected sources and additional users add fees.

What should you ask a vendor about data observability pricing?

Bring this checklist to any pricing conversation. The answers, in writing, are what separate a predictable bill from a surprise.

Is the rate per monitored table, per seat, per row scanned, or a bundle, and which axis grows fastest at my projected scale?
Is the quoted rate all-in, or do connected sources, seats, or onboarding add fees?
Is the rate guaranteed for the full term, or only year one?
What is the written notice period for any pricing or packaging change?
What happens to my bill if my table count or data volume doubles?
Is implementation included, or a separate services charge?
What is the contract length and the cancellation term?

Data observability pricing comparison

A like-for-like view of the main options on the axes that drive total cost. Verify each against the vendor's current pricing page; the category moves.

Tool	Pricing model	List price published?	Typical 100-table annual	Self-hosting required?
AnomalyArmor	Per monitored table	Yes ($5/table/mo)	$6,000	No
Metaplane by Datadog	Per monitored table	Yes ($10/table/mo)	$12,000	No
Monte Carlo	Custom enterprise	No	Five figures	No
Bigeye	Custom enterprise	No	Five figures	No
Soda Core	Open-source	Free license	$0 + eng time	Yes
Elementary	Open-source	Free license	$0 + eng time	Yes
Great Expectations	Open-source	Free license	$0 + eng time	Yes

For a deeper feature-by-feature view of the managed options, see the comparisons of the best Metaplane alternative in 2026, the best Monte Carlo alternative in 2026, and the best Bigeye alternative in 2026. For the category overview, see what tools should I use for data observability in 2026, and to scope what the detection should cover before you price it, see how to monitor schema changes in a data warehouse.

The actionable takeaway

Before you read a single pricing page, count the tables you would actually monitor (the consumer-facing ones, not the warehouse total) and run the five-part total-cost formula for two or three options. The list price is the cheapest part of the decision to get right; the expensive mistakes are choosing a pricing axis you do not control, underbudgeting maintenance on a self-hosted tool, or signing an opaque contract whose renewal steps up. A transparent per-table price that scales on an axis you decide is the most predictable default for most mid-market teams, which is the model AnomalyArmor is built on. Whichever you choose, decide on total cost over three years, not list price in year one.

Data Observability Pricing FAQ

How much does data observability cost per month?

For a mid-sized warehouse of around 100 monitored tables, transparent per-table tools run $500 to $1,000 per month ($5 to $10 per table). Enterprise platforms are custom-quoted and typically higher. Open-source tools have no monthly license but cost engineering time to run.

What is the cheapest data observability tool?

On license alone, open-source tools (Soda Core, Elementary, Great Expectations) are cheapest at $0. On total cost including engineering time, a transparent per-table managed tool is often cheaper than self-hosting once you price the maintenance hours. The cheapest option depends on whether you have spare engineering capacity.

How does Monte Carlo pricing work?

Monte Carlo uses custom enterprise pricing and does not publish list rates. The number is scoped through a sales conversation based on table count, sources, monitor depth, and seats, and typically lands in the five-figure annual range for mid-to-large warehouses.

How much does Metaplane cost?

Metaplane by Datadog lists its Pro plan at $10 per monitored table per month, billed on tables with monitors running. At 100 tables that is $12,000 per year before any volume or multi-year discount.

Is data observability worth the cost?

It is worth it when the cost of data downtime exceeds the cost of the tool. A single executive dashboard showing wrong revenue numbers, or a machine learning model trained on broken data, can cost more than a year of monitoring. The math favors monitoring once you have data feeding decisions or customers.

What is the difference between per-table and consumption pricing?

Per-table pricing charges a flat rate for each monitored table, an axis you control. Consumption pricing charges by rows scanned, compute used, or monitor runs, an axis that varies with your data volume. Per-table is more predictable; consumption can be cheaper or more expensive depending on volume.

Why don't enterprise tools publish pricing?

Enterprise tools price across multiple axes (tables, sources, seats, monitor depth) and use sales-led scoping to set the number. For accounts where scoping genuinely affects price, this is defensible. For mid-market teams, the scoping conversation often adds weeks without changing the answer.

How many tables should I monitor?

Monitor the tables that feed dashboards, models, or downstream consumers, typically 50 to 300 for a mid-sized warehouse, not the full warehouse object count. Use access or lineage history to find tables that downstream consumers actually touch.

Does open-source data observability really cost nothing?

The license costs nothing. The total cost includes setup (often a quarter to half an engineer during onboarding) and ongoing maintenance (a few hours a week). At loaded engineering rates, that maintenance alone can exceed the license cost of a managed tool.

How do I budget for data observability?

Run the total-cost formula: License + Implementation + Maintenance + Triage + Switching risk, over three years. Start by counting monitored tables, the input that drives most pricing models. Then add the hidden costs (per-source fees, seats, onboarding, triage time, renewal step-ups).

What hidden costs come with data observability tools?

Per-source surcharges, seat expansion, paid onboarding services, alert triage time on false positives, and renewal price step-ups on acquired or enterprise products. None typically appear in the initial quote; all appear in your actual spend.

How much should a mid-market team spend on data observability?

A first-year total in the range of $12,000 to $25,000 is typical for a 100-table warehouse across managed options, once implementation and maintenance are included. Open-source can be lower or higher depending on how you value engineering time. The right number is the one that is less than your cost of data downtime.

Do data observability tools charge per user or per table?

Both models exist. Per-table pricing scales with warehouse size and is independent of team size. Per-seat pricing scales with team size and is independent of warehouse size. A small team on a large warehouse is cheaper on per-table; a large team on a small warehouse may prefer per-seat.

How do I reduce my data observability bill?

Cut monitored tables down to the consumer-facing set, match the pricing model to your team-and-warehouse shape, negotiate at renewal, measure and minimize alert noise, and confirm there are no per-source or seat surcharges on top of the headline rate.

What Is the Best Bigeye Alternative in 2026?

Blaine Elliott — Mon, 01 Jun 2026 19:54:43 +0000

A Bigeye alternative is a data observability tool that delivers schema, freshness, volume, and distribution monitoring without Bigeye's enterprise contract structure, which is custom-quoted, typically lands in the five-figure annual range, and requires a sales cycle before you see a price. The most common reason teams search for one in 2026 is the same reason that drives buyers off Monte Carlo: the gap between what mid-market warehouses actually need and what enterprise observability vendors charge to provide it.

I built AnomalyArmor, so this is a biased source. Verify the numbers against Bigeye's own site and your own quote, and decide for yourself. What follows is the comparison I would want as a buyer: where Bigeye is genuinely strong, where its sales motion is a poor fit for non-enterprise teams, and where a transparent standalone tool is a more sensible default.

Why are data teams looking for a Bigeye alternative now?

Three reasons, all structural.

First, pricing is opaque. Bigeye does not publish list pricing. Buyers learn the number through a sales conversation that includes scoping, source counts, and table volume. For an enterprise procurement team that is routine. For a mid-sized data team trying to evaluate three tools in a week, it is a friction tax that competing vendors with published per-table pricing do not impose.

Second, the sales motion assumes enterprise. Bigeye's product is built for, and sold to, organizations with a data platform team, an internal champion, and a procurement process that absorbs months of evaluation. That is a real and defensible market. It is also not where most data teams live.

Third, the data observability category has consolidated. Metaplane was acquired by Datadog in April 2025. Monte Carlo cut roughly 30% of staff in March 2026. Buyers are reasonably weighing vendor independence and pricing transparency as first-class criteria now, not afterthoughts.

Trigger	Detail	Buyer concern
Opaque pricing	No public price; sales-led quoting	Hard to evaluate or budget without a sales cycle
Enterprise sales motion	Built around large accounts with internal champions	Mismatch with mid-market team size and procurement
Category consolidation	Datadog acquired Metaplane; Monte Carlo restructured	Independence and pricing stability matter more than they used to

How much does Bigeye cost compared to AnomalyArmor?

Bigeye does not publish pricing. Third-party marketplace data and public references put enterprise deployments in the same five-figure range as other enterprise observability tools, typically scaling with table count, source count, and monitor depth. AnomalyArmor is a published $5 per table per month.

Scenario	Bigeye (custom quote)	AnomalyArmor ($5/table/mo)
50 tables	Custom (typically five figures)	$3,000/yr
100 tables	Custom (higher)	$6,000/yr
250 tables	Custom (enterprise quote)	$15,000/yr
500 tables	Custom (enterprise quote)	$30,000/yr

The unknown number is itself the relevant data point. A tool that requires a sales cycle to learn the price is structurally a slower decision than a tool that publishes one. That is fine if your buying motion is enterprise. It is friction if your buying motion is "evaluate three tools, pick one, deploy this quarter."

What does Bigeye do well?

A fair comparison says what the competitor is good at, plainly.

Autometrics and SLA-driven monitoring. Bigeye built its reputation on automatic metric coverage and SLA workflows that map cleanly to enterprise reliability practices. If your data team operates with formal SLAs and runbooks, that workflow shape is well-supported.
Enterprise integrations and security. SSO, RBAC, deployment options, and the procurement and compliance apparatus large enterprises require are all there.
Origins in real production data work. Bigeye's founding team came out of Uber's data quality group, and that pedigree shows in the depth of the platform's metric coverage and incident workflow.
Track record at scale. Bigeye has been deployed in large, complex warehouses with hundreds to thousands of tables and the platform handles that footprint.

If you are an enterprise with an SLA culture and a procurement team, Bigeye is a defensible default and you should evaluate it seriously on its merits.

What does AnomalyArmor do that Bigeye does not?

AnomalyArmor covers the same monitoring core (schema drift, freshness, volume, distribution, custom SQL, alerting, dbt, lineage) and adds:

Published, transparent pricing. $5 per table per month, visible before any conversation. No quote, no scoping call, no NDA.
Time to value measured in hours, not quarters. Auto-discovery inventories your warehouse and proposes monitors on day one. No services-led rollout, no internal champion required.
AI-native question answering. Ask "which tables feed the revenue dashboard and have any of them changed this week" in natural language and get an answer grounded in your actual metadata.
Runs inside your AI assistant. AnomalyArmor ships an MCP server and a skill pack, so monitoring lives where your engineers already are, not in a separate dashboard.
Standalone and staying that way. Not part of a larger platform, not being cross-sold into one.

When should you stay with Bigeye?

Stay if: you are a large enterprise with formal data SLAs and an incident-management culture Bigeye's workflow is designed around; your buying motion runs on procurement timelines where a custom quote is normal; you have an internal champion willing to drive a multi-month evaluation; or you have already negotiated a multi-year rate at a number that works for you and the relationship is healthy.

When should you switch to a Bigeye alternative?

Switch if: pricing transparency matters and "request a quote" is friction your team will not absorb; your warehouse is in the tens to low hundreds of tables and a five-figure floor is disproportionate; you need to be in production in days, not after a quarter of services-led onboarding; you want AI-native Q&A and assistant-side workflows; or you are re-evaluating because the category consolidation around you (Datadog acquiring Metaplane, Monte Carlo's restructuring) made vendor independence and price stability a real criterion.

How do you migrate from Bigeye to AnomalyArmor?

Both tools monitor the same warehouse objects, so migration is a swap, not a rebuild.

Inventory what Bigeye is actually monitoring. Export monitored tables and monitor types. A meaningful fraction of monitors in any long-running deployment are stale; triage into keep, replace, and retire before reproducing them.
Connect AnomalyArmor read-only. Read-only credentials on Snowflake, Databricks, BigQuery, Redshift, or Postgres. Snowflake and Databricks are first-class equally. Auto-discovery proposes monitors; you do not reconfigure by hand.
Import existing checks. dbt tests and similar config import through the adapter framework, so business-rule checks are not retyped from scratch.
Run in parallel. Keep Bigeye fully active for at least one full alerting cycle, ideally including a month-end or known-noisy window. Score detection head to head. This is the evidence that replaces the spec sheet.
Cut over at renewal. Enterprise contracts are annual; the renewal date is the natural cutover and the moment of maximum leverage.

For what the underlying detection should cover so you can compare like for like, see how to monitor schema changes in a data warehouse and the category-wide overview in what tools should I use for data observability in 2026.

What to ask before you sign or renew with Bigeye

A custom quote is not a problem if you ask the right questions before signing. Bring this list to the conversation:

What is our per-table or per-monitor rate, in writing, and how does it scale at our 12-month projected growth? Enterprise pricing scales on multiple axes; model the number you will actually pay next year, not today.
What is the written notice period for any pricing or packaging change? The category has consolidated; clarify what stability you are buying.
Are the features we depend on on the funded roadmap? Confirm specifically. Capability can stagnate quietly when a vendor refocuses.
What is our support tier's response SLA, and who is our account contact? Account coverage commonly thins after vendor restructuring; do not infer, ask.
What are the exit terms if service levels degrade? This is the question that determines whether "wait and see" is safe.

Clear written answers mean Bigeye is low risk for your specific situation. Vague verbal ones mean you should have an evaluated alternative ready before you sign.

The objections, answered honestly

"A tool at a fraction of the price can't match an enterprise platform." On enterprise breadth (formal SLA workflows, deep incident management, the procurement and compliance apparatus large enterprises require), that is fair and stated plainly above. On core detection (freshness, volume, schema, distribution, custom SQL) the claim is testable, and a parallel run measures it on your actual data in days, not on a spec sheet.

"Sales-led pricing exists for a reason." It does, for accounts where scoping genuinely affects price. For most mid-market data teams, the scoping conversation is a tax that adds weeks to the buying cycle without changing the answer. A transparent per-table price removes that tax. Both models are legitimate; the question is which one fits your team.

"Switching tools is expensive." It is, if you do it as a big-bang cutover. The parallel-run method above is incremental, reversible, and aligned to the renewal boundary so the only year you pay for two tools is the year you measure them against each other. Run the multi-year number, not the first-year number.

How AnomalyArmor compares to Bigeye: full feature table

Capability	Bigeye	AnomalyArmor
Schema drift detection	Yes	Yes
Freshness monitoring	Yes	Yes
Volume monitoring	Yes	Yes
Distribution anomalies	Yes	Yes
Custom SQL monitors	Yes	Yes
Lineage	Yes	Yes
SLA-driven workflow	Yes (mature)	Core
Slack / email / PagerDuty	Yes	Yes
dbt integration	Yes	Yes
Natural-language Q&A	Limited	Yes
Runs inside AI assistant (MCP)	No	Yes
Published pricing	No	Yes ($5/table/mo)
Time to first value	Services-led, weeks	Same day
Fits non-enterprise teams	Hard	Yes

Bigeye alternative FAQ

How much does Bigeye cost in 2026?

Bigeye does not publish pricing. Deployments are custom-quoted based on table count, source count, and monitor depth, and typically land in the five-figure annual range for mid-to-large warehouses. The lack of a published number is itself a meaningful difference from tools that price transparently.

What is the cheapest Bigeye alternative?

Among managed tools, AnomalyArmor at $5 per table per month is dramatically below typical enterprise observability pricing. Open-source options like Soda Core or Elementary have no license cost but require self-hosting and maintenance.

Does AnomalyArmor have feature parity with Bigeye?

For the core monitoring set (schema, freshness, volume, distribution, custom SQL, alerting, dbt, lineage) yes. Bigeye's SLA-driven enterprise workflow is more mature; evaluate that specifically if it is your primary requirement. AnomalyArmor adds transparent pricing, same-day setup, natural-language Q&A, and AI-assistant integration.

How long does it take to migrate from Bigeye?

Technical setup (connect warehouse, auto-discover tables, import existing checks) is typically a day or less. The recommended end-to-end timeline is longer because you should run both tools in parallel for at least one alerting cycle and cut over at your annual renewal.

Does AnomalyArmor work with Snowflake and Databricks?

Yes, both are first-class and treated equally, along with BigQuery, Redshift, and Postgres.

Is there an honest reason to pick Bigeye over AnomalyArmor?

Yes. If you are an enterprise with formal SLA culture, an internal champion, and a procurement motion that absorbs custom-quoted multi-month evaluations, Bigeye is a defensible choice. The comparison is about fit, not a claim that one tool wins universally.

The bottom line

Bigeye is a capable enterprise observability platform built around an enterprise sales motion. If you are the enterprise it is built for, it is a defensible default. If you are a mid-sized data team that needs to evaluate, buy, and deploy in a quarter rather than a year, the opaque pricing and services-led onboarding are the wrong shape regardless of the product's quality. A transparent, standalone, per-table-priced tool removes the friction that made the comparison hard in the first place.

AnomalyArmor is in private beta. If you want to see what it catches on your own warehouse, reach out and we will get you access.

What Is the Best Metaplane Alternative in 2026?

Blaine Elliott — Mon, 18 May 2026 17:16:37 +0000

A Metaplane alternative is a data observability tool that monitors warehouse tables for freshness, volume, schema, and distribution issues, the same job Metaplane does, without Metaplane's per-table pricing or its new ownership by Datadog. The most common reasons teams look for one in 2026 are the $10 per-table cost and the April 2025 Datadog acquisition that turned Metaplane into "Metaplane by Datadog."

I built AnomalyArmor, so treat this as a biased source and verify the numbers yourself. The links to Metaplane's own pricing and Datadog's acquisition announcement are below so you can. What follows is the comparison I would want if I were the one evaluating: where Metaplane is genuinely good, where the acquisition introduces risk, and where a cheaper standalone tool makes more sense.

Why are data teams looking for a Metaplane alternative now?

Two things changed in 2025 and 2026.

First, Datadog acquired Metaplane in April 2025. Metaplane now operates as "Metaplane by Datadog." Datadog has stated existing contracts will be honored with at least three months of notice for changes, and that it will reach out to every Metaplane customer. That outreach is the part worth thinking about. If you chose Metaplane because you specifically did not want to be inside a large, consumption-priced observability platform, the acquisition moved the product toward exactly that.

Second, the per-table cost adds up. Metaplane's Pro plan is $10 per monitored table per month, billed on tables with monitors running more than 30 days. At 100 tables that is $1,000 per month, or $12,000 per year, before any negotiation. For a mid-sized warehouse the table count climbs fast once you monitor staging and mart layers.

Trigger	What changed	Why it sends people searching
Datadog acquisition (Apr 2025)	Metaplane is now a Datadog product	Concern about cross-sell pressure and renewal pricing
Per-table pricing	$10/table/month Pro plan	Cost scales linearly with warehouse growth
Category consolidation	Several tools acquired or restructured in 2026	Buyers want a vendor that will still be independent next year

How much does Metaplane cost compared to AnomalyArmor?

Metaplane's published Pro pricing is $10 per table per month. AnomalyArmor is $5 per table per month. Both bill on monitored tables, so the comparison is direct.

Tables monitored	Metaplane (Pro, $10/table/mo)	AnomalyArmor ($5/table/mo)	Annual difference
50	$6,000/yr	$3,000/yr	$3,000
100	$12,000/yr	$6,000/yr	$6,000
250	$30,000/yr	$15,000/yr	$15,000
500	$60,000/yr	$30,000/yr	$30,000

The list price gap is roughly 2x. Negotiated enterprise deals vary, and Metaplane has stated existing customer contracts are honored post-acquisition, so your renewal number is the one that matters. The structural point holds regardless of discount: a standalone tool priced at half the per-table rate compounds in your favor as the warehouse grows.

What does Metaplane do well?

A fair comparison names what the competitor is good at. Metaplane earned its customer base, and the case-study list on its site is real.

Column-level lineage. Metaplane's lineage graph is mature and one of the better implementations in the category. If lineage visualization is your primary buying criterion, evaluate it seriously.
Polished UX. The product is well-designed and the onboarding is smooth. This is not a rough tool.
Integration breadth. Snowflake, Databricks, BigQuery, Redshift, dbt, Slack, PagerDuty, and the rest of the expected stack are all covered.
ML-based anomaly detection. Metaplane's baseline learning for volume and freshness anomalies is solid and well-documented.
Datadog backing, if you want it. If your organization is already a heavy Datadog shop, the acquisition is a feature, not a risk. Unified billing and a single observability vendor is a legitimate preference.

If those map to your situation, Metaplane is a defensible choice and you should weigh the acquisition as neutral or positive.

What does AnomalyArmor do that Metaplane does not?

AnomalyArmor covers the same monitoring core: schema drift, freshness, volume, distribution, custom SQL checks, Slack and PagerDuty alerts, dbt integration, and lineage. The differences are these:

Half the per-table price. $5 versus $10, with no per-source surcharge.
AI-native question answering. You can ask, in natural language, "which tables feed the revenue dashboard and have any of them changed this week," and get an answer grounded in your actual metadata. This is built in, not a roadmap item.
Works from inside your AI assistant. AnomalyArmor ships an MCP server and a skill pack, so you can set up and query monitoring from Claude Code, Cursor, or your agent of choice without opening a separate UI. Most observability tools, Metaplane included, assume you live in their dashboard.
Standalone and staying that way. AnomalyArmor is not part of a larger consumption-priced platform and is not being cross-sold into one.

The acquisition question: does it actually matter?

It depends entirely on what you wanted Metaplane to be.

If you bought Metaplane because it was a focused, independent, mid-market data observability tool, the acquisition moved it away from that. Datadog is a large, consumption-priced platform with a sales motion built around expansion. Metaplane has committed to honoring contracts and giving three months of notice, which is reasonable, but the medium-term direction of any acquired product is set by the acquirer's incentives, not the original team's.

If you are already a Datadog customer, the same facts read as an advantage: one vendor, one bill, one support relationship.

Here is a decision framework. I call it the Acquisition Fit Test. Score each statement true or false:

We are already a significant Datadog customer.
We prefer one observability vendor across infrastructure, apps, and data.
We are comfortable with consumption-based pricing that scales with usage.
Our Metaplane contract renewal is more than 12 months away.

Three or four "true" answers: the acquisition is low risk for you, and Metaplane remains a reasonable choice. Zero, one, or two "true": you chose Metaplane for reasons the acquisition undercuts, and evaluating a standalone alternative before renewal is rational.

How do you migrate from Metaplane to AnomalyArmor?

The migration is mechanical, not a rebuild, because both tools monitor the same warehouse objects.

Connect the warehouse. Read-only credentials to Snowflake, Databricks, BigQuery, Redshift, or Postgres. AnomalyArmor treats Snowflake and Databricks as first-class equally.
Auto-discover tables. The platform inventories your schemas and proposes monitors instead of making you hand-configure each table.
Import existing definitions. If you have data quality checks in dbt tests or other config, the adapter framework imports them so you are not rewriting rules.
Run both in parallel. Keep Metaplane active through one alerting cycle and compare what each catches. Do not cut over on faith.
Cut over at contract boundary. Move fully when your Metaplane renewal comes up, which is also when the acquisition's pricing direction becomes concrete.

For the schema-monitoring portion specifically, how to monitor schema changes in a data warehouse walks through what the underlying detection should cover so you can compare implementations directly.

When should you stay with Metaplane?

Be honest about this rather than assuming the cheaper tool always wins.

Stay with Metaplane if: you are a committed Datadog customer and want one observability vendor; column-level lineage is your single most important feature and you have evaluated both; your contract is locked at a negotiated rate well below list and your renewal is far out; or your team has deep workflow investment in Metaplane that would cost more to unwind than the price difference saves.

When should you switch to a Metaplane alternative?

Switch if: per-table cost is a real line item and 2x matters at your table count; you specifically did not want to be inside a large consumption-priced platform and the Datadog acquisition changed that; you want AI-native question answering and assistant-side workflows that Metaplane does not offer; or you want a vendor whose roadmap is set by data engineers' needs rather than a parent company's expansion targets.

A worked migration: Metaplane to AnomalyArmor in one sprint

Abstract migration steps are easy to write and hard to trust. Here is the concrete version, the way a data engineer would actually run it over a two-week sprint without taking monitoring offline.

Day 1: Inventory what Metaplane is actually monitoring. Export the list of monitored tables and the monitor types per table from Metaplane. The number that matters is not "tables in the warehouse," it is "tables with an active monitor," because that is what you pay for and what you need to reproduce. Most teams discover here that 20 to 40 percent of their monitored tables are low-value staging objects that were monitored by default, not by decision. Note them; you may not re-create all of them.

Day 1, same afternoon: Connect AnomalyArmor read-only. Create a read-only role in Snowflake, Databricks, BigQuery, Redshift, or Postgres. The connection is metadata and sampling only; AnomalyArmor does not copy your data. Auto-discovery inventories schemas and proposes monitors. This is the step that takes minutes rather than the services-led weeks an enterprise rollout assumes.

Days 2 to 3: Reconcile the monitor sets. Put the Metaplane export next to the AnomalyArmor proposed set and resolve three buckets: monitors that map one-to-one (most freshness and volume checks), monitors that need a custom SQL equivalent (business-rule checks Metaplane implemented as custom monitors), and monitors you will deliberately drop (the low-value staging defaults from Day 1). Importing dbt test definitions through the adapter framework covers the rules you already expressed as tests, so you are not re-typing them.

Days 4 to 10: Run both tools in parallel. This is the step teams are tempted to skip and should not. Keep Metaplane fully active. Let AnomalyArmor run against the same warehouse for at least one full alerting cycle, ideally including a month-end or a known-noisy load window. Compare every alert: did both catch it, did one catch it earlier, did either produce a false positive. You are not looking for feature parity on a spec sheet, you are looking for detection parity on your actual data.

Days 11 to 14: Decide per monitor, not all-or-nothing. Migration does not have to be a single cutover. You can move freshness and volume monitoring first, keep one tool for lineage during a transition, and retire the second tool at the contract boundary. The only hard rule is that no production-critical table goes through a window with zero active monitoring on either tool.

The reason to align the final cutover with your Metaplane renewal is not just cost. Renewal is when the acquisition's pricing and packaging direction becomes a concrete number on a quote instead of a hypothetical, which means it is the moment you have the most information and the most leverage.

What to ask before you renew Metaplane

If you are an existing Metaplane customer, the highest-leverage thing you can do is not switch tools, it is ask the right questions at renewal. Bring this list to the conversation:

Is our per-table rate guaranteed for the full next term, or only the first year? Acquired-product pricing often holds at year one and steps up after.
What is the written notice period for any pricing or packaging change, and what triggers it? Datadog has stated at least three months; get your specific terms in writing.
Will our plan remain available as a standalone SKU, or is it being merged into a Datadog platform bundle? This determines whether you can stay on what you bought.
What happens to our pricing if our table count grows 2x? Model the renewal at your projected scale, not today's.
Is the feature set we depend on (lineage depth, specific integrations) on the funded roadmap post-acquisition? Capability can stagnate quietly after an acquisition even when the product technically still works.

A vendor that answers these clearly and in writing is one you can stay with confidently. A vendor that will only answer them verbally and vaguely is one you should have an evaluated alternative ready for. Either way, you are better off than renewing on autopilot.

The objections, answered honestly

"Switching tools is more expensive than the price difference." Sometimes true. If the annual saving is $3,000 and the migration costs a senior engineer two weeks, the first-year math is close to neutral. But the saving recurs every year and the migration is paid once, and the parallel-run approach above caps the migration cost by making it incremental rather than a big-bang project. Run the multi-year number, not the first-year number.

"A cheaper tool must be cutting corners somewhere." A fair challenge. The honest answer is that AnomalyArmor's lower price comes from being a focused standalone product without an enterprise field-sales organization and a multi-product platform to fund, not from thinner detection. The place to verify this is the parallel run: detection parity on your data is the only proof that matters, and it is observable in days.

"We will just wait and see what Datadog does." Defensible, with one caveat: "wait and see" without an evaluated alternative is not a strategy, it is a hope. The low-cost version of due diligence is to run the parallel evaluation now so that if the renewal number is bad, switching is a decision you can execute in a sprint instead of a quarter.

How AnomalyArmor compares to Metaplane: full feature table

Capability	Metaplane by Datadog	AnomalyArmor
Schema drift detection	Yes	Yes
Freshness monitoring	Yes	Yes
Volume monitoring	Yes	Yes
Distribution anomalies	Yes	Yes
Custom SQL monitors	Yes	Yes
Column-level lineage	Yes (mature)	Yes
Slack / email / PagerDuty	Yes	Yes
dbt integration	Yes	Yes
ML baseline learning	Yes	Yes
Natural-language Q&A	No	Yes
Runs inside AI assistant (MCP)	No	Yes
List price per table / month	$10	$5
Per-source surcharge	Varies	None
Standalone (not part of larger platform)	No (Datadog)	Yes

The broader category context

The data observability category consolidated and contracted in 2025 and 2026. Metaplane went to Datadog. Other tools restructured. Buyers are right to weigh vendor independence as a real criterion now, not a hypothetical one. For a category-wide view rather than a head-to-head, see what tools should I use for data observability in 2026, and for the underlying question of what these tools actually measure, data observability vs data quality.

Metaplane alternative FAQ

Is Metaplane still available after the Datadog acquisition?

Yes. Metaplane operates as "Metaplane by Datadog" and continues as a standalone product. Datadog has stated existing features, support, and pricing in current contracts are honored, with at least three months of notice for changes.

How much does Metaplane cost in 2026?

Metaplane's published Pro plan is $10 per monitored table per month, billed on tables with monitors active more than 30 days. Enterprise pricing is negotiated separately.

What is the cheapest Metaplane alternative?

Among managed tools, AnomalyArmor at $5 per table per month is roughly half Metaplane's list price. Open-source options like Soda Core or Elementary have no license cost but require you to host and maintain them.

Does AnomalyArmor have feature parity with Metaplane?

For the core monitoring set (schema, freshness, volume, distribution, custom SQL, alerting, dbt, lineage) yes. AnomalyArmor adds natural-language Q&A and AI-assistant integration. Metaplane's column-level lineage is more mature; evaluate that specifically if it is your primary criterion.

Will Metaplane's pricing change now that it is owned by Datadog?

Datadog has committed to honoring existing contracts with three months of notice for changes. Industry commentary expects cross-sell activity and the possibility of pricing changes at contract renewal, which is normal acquirer behavior. Treat your renewal date as the real decision point.

Should I switch off Metaplane just because of the acquisition?

Not automatically. If you are a Datadog customer, the acquisition is likely positive. If you chose Metaplane to avoid a large consumption-priced platform, the acquisition undercuts that reason and evaluating alternatives before renewal is rational. Use the Acquisition Fit Test above.

How long does it take to migrate from Metaplane?

The technical setup (connect warehouse, auto-discover tables, import existing checks) is typically a day or less. The recommended timeline is longer because you should run both tools in parallel for at least one alerting cycle before cutting over at your contract boundary.

Does AnomalyArmor work with Snowflake and Databricks?

Yes, both are first-class and treated equally, along with BigQuery, Redshift, and Postgres.

Is there an honest reason to pick Metaplane over AnomalyArmor?

Yes. If you are a heavy Datadog shop wanting one observability vendor, or column-level lineage is your single most important feature, Metaplane is a defensible choice. The comparison is about fit, not a claim that one tool wins universally.

The bottom line

Metaplane is a good product that is now a Datadog product. That single fact reorganizes the decision. If Datadog ownership is neutral or positive for you and lineage is your priority, stay. If you picked Metaplane to be independent and mid-market, the thing you bought changed, and a standalone tool at half the per-table price is worth evaluating before your renewal forces the question for you.

If you want to see what AnomalyArmor catches on your own warehouse, AnomalyArmor is in private beta. Reach out and we will get you access.

Why Do Data Teams Use AI to Write Code but Not to Monitor Pipelines?

Blaine Elliott — Mon, 11 May 2026 14:47:34 +0000

The AI gap in analytics engineering is a 48-percentage-point difference between how many data teams use AI to write code (72%) and how many use AI to monitor, test, or observe their pipelines (24%). It is the single most important structural finding in dbt's 2026 State of Analytics Engineering report, and it describes a reliability problem that will get worse before it gets better.

The short version: teams are building data pipelines faster than ever because AI writes the code, but nobody is paying proportional attention to whether those pipelines produce correct data. AI has been invited into the creation step. It has not been invited into the quality step. This post explains why that gap exists, what it costs, and what closing it looks like in practice.

What does the AI gap in data engineering mean?

The gap is measured in a single dbt survey across thousands of analytics engineers. The relevant numbers:

AI use case	2026 prioritization
AI-assisted coding (writing SQL, dbt models, scripts)	72%
AI-assisted pipeline management (testing, observability, quality controls)	24%
Delta	48 percentage points

The same survey also reported that 71% of teams are concerned about "hallucinated or incorrect data reaching stakeholders." So the industry is simultaneously: (a) accelerating pipeline creation with AI, (b) afraid of AI-caused data errors reaching business users, and (c) not using AI to catch those errors. That combination is not sustainable.

Why are data teams adopting AI-assisted coding first?

The creation side of the pipeline is where AI adoption is easiest and the benefit is most visible. Three reasons this happened first:

The loop is tight. A data engineer writes a dbt model, asks Copilot or Cursor to improve it, reads the result, commits. The feedback cycle is seconds. The developer sees the value immediately.

The failure mode is visible. If AI writes bad SQL, the query errors out or returns obviously wrong results at build time. Code failures are noisy, which makes them safe to accept AI help on.

The tools already exist. GitHub Copilot, Cursor, Claude Code, Codex CLI, and ChatGPT all hook into the IDE seamlessly. Writing code is a solved interface problem. Every AI coding tool competes on the same surface.

The productivity story is quantifiable. "I wrote this dbt model in 5 minutes instead of 30" is easy to measure and celebrate. Managers greenlight the tool because the demo is obvious.

None of those conditions hold for pipeline management.

Why is pipeline monitoring stuck at 24% AI adoption?

Pipeline management has none of the conditions that accelerated AI-assisted coding. It has the opposite of all of them.

The loop is slow. A monitoring system runs continuously and only fires alerts when something deviates. The value of "I caught this bad load" shows up hours or days after setup, not seconds. That makes the ROI story harder to sell internally.

The failure mode is silent. Unlike a broken query that throws an error, a bad data pipeline runs green, produces plausible-looking numbers, and nobody notices until a stakeholder asks why the dashboard is wrong. Data downtime is the metric that quantifies this invisibility. Teams often don't know they need AI help until they measure how much downtime they've accumulated.

The tools don't exist yet at scale. The data observability category has been dominated by dashboard-first products like Monte Carlo, Metaplane, and Bigeye. These tools use AI for isolated features (anomaly sensitivity tuning, alert summarization) but not as the primary interface. The AI-native equivalent of Cursor for data reliability has not reached critical mass.

The integration surface is bigger. AI-assisted coding needs to understand your file. AI-assisted pipeline management needs to understand your warehouse, your lineage, your historical baselines, your team's alert fatigue tolerance, and your on-call runbook. That is an order of magnitude more context.

The productivity story is inverted. "I prevented a bad load from reaching the dashboard" has a counterfactual quality that is harder to celebrate than "I wrote a new model in 5 minutes." The value shows up as incidents that didn't happen.

What are the consequences of the 72/24 asymmetry?

Three specific failure modes emerge when creation outpaces reliability.

Faster pipeline growth without proportional monitoring coverage. A team that ships 3x more models with AI assistance but does not scale its monitoring practice will have 3x more pipelines that can silently break. The blast radius per unaddressed failure grows because more downstream consumers depend on the pipeline.

Higher baseline rate of AI-introduced quality bugs. AI-generated SQL, especially under time pressure, produces plausible-looking queries that can silently miscount, drop edge cases, or misuse joins. A human reviewer catches some. Fresh monitoring would catch the rest. But if monitoring is the manual bottleneck while coding is AI-accelerated, the balance tilts toward bugs reaching production.

Decay of institutional knowledge about data correctness. When a human writes a pipeline, they usually have a rough model of what "correct" looks like for its outputs. When AI writes the pipeline, that model lives in the prompt, not in the engineer's head. The check against "did the data come out right?" needs to be externalized into automated monitoring, or it doesn't happen.

The dbt survey directly measures the anxiety this creates: 71% of respondents are concerned about "hallucinated or incorrect data reaching stakeholders." The fear is justified. The tooling hasn't caught up.

The AI Reliability Lag: a framework for thinking about the gap

The pattern in the dbt numbers repeats across other engineering disciplines. AI adoption in any creation step (writing code, designing, drafting) typically precedes AI adoption in the corresponding reliability step (testing, reviewing, monitoring) by 12-36 months. We can call this the AI Reliability Lag.

Stage	Creation-side AI	Reliability-side AI	Typical lag
Software engineering	Copilot, Cursor (mature)	AI test generation, AI code review (emerging)	18-24 months
Data engineering	Copilot for SQL, dbt copilots (mature)	AI observability, AI data quality (early)	24-36 months
Technical writing	ChatGPT for drafts (mature)	AI content fact-checking (rare)	24+ months
Design	Midjourney, Figma AI (mature)	AI design QA, accessibility checks (rare)	30+ months

The lag has the same root cause everywhere. Creation is a bounded, visible, immediate-reward task that any team can adopt unilaterally. Reliability is continuous, invisible, deferred-reward, and requires integration with infrastructure the team often doesn't own. The former spreads through pull; the latter requires push.

For data engineering specifically, the lag has an asymmetric cost. In code, a bug that slips past is caught by users, tests, or the next deploy. In data, a bug that slips past corrupts analytics, trains bad ML features, misinforms business decisions, and erodes trust in the platform. The data reliability step is more expensive to skip than the code reliability step.

What would AI-assisted pipeline management actually look like?

Four concrete capabilities, roughly in order of current maturity:

1. Automated anomaly detection with learned baselines

Instead of writing explicit assertions, you point the system at your warehouse and it learns what "normal" looks like for every table: update cadence, row count distribution, null rate, value ranges. When production data deviates from the baseline, it alerts. No threshold configuration required. This is the pattern covered in Data Anomaly Detection: The Complete Guide.

2. AI agents that set up monitoring from natural language

The user says: "watch the orders table for schema changes and alert the team on Slack." The agent translates intent into concrete monitors (schema snapshot cadence, diff rules, alert routing, severity), configures them, and reports back. No YAML. No rules engine. No documentation deep-dive. This is the paradigm AnomalyArmor ships today and what "AI-assisted pipeline management" should mean by default.

3. Natural-language data quality queries

A data engineer in an incident asks: "when did revenue_daily last update, and what changed in the schema in the past week?" The agent queries lineage, metadata, and audit logs and returns a structured answer. This replaces a 20-minute manual dig through INFORMATION_SCHEMA, dbt logs, and Slack channels.

4. AI-generated alert context and runbooks

When an alert fires, the system automatically summarizes: which table broke, what changed upstream, what downstream consumers are affected, what the fix usually looks like based on the team's incident history. The on-call engineer reads a two-paragraph brief instead of starting from a blank page at 3am. This is the difference between a 2-hour and a 20-minute TTR.

All four capabilities exist in nascent form somewhere in the market today. Only the first (automated anomaly detection) is close to mainstream adoption. The other three are where the next 24 months of category competition will happen.

How close the AI gap on your team today

Three practical moves that a data team can make in a week:

1. Measure your team's AI Reliability Lag. Calculate what percentage of your pipelines have any automated monitoring beyond orchestration success/failure checks. Most teams discover the number is shockingly low, often under 20%. That number is the size of the gap AI-assisted monitoring would close.

2. Pilot AI-assisted monitoring on one critical pipeline. Pick the pipeline that would hurt most if it silently broke (revenue, payments, top-of-funnel). Connect an AI-native monitoring tool (AnomalyArmor, Monte Carlo, Metaplane, Bigeye) and let it learn baselines for 7-14 days. Compare the alerts it generates against the manual checks you already have. The delta is the gap closing.

3. Measure data downtime before and after. The real metric is data downtime. Track TTD (time to detection) and TTR (time to resolution) for a month before and after introducing AI-assisted monitoring. Teams usually see TTD drop by 90%+ once detection is automated. That reduction compounds because fewer issues escape to stakeholders, which reduces the trust cost per incident.

The broader picture: 71% fear hallucinated data

The same dbt survey reported that 71% of data teams are concerned about hallucinated or incorrect data reaching stakeholders. This number sits uncomfortably next to the 72% AI-assisted coding adoption, because it implies that teams are already nervous about AI contributing to data bugs while simultaneously not using AI to catch those bugs.

Two forces are likely to close this gap over the next 12-24 months:

First, boards and executive teams will start asking about data reliability at the AI-speed of creation. "We accelerated our pipeline delivery with AI, why haven't we scaled our reliability investment?" will become a standard quarterly question. The answer "we haven't because the tools are new" has a short shelf life.

Second, as more AI-generated data errors reach stakeholders, the reputational cost of "the dashboard is wrong" will spike beyond what ambiguous ownership (41% cite this as a challenge) or data literacy gaps (36% cite this) cost today. When a board report is wrong because of AI, the board asks who is responsible. That escalation reshapes how much budget the reliability stack commands.

For context, dbt's report also showed that trust in data rose from 66% to 83% in importance year-over-year, and speed rose from 50% to 71%. Teams are asking their platforms for both, and the two usually fight each other. AI-assisted pipeline management is the only way to get both at once.

What this means for how data platforms should evolve

Three predictions based on the gap:

Observability will merge with assistants. The typical data observability product will shift from "open our dashboard" to "ask the AI in your IDE or Slack." The dashboard becomes secondary. Tools that cannot be operated from Claude Code, Cursor, ChatGPT, or Slack will get disintermediated.
Monitoring setup will become a prompt, not a process. The hours of click-through configuration that current data observability tools require will collapse to "watch my warehouse" and the agent handles the rest. Sub-10-minute time-to-value will be the minimum bar.
The $5/table price point will pull the category down. Enterprise-priced data observability (Monte Carlo, Bigeye at $50-150K/year) will lose share to AI-native tools that pass the savings of automation through to the customer. Monte Carlo's 30% layoff in April 2026 is probably the first public signal of this shift.

Data Engineering AI Gap FAQ

What is the AI gap in analytics engineering?

The AI gap in analytics engineering is the 48-percentage-point difference between how many data teams prioritize AI-assisted coding (72%) versus AI-assisted pipeline management (24%), per dbt's 2026 State of Analytics Engineering report. The gap describes an industry-wide pattern where AI accelerates pipeline creation but not pipeline reliability, leaving teams with faster-growing infrastructure that is no better monitored than it was before.

Why are data teams slower to adopt AI for pipeline management than for coding?

AI-assisted coding has immediate feedback (seconds), visible failure modes (code errors), mature tools (Copilot, Cursor, Claude Code), and a quantifiable productivity story ("I wrote this in 5 minutes"). Pipeline management has delayed feedback (hours to days), silent failure modes (pipelines run green while producing wrong data), immature AI-native tools, and an inverted productivity story (value shows up as incidents that didn't happen).

What is the AI Reliability Lag?

The AI Reliability Lag is the 12-36 month delay between AI adoption in a creation step (writing code, drafting content, designing) and AI adoption in the corresponding reliability step (testing, reviewing, monitoring). In data engineering specifically, the lag is 24-36 months and has a high cost because bugs that slip past creation silently corrupt downstream analytics.

How much does the AI gap cost in data downtime?

Teams without automated monitoring typically experience 100+ hours of data downtime per month. Teams with basic monitoring see 40-80 hours. Teams with full AI-assisted data observability target less than 4 hours. At a conservative $100/hour engineering cost and $1000/incident business impact, a team with 10 incidents per month can spend $20,000+/month on preventable downtime.

What does AI-assisted pipeline management actually do?

AI-assisted pipeline management does four things: (1) learns statistical baselines for freshness, volume, and distribution so monitors do not require manual thresholds; (2) accepts natural-language intent to set up new monitors ("watch the orders table for schema changes"); (3) answers natural-language questions about data state during incidents; (4) generates context and runbooks when alerts fire so on-call engineers can resolve faster.

Which tools count as AI-assisted pipeline management?

Data observability platforms with meaningful AI integration include AnomalyArmor, Monte Carlo, Metaplane, and Bigeye. Among these, AnomalyArmor is the most aggressively AI-native: an agent sets up monitoring from a prompt, and natural-language Q&A is a first-class interface. Monte Carlo, Metaplane, and Bigeye use AI for isolated features (anomaly sensitivity tuning, alert summarization) but retain a dashboard-first workflow. Compare the category in our data observability tools 2026 roundup.

Does AI-assisted pipeline management replace dbt tests?

No. dbt tests catch what you anticipate (known constraints, specific business rules). AI-assisted pipeline management catches what you don't anticipate (unexpected schema changes, silent volume drops, distribution shifts). The two are complementary. Teams that maintain dbt tests for critical invariants and use AI-assisted monitoring for baseline coverage get both rule-based and statistical protection, which is a stronger posture than either approach alone. For more on this, see You Don't Need to Write Data Tests.

What is the difference between AI-assisted coding and AI-assisted pipeline management?

AI-assisted coding helps you write new queries, dbt models, and scripts faster. It operates at build time, before data flows through the pipeline. AI-assisted pipeline management operates at run time, after data flows through the pipeline. It watches for freshness, volume, schema, and distribution anomalies and alerts when production data deviates from expected patterns. Creation-side AI makes pipelines. Reliability-side AI keeps them alive.

Will AI replace data engineers?

No, but it will shift what data engineers spend time on. Teams that adopt AI-assisted pipeline management typically reallocate time from manual monitoring configuration and incident triage to higher-leverage work: data contracts with upstream teams, lineage hygiene, and domain modeling. The role shifts from firefighter to architect. This matches the broader industry pattern where 60% of data engineering time currently goes to firefighting, per multiple surveys.

How long does it take to close the AI gap on one team?

The minimal pilot is 7-14 days: connect an AI-native monitoring tool to one critical pipeline, let it learn baselines, and measure alert accuracy against your existing manual checks. Full rollout across a data platform usually takes 4-8 weeks, gated by the number of warehouses, tables, and integration points. Most teams see TTD drop 90%+ within the first month of serious adoption.

What is the simplest way to start?

Start by measuring your team's current TTD and TTR. Without baseline numbers, you cannot prove improvement. Then pick one pipeline (usually the one that hurts most if it breaks silently) and connect an AI-native monitoring tool. Compare the alerts it generates against manual checks for two weeks. If the automated alerts catch something the manual checks missed, the gap is real and closing it is worth scaling. If not, the gap is not your problem yet.

Is this about analytics engineering specifically or all of data?

The dbt survey focused on analytics engineering, but the pattern generalizes to any data-producing discipline: data engineering, ML engineering, data platform teams. Wherever AI is accelerating creation without proportional investment in reliability, the AI Reliability Lag applies. Analytics engineering is just where the numbers happen to be published and measurable.

Where can I read the original dbt survey?

The full report is at getdbt.com/resources/state-of-analytics-engineering-2026. The specific AI adoption numbers (72% creation, 24% monitoring) and the 71% hallucination concern are in the AI adoption section.

The AI gap closes when your data platform adopts AI for reliability, not just for creation. See how AnomalyArmor's AI agent sets up freshness, schema, and anomaly monitoring from a single prompt.