Atlan vs Collibra vs Alation vs DataHub: Data Catalogs Compared for 2026

data catalog tools are the single most-consequential platform decision a senior data engineer makes in the first ninety days on the job — and the single most-misunderstood one on both sides of the interview table. A modern warehouse holds five hundred tables the day the platform team ships it, five thousand by year two, and by year three nobody remembers which orders is the authoritative one, whether revenue is net or gross, or which downstream dashboard breaks when the int_customers mart is renamed. A data catalog comparison is not a shopping trip; it is a bet on how the organisation will find, trust, and govern its data for the next five years, and the axes that matter — active metadata versus static, data discovery UX, column-level lineage tool depth, metadata management workflow — differ so sharply between vendors that the "catalog for X" question is really "which of these four operational models fits our stewardship posture."

This guide is the senior-DE walkthrough you wished existed the last time an interviewer asked "walk me through atlan vs collibra in one paragraph, then tell me why you'd pick alation over either" or "when does a catalog for data mesh need datahub instead of a closed SaaS?" or "score the four majors on a UX × lineage × governance × cost × OSS matrix and defend every cell." It walks through why every 2026 platform team needs a catalog (the mesh-scale problem, the four axes, what interviewers probe), Atlan's discovery-first modern entrant story (design-led UX, dbt-tight ingestion, Slack workflow), Collibra's governance-heavy enterprise story (business glossary, policy engine, stewardship workflow), Alation's query-log-driven veteran catalog and DataHub's OSS-first pluggable-ingestion challenger, and the decision-matrix + 90-day-rollout patterns senior engineers ship into every catalog project. Each section pairs a teaching block with a Solution-Tail interview answer — code, a step-by-step trace, an output table, then a concept-by-concept breakdown of why it works.

When you want hands-on reps immediately after reading, drill the SQL practice library →, rehearse on the ETL practice library →, and sharpen the tuning axis with the optimization practice library →.

---

On this page

• Why every 2026 platform team needs a catalog
• Atlan — the modern discovery-first entrant
• Collibra — the governance-heavy enterprise
• Alation + DataHub — the veteran + the OSS challenger
• Picking a catalog + rollout patterns
• Cheat sheet — data catalog recipes
• Frequently asked questions
• Practice on PipeCode

---

1. Why every 2026 platform team needs a catalog

The mesh-scale problem — 500+ tables, no one knows what's authoritative, and every dashboard has a "which one is real?" comment

The one-sentence invariant: a modern data platform crosses the "nobody knows what's authoritative" threshold somewhere between 200 and 500 tables, and the cost of not having a catalog is measured in duplicated work, wrong-source dashboards, and stalled onboarding — long before it shows up as a compliance line item on the annual audit. A single warehouse in a 200-person company routinely holds five hundred physical tables, another two hundred views, three hundred dbt models across five layers, twelve BI dashboards each pulling from twenty tables, and a Slack channel where every third message is "which orders table is the real one?" The catalog is the operational answer to that question — not a nice-to-have documentation site, but the layer that makes the platform survive its own scale.

The four axes interviewers actually probe.

• Discovery UX. How fast can a new analyst find "the table with revenue for enterprise customers in Q3"? Modern catalogs bet on search-first UX with keyword + column + tag + owner facets; older catalogs bet on curated navigation. The senior signal in an interview is naming the specific search primitives (facets, filters, saved queries) rather than waving at "it has search."
• Lineage depth. Does the catalog resolve column-level lineage across dbt models, warehouse views, and BI dashboards — or does it stop at table-level edges? Column lineage is expensive to compute and rare to maintain, but it is the single feature that changes lineage from "a pretty graph nobody trusts" into "the tool the on-call runs during a schema-change incident."
• Governance workflow. How does an owner get assigned to a column? How does a PII tag propagate downstream? How does a data-subject-access-request (DSAR) query the catalog for every table containing a given user's data? The catalog is only as good as the workflow that mutates it.
• Extensibility — OSS vs closed. Can the platform team plug in a new ingestion source without a vendor engagement? Can it extend the metadata model with custom aspects (dbt tags, data contract status, cost score)? OSS-first catalogs (DataHub) treat extensibility as the primary axis; closed-SaaS catalogs (Atlan, Collibra, Alation) trade extensibility for polish.

The 2026 reality on where each vendor sits.

• Atlan. The modern discovery-first entrant. Bet: search-led UX + tight dbt/Fivetran/Snowflake ingestion + Slack-native workflow. Wins in modern warehouse shops (Snowflake + dbt + Fivetran + Looker) where the platform team wants a low-friction discovery layer without spinning up a governance council.
• Collibra. The governance-heavy enterprise incumbent. Bet: business glossary + policy engine + stewardship workflow + deep RBAC and compliance certifications. Wins in regulated industries (finance, healthcare, insurance) where the catalog must double as the system of record for policy and lineage sign-off.
• Alation. The veteran query-log-driven catalog. Bet: catalog seeded from the warehouse's query log (who queried what, how often), plus a mature stewardship model and years of enterprise deployments. Wins in mature governance shops that already have a data-steward function and want to layer the catalog onto existing warehouse traffic.
• DataHub (LinkedIn OSS + Acryl Cloud). The OSS challenger. Bet: pluggable ingestion framework + GraphQL API + column-level lineage + a Python SDK the platform team can extend without vendor tickets. Wins in engineering-heavy teams that want to own their metadata model.

Active metadata vs static metadata — the vocabulary that separates senior from mid answers.

• Static metadata. The catalog is a reflection of the warehouse — crawlers pull table names, column types, row counts on a schedule, and the catalog UI presents them. The catalog does not act on the world; it only describes it.
• Active metadata. The catalog pushes metadata into the world — a PII tag applied in the catalog propagates to a Snowflake MASKING POLICY, a deprecated flag on a table posts to a Slack channel, an ownership change triggers a Jira ticket. The catalog becomes a control plane, not just a mirror.
• The interviewer probe. "Which vendors support active metadata, and where does the boundary between reflection and action actually live?" The senior answer names two or three specific integrations (Atlan → Snowflake masking policy, DataHub → dbt exposure, Collibra → workflow-triggered Slack) rather than repeating the marketing phrase.
• The trap. Every vendor claims "active metadata" in the marketing site. The interview-defensible answer is naming the concrete write paths, not the abstract concept.

What interviewers listen for at the "why a catalog?" opening.

• Do you name the 500-table threshold or a similar scale-driven trigger? — senior signal.
• Do you cite column-level lineage as the axis that separates real from decorative catalogs? — senior signal.
• Do you distinguish active vs static metadata with a concrete integration example? — required answer.
• Do you push back on "we don't need a catalog, we have dbt docs" with the discovery-across-non-dbt-sources argument? — required answer.

Worked example — mapping the "which orders table is real?" problem to the four axes

Detailed explanation. The most common senior-DE catalog interview opens with a small, deliberately underspecified scenario: "You join a 40-person data team with 800 tables in Snowflake, five dbt projects, four Looker instances, and no catalog. Users routinely pick the wrong orders table for their dashboards. Walk me through how you'd approach this." Walk the interviewer through the four axes with concrete numbers rather than reaching for "we'd buy Collibra" in the first sentence.

• The discovery axis. With 800 tables and no search index, an analyst finds orders by asking in Slack. Time-to-answer: 20 minutes. A catalog with column + tag facets brings this to under 60 seconds.
• The lineage axis. Without lineage, "which orders feeds the CFO dashboard?" requires greping dbt SQL. With column-level lineage, the same question is one query.
• The governance axis. Without ownership, no one knows who to page when orders breaks at 3 AM. With ownership tags, the on-call pages the owning team in seconds.
• The extensibility axis. If the roadmap includes a data contract system, the catalog must let the platform team add a contract_status aspect without a vendor engagement.

Question. A 40-person data team has 800 tables in Snowflake, five dbt projects, four Looker instances, and no catalog. Users pick the wrong orders table for dashboards, on-call cannot find owners at 3 AM, and the platform lead wants to introduce a catalog in 90 days. Score the four axes and recommend the catalog.

Input.

Axis	Current state	Target state	Impact if unresolved
Discovery UX	Slack-search only	Facet search on 800 tables in < 60 s	20-min waste × 30 analysts × N per day
Lineage depth	dbt docs only, no BI edges	Column-level across dbt → Snowflake → Looker	Broken CFO dashboards on schema change
Governance	No owners, no tags	Owner + PII tag on every table	3 AM on-call cannot escalate
Extensibility	None	Data-contract aspect	Blocks contract roadmap

Code.

# Catalog scoring rubric — score 0-3 per axis
scoring:
  discovery_ux:
    weight: 0.30
    atlan:    3    # search-first UX; column facets
    collibra: 2    # UX is dated; navigation-heavy
    alation:  2    # query-log seed; workable search
    datahub:  2    # good but engineer-flavoured
  lineage_depth:
    weight: 0.25
    atlan:    3    # column-level across dbt + BI
    collibra: 2    # column-level for enterprise; slower
    alation:  2    # good for warehouse; BI edges weaker
    datahub:  3    # column-level via pluggable parsers
  governance_workflow:
    weight: 0.20
    atlan:    2    # Slack + Jira integrations
    collibra: 3    # policy engine + workflows
    alation:  3    # mature stewardship
    datahub:  1    # bring-your-own workflow
  extensibility:
    weight: 0.15
    atlan:    1    # closed SaaS
    collibra: 1    # closed SaaS
    alation:  1    # closed SaaS
    datahub:  3    # OSS + SDK + custom aspects
  cost:
    weight: 0.10
    atlan:    2    # seat-based, mid-market friendly
    collibra: 1    # enterprise seat + module
    alation:  2    # enterprise but flexible
    datahub:  3    # OSS free; Acryl Cloud tiered

# Score computer — pick the highest weighted total
def score(vendors, rubric):
    totals = {v: 0.0 for v in vendors}
    for axis, cfg in rubric.items():
        w = cfg["weight"]
        for v in vendors:
            totals[v] += w * cfg[v]
    return sorted(totals.items(), key=lambda kv: -kv[1])

rubric   = {...}   # as above
vendors  = ["atlan", "collibra", "alation", "datahub"]
ranking  = score(vendors, rubric)
# [('atlan', 2.30), ('datahub', 2.30), ('alation', 2.10), ('collibra', 1.85)]

Step-by-step explanation.

1. The rubric enumerates the four axes plus cost, with weights that reflect the team's priorities. For a mid-market platform team with a discovery-first pain point, discovery UX + lineage depth dominate (0.55 combined weight). A regulated finance team would flip this and put 0.40 on governance workflow.
2. Each vendor gets a 0–3 score per axis. The numbers are not marketing scores; they reflect the specific pain point of the 800-table Snowflake shop with dbt + Looker. Atlan scores 3 on discovery UX because dbt + Snowflake + Looker are its native integrations; the same score against a mainframe workload would drop.
3. The Python score computer multiplies weights by scores and sums per vendor. In this example Atlan and DataHub tie at 2.30, Alation at 2.10, Collibra at 1.85. The tie-break criterion is "extensibility roadmap" — if the data-contract system is a hard 6-month commitment, DataHub wins.
4. Notice that Collibra scores lowest here despite being the most-mature enterprise catalog. That is the point — the rubric encodes fit, not general market position. A different team profile would flip the ranking.
5. The recommendation is defensible because every cell can be justified with a concrete axis probe. In the interview, walk through two or three cells (Atlan on discovery, DataHub on extensibility) rather than reciting the whole table.

Output.

Vendor	Discovery	Lineage	Governance	Extensibility	Cost	Weighted total
Atlan	3	3	2	1	2	2.30
DataHub	2	3	1	3	3	2.30
Alation	2	2	3	1	2	2.10
Collibra	2	2	3	1	1	1.85

Rule of thumb. Score the catalog against your team's concrete pain, not against the vendors' marketing sites. The rubric above is a template; the weights are the interview-defensible part. Ship a rubric — even a rough one — before entering vendor negotiations.

Worked example — the 500-table threshold and the cost of no catalog

Detailed explanation. A common follow-up: "Prove the catalog pays for itself." The senior answer quantifies the cost of not having one — duplicated pipelines, wrong-source dashboards, on-call time lost to lineage searches, onboarding drag on new hires. The numbers are surprisingly easy to defend once you frame them as engineer-hours per week.

• Duplicated pipelines. With no catalog, two teams routinely build the same aggregation table. Cost: one senior-engineer-week per duplicated table × N duplications per quarter.
• Wrong-source dashboards. With no lineage, a dashboard pulls from the pre-dedup orders and shows revenue 15% high. Cost: one CFO-facing meeting to correct + weeks of erosion of trust.
• On-call lineage time. With no lineage, an on-call debugging a broken dashboard greps dbt SQL. Cost: 30 minutes per incident × 20 incidents per quarter.
• Onboarding drag. With no catalog, a new hire takes 4 weeks to reach parity on data familiarity. With a catalog, 1–2 weeks.

Question. For a 40-engineer data team, compute the annualised cost of not having a catalog. Compare against the fully-loaded seat cost of a mid-market Atlan or DataHub Acryl deployment.

Input.

Cost item	Frequency	Time cost per event	Rate	Annualised
Duplicated pipelines	3 per quarter	1 senior-week	$3000/wk	$36,000
Wrong-source dashboards	2 per quarter	3 senior-days	$600/day	$14,400
On-call lineage grep	20 per quarter	30 min	$100/hr	$4,000
Onboarding drag	8 hires per year	2 extra weeks	$2000/wk	$32,000
Trust-loss meetings	4 per year	4 senior-hours	$100/hr	$1,600

Code.

# Annualised cost-of-no-catalog calculator
COSTS = [
    ("Duplicated pipelines",   3 * 4, 1,   3000),          # per quarter, weeks, $/wk
    ("Wrong-source dashboards", 2 * 4, 3,   600),           # per quarter, days,  $/day
    ("On-call lineage grep",   20 * 4, 0.5, 100),           # per quarter, hrs,   $/hr
    ("Onboarding drag",         8,     2,   2000),          # per year,   weeks, $/wk
    ("Trust-loss meetings",     4,     4,   100),           # per year,   hrs,   $/hr
]

total = sum(freq * hours * rate for _, freq, hours, rate in COSTS)
# total = 88,000

# Fully-loaded catalog spend (mid-market, 40 seats)
CATALOG_SEATS       = 40
ATLAN_PER_SEAT_YEAR = 900     # illustrative
DATAHUB_ACRYL_YEAR  = 24_000  # tiered; illustrative

atlan_year   = CATALOG_SEATS * ATLAN_PER_SEAT_YEAR   # 36,000
datahub_year = DATAHUB_ACRYL_YEAR                    # 24,000

roi_atlan   = (total - atlan_year)   / atlan_year    # 1.44x
roi_datahub = (total - datahub_year) / datahub_year  # 2.67x

Step-by-step explanation.

1. The cost model enumerates five failure modes with a per-event time cost and a frequency. The numbers are conservative — one duplicated pipeline per quarter is a real 40-person-team baseline; two wrong-source dashboards per quarter is a routine forensics load.
2. The Python calculator sums frequency × hours × rate across all five failure modes. In this example the annualised waste is $88,000 — an entire senior-engineer for two months of the year.
3. The fully-loaded catalog spend is compared. Atlan seat pricing (illustrative $900/seat-year × 40 seats = $36,000) shows a 1.44× first-year ROI. DataHub Acryl at a tiered $24,000 shows 2.67× first-year ROI. Both defend themselves; the choice is on the qualitative axes, not the ROI axis.
4. The framing matters in the interview — the answer is not "the catalog costs $36k so it's worth it." The answer is "the cost of not having one is $88k in engineer-hours, so any catalog that clears its own price point wins." The rubric decides which catalog.
5. The most common pushback from finance is "the failure modes would happen anyway." The rebuttal is that the frequency halves under a working catalog — one duplicated pipeline per quarter becomes one per year — and the time to resolve on-call lineage drops from 30 minutes to 2 minutes.

Output.

Metric	Amount
Annualised waste with no catalog	$88,000
Atlan seat cost (40 seats)	$36,000
DataHub Acryl cost (mid-market)	$24,000
Atlan first-year ROI	1.44×
DataHub first-year ROI	2.67×

Rule of thumb. Ship an ROI model before the vendor conversation. Anchor it in engineer-hours per week, not in the vendor's slide deck. The catalog that fails the ROI test is the one you cannot afford at any price; the one that passes is the one you pick on fit, not on cost.

Worked example — the active vs static metadata boundary

Detailed explanation. A frequent senior-DE probe: "Explain the boundary between active and static metadata. Give me a concrete write path per vendor." The interviewer is checking whether the candidate has actually shipped a catalog integration or has only read the marketing site. Walk through a specific example: a PII column tag applied in the catalog must reach the warehouse as a masking policy.

• The tag. A steward marks customers.email as PII in the catalog.
• The static behaviour. The catalog UI shows a PII badge; no downstream action.
• The active behaviour. The catalog pushes a CREATE MASKING POLICY to Snowflake, tags the column, and the warehouse enforces the policy on every query.
• The failure mode. The catalog cannot push, so the steward files a Jira ticket, the platform team writes the SQL, and three weeks later the policy ships. In the meantime PII leaks.

Question. For a customers.email PII tag applied in Atlan, Collibra, Alation, and DataHub, describe the write path (or the absence of one) to a Snowflake masking policy.

Input.

Vendor	Native active-metadata support	Snowflake write path
Atlan	Yes (native integration)	Push masking policy via Snowflake API
Collibra	Yes (workflow-driven)	Workflow generates SQL and runs via Snowflake connector
Alation	Partial	Tag propagates; policy generation is a separate module
DataHub	OSS action framework	Custom action pushes policy via warehouse plugin

Code.

# DataHub OSS "action" — apply masking policy on tag change
# datahub_actions/plugin/action/masking_policy_action.py

from datahub_actions.action.action import Action
from datahub_actions.event.event_envelope import EventEnvelope
import snowflake.connector

class MaskingPolicyAction(Action):
    def act(self, event: EventEnvelope) -> None:
        if event.event_type != "EntityChangeEvent_v1":
            return
        change = event.event
        if change.category != "TAG":
            return
        if change.parameters.get("tagUrn") != "urn:li:tag:PII":
            return

        table_urn  = change.entityUrn
        column     = change.parameters.get("column")
        db, schema, table = self._parse_table_urn(table_urn)
        policy_sql = f"""
            CREATE MASKING POLICY IF NOT EXISTS pii_mask AS (val STRING)
              RETURNS STRING ->
              CASE WHEN CURRENT_ROLE() IN ('ANALYST_PII') THEN val
                   ELSE '****' END;
            ALTER TABLE {db}.{schema}.{table}
              MODIFY COLUMN {column} SET MASKING POLICY pii_mask;
        """
        with snowflake.connector.connect(**self.snowflake_config) as conn:
            conn.cursor().execute(policy_sql)

# datahub-actions config — register the action
actions:
  - type: masking_policy
    config:
      snowflake:
        account: ORG-ACC
        user: DATAHUB_ACTIONS
        role: SYSADMIN
        warehouse: OPS_WH

Step-by-step explanation.

1. The DataHub OSS action framework subscribes to EntityChangeEvent_v1 events. When a steward applies a PII tag to a column in the DataHub UI, the framework emits an event containing the entity URN, the tag URN, and the column name.
2. The MaskingPolicyAction.act method filters to tag changes on the PII tag, parses the table URN into (db, schema, table), and generates the Snowflake masking-policy SQL. The policy is idempotent (CREATE ... IF NOT EXISTS) so re-applications are safe.
3. The Snowflake connector executes the SQL as the DATAHUB_ACTIONS role, which needs USAGE on the warehouse, OWNERSHIP on the target column (or a delegated grant), and permission to create masking policies.
4. Atlan, Collibra, and Alation offer equivalent workflows but through vendor UI instead of a Python action. Atlan's Snowflake integration lets a steward click "apply policy" and pushes the SQL through a native connector. Collibra runs a workflow that generates the SQL and hands it to a Snowflake operator to apply. Alation's tag propagation is more static; policy generation is a separate module.
5. The interview-defensible answer names the concrete write path — "DataHub's OSS action framework, Atlan's Snowflake native integration, Collibra's workflow engine" — rather than the generic "yes, they support active metadata." The write path is the axis; the marketing bullet is not.

Output.

Vendor	Active-metadata write path (Snowflake PII → masking policy)
Atlan	Native Snowflake integration; one click from tag
Collibra	Workflow → SQL generator → Snowflake operator
Alation	Tag static; policy module optional
DataHub	OSS action framework; write custom Python action

Rule of thumb. In every catalog interview, be ready to name one concrete active-metadata write path per vendor. Marketing-site language ("active metadata") fails the senior probe; naming the connector, the event, the SQL, and the failure mode passes it.

Senior interview question on data catalog motivation and the four axes

A senior interviewer often opens with: "You inherit a 40-engineer data platform with 800 tables and no catalog. Walk me through the four axes you'd score, name the failure modes the catalog fixes in the first quarter, and defend the rubric weights against a CFO asking why we can't just improve dbt docs."

Solution Using the four-axis rubric plus a defensible ROI narrative

# Catalog decision — end-to-end reasoning for the "no catalog today" interview

RUBRIC = {
    # axis         : (weight, per-vendor score 0-3 for THIS team profile)
    "discovery_ux":       (0.30, {"atlan": 3, "collibra": 2, "alation": 2, "datahub": 2}),
    "lineage_depth":      (0.25, {"atlan": 3, "collibra": 2, "alation": 2, "datahub": 3}),
    "governance_flow":    (0.20, {"atlan": 2, "collibra": 3, "alation": 3, "datahub": 1}),
    "extensibility":      (0.15, {"atlan": 1, "collibra": 1, "alation": 1, "datahub": 3}),
    "cost_and_ops":       (0.10, {"atlan": 2, "collibra": 1, "alation": 2, "datahub": 3}),
}

def rank(rubric):
    scores = {}
    for axis, (weight, per_vendor) in rubric.items():
        for v, s in per_vendor.items():
            scores[v] = scores.get(v, 0.0) + weight * s
    return sorted(scores.items(), key=lambda kv: -kv[1])

print(rank(RUBRIC))
# [('atlan', 2.30), ('datahub', 2.30), ('alation', 2.10), ('collibra', 1.85)]

# Failure modes fixed in first quarter — with rubric axis provenance:
FIRST_QUARTER = [
    ("Duplicated pipelines killed by search-first discovery",   "discovery_ux",  3),
    ("On-call lineage grep replaced by column-level lineage",   "lineage_depth", 3),
    ("Owners visible in Slack via catalog-driven `/owner` cmd", "governance_flow", 2),
    ("Data-contract aspect added via custom Python action",     "extensibility", 3),
]

CFO defence — "why not just improve dbt docs?"
================================================

dbt docs cover: dbt models only (about 30% of the surface area for this team).

dbt docs miss:
  - Raw ingested tables (Fivetran, Airbyte, Kafka sinks) — 40% of surface
  - Views not authored in dbt (legacy warehouse views) — 10% of surface
  - BI dashboards (Looker, Tableau, Mode) — 15% of surface
  - Downstream products and ML feature stores — 5% of surface

dbt docs cannot: search across sources, own PII tags, integrate Slack,
                 push masking policies, page owners at 3 AM.

A catalog is the discovery + governance layer *for the 70% of surface area
that lives outside dbt*. dbt docs remain — the catalog links back to them.

Step-by-step trace.

Step	Action	Outcome
1	Score four axes with team-profile weights	Atlan + DataHub tie at 2.30
2	Break tie on extensibility roadmap	DataHub wins if contract system is 6-month commit
3	Break tie on ecosystem fit	Atlan wins if dbt + Snowflake + Looker is 90% surface
4	Quantify no-catalog cost	$88k/year engineer-time waste
5	Compare against catalog seat + ops cost	1.44×–2.67× first-year ROI
6	Present rubric + ROI + failure modes to CFO	Approval within 30 minutes

After the exercise, the platform lead has a defensible recommendation with a documented rubric, an ROI model that anchors the budget conversation, and a first-quarter roadmap that maps every activity to a rubric axis. The interview signal here is that the candidate treats the catalog decision as a bounded scoring problem, not a "gut feel" vendor pick.

Output:

Deliverable	State after exercise
Vendor ranking	Atlan and DataHub tied at 2.30
Tie-break	Extensibility (DataHub) or ecosystem fit (Atlan)
Annualised no-catalog cost	$88,000
First-year ROI (both winners)	1.44×–2.67×
First-quarter deliverables	4, each with rubric axis provenance
CFO approval time	Same-meeting, single-slide

Why this works — concept by concept:

• Four-axis rubric — a bounded scoring frame (discovery UX, lineage depth, governance workflow, extensibility, plus cost) forces vendor evaluation to be comparable rather than opinionated. The rubric is the interview artifact; the numbers are the argument.
• Team-profile weights — the weights encode this team's priorities. A regulated finance team would swap 0.30 discovery for 0.30 governance; the same rubric gives a different winner. Ship the weights explicitly so the CFO can see them.
• Failure-mode mapping — every first-quarter deliverable maps to a rubric axis. This closes the loop from "we picked axis X" to "here is the concrete outcome axis X funds."
• dbt-docs rebuttal — the "why not dbt docs" pushback is the single most common CFO question. Anchor the answer in the coverage percentage — dbt docs cover ~30% of surface area; the catalog owns the other 70%. Concrete numbers beat abstract vendor claims.
• Cost — the rubric build takes one senior-DE day. The ROI model takes another half day. Both artifacts persist across every vendor conversation. The alternative — vendor-driven scoring in vendor slides — is not free; it costs the option value of a defensible pick.

SQL
Topic — sql
SQL discovery, lineage, and metadata problems

Practice →

ETL Topic — etl ETL problems on catalog-crawler and ingestion patterns

Practice →

---

2. Atlan — the modern discovery-first entrant

Atlan's bet — search-first UX, dbt-tight ingestion, Slack-native workflow, active metadata as a first-class primitive

The mental model in one line: Atlan is what you get when you rebuild a data catalog from scratch in 2020 with the assumption that Snowflake + dbt + Fivetran + Looker + Slack is the modal enterprise stack, and the primary user is an analyst who wants to find a table in ten seconds and open a workflow in three clicks. Where Collibra optimises for governance council and Alation optimises for query-log-driven discovery, Atlan optimises for the modern-warehouse platform team that wants a low-friction discovery layer with just enough governance surface to avoid an audit finding.

The four axes for Atlan.

• Discovery UX. Search-first, keyword + facet, near-instant results across tables, columns, dashboards, terms, and people. The "who owns this?" panel is one click from every asset. Users report time-to-find in seconds, not minutes.
• Lineage depth. Column-level lineage across dbt models, Snowflake views, Fivetran/Airbyte pipelines, and Looker LookML explores. The BI edges are the differentiator — most catalogs stop at the warehouse boundary; Atlan continues to the dashboard.
• Governance workflow. Slack-native — approvals, ownership requests, term-linking questions all route to Slack channels. Jira integration handles longer-running work items. The workflow engine is intentionally lightweight; complex approval chains are Collibra territory.
• Extensibility — closed SaaS. No self-host. Custom metadata (custom attributes on assets, custom types) is supported via UI + API; deep model extension requires vendor engagement. Trade-off: fast time to value, ceiling on customisation.

The "active metadata" story in concrete terms.

• Push to Snowflake. Apply a PII classification in Atlan → Atlan pushes a masking policy attachment to the corresponding Snowflake column. Bidirectional: policy changes in Snowflake reflect back in Atlan.
• Push to dbt. Apply a deprecated tag in Atlan → Atlan surfaces the tag in the dbt project's exposures and can post a Slack notification to the owning team.
• Push to Slack. Every asset has an owner; every question about the asset routes to the owner's Slack DM or a designated channel. The catalog becomes a Slack workflow trigger.
• Push to Jira. Ownership-change requests, term-definition changes, and access requests can be routed to Jira for long-running approvals.

Pricing and packaging in 2026.

• Seat-based, mid-market friendly. Editor seats (people who write metadata) are more expensive; viewer seats are cheaper. Most 40-engineer teams provision 10–15 editor seats + all-team viewer seats.
• Modules. Core catalog + lineage is the base; column-level lineage across BI, data quality integration, and advanced governance are add-ons.
• Rollout friction. Low. Atlan ships a Snowflake connector, a dbt connector, a Fivetran connector, and Looker/Tableau connectors that can crawl a typical mid-market warehouse in a day.

The Atlan interview probes.

• "Walk me through Atlan search — what facets does it support?" — required: keyword, asset type, owner, term, tag, source system, popularity.
• "How does Atlan handle column-level lineage across dbt and Looker?" — required: named the parser (dbt manifest, Looker LookML), the propagation direction, and the failure mode (unresolvable dashboard fields).
• "When would you not pick Atlan?" — senior signal: regulated industries needing deep RBAC + workflow engine; OSS-first teams; teams outside the Snowflake + dbt + Looker corridor.
• "What's the active-metadata write path from Atlan to Snowflake?" — required: named the classification, the masking policy, the connector, and the reverse-sync.

Worked example — Atlan search operator syntax for the 800-table warehouse

Detailed explanation. An analyst on the 40-engineer platform team needs to find "the enterprise revenue table for Q3, owned by the finance team, still in production." Atlan's search bar is a keyword + facet + operator hybrid; the senior workflow uses a small vocabulary of operators (owner:, tag:, type:, term:, certified:, source:) to narrow from 800 tables to one.

• The naive workflow. Type "revenue" → scroll through 40 hits → give up.
• The senior workflow. Type owner:finance tag:enterprise term:"quarterly revenue" certified:true → three hits, one obvious winner.
• The lesson. Search primitives are worth teaching. A 30-minute internal workshop pays for itself in the first day of the pilot.

Question. For an analyst hunting the "enterprise revenue table for Q3, owned by finance, certified," write the Atlan search string and enumerate the fallback searches if the first returns zero hits.

Input.

Facet	Value
Keyword	revenue
Term	quarterly revenue
Owner	finance
Tag	enterprise
Certified	true
Time window	Q3

Code.

# Atlan search operators cheat sheet
operators:
  keyword:      free text — searches asset names, descriptions, columns
  owner:        owner:username or owner:group_slug
  term:         term:"business glossary term name"
  tag:          tag:tag_slug — filter by classification tag
  type:         type:Table|View|Column|Dashboard|BusinessTerm|Schema
  source:       source:snowflake|bigquery|redshift|looker|tableau
  certified:    certified:true|false — verified/certified assets only
  updated:      updated:>2026-06-01 — assets updated after date
  popularity:   popularity:high — sorts by query-log popularity
  deprecated:   deprecated:false — exclude deprecated assets

# The senior search string for the scenario
query: >
  revenue
  term:"quarterly revenue"
  owner:finance
  tag:enterprise
  certified:true
  source:snowflake
  deprecated:false

# Fallback ladder — relax one facet at a time if hits == 0
fallbacks:
  - drop certified:true      # accept uncertified until a certified one is created
  - drop tag:enterprise       # widen to all customer segments
  - drop term:"quarterly revenue"   # widen keyword only
  - drop owner:finance        # ask "does any team own a revenue table?"

# Programmatic search via Atlan Python SDK
from pyatlan.client.atlan import AtlanClient
from pyatlan.model.search import IndexSearchRequest, DSL, Term, Bool

client = AtlanClient()
request = IndexSearchRequest(
    dsl=DSL(
        query=Bool(
            must=[
                Term(field="name.text",             value="revenue"),
                Term(field="ownerGroups",           value="finance"),
                Term(field="atlanTags",             value="enterprise"),
                Term(field="certificateStatus",     value="VERIFIED"),
                Term(field="sourceEmbedded.source", value="snowflake"),
            ],
            must_not=[
                Term(field="isDeprecated", value=True),
            ],
        ),
        size=25,
    ),
    attributes=["name", "description", "ownerGroups", "atlanTags", "certificateStatus"],
)
result = client.asset.search(request)
for asset in result.assets:
    print(asset.name, "-", asset.description[:60])

Step-by-step explanation.

1. The Atlan search language is a keyword + facet + operator hybrid. The bare keyword revenue searches asset names, descriptions, and column names. Each additional operator (owner:, tag:, term:, certified:) narrows the result set.
2. The senior workflow front-loads the operators. Starting with a broad keyword and adding operators as the result set stays too large is the mid-level workflow. The senior workflow writes the fully-qualified string in one pass.
3. The fallback ladder is the discipline that separates a working search from a frustrated one. If the first string returns zero hits, drop one facet at a time in reverse priority order: certification first (accept uncertified), then tag (widen segment), then term (widen keyword), then owner (ask for any team).
4. The Python SDK example is the automation path — the same query as a programmatic call, used for building Slack bots that answer "find revenue table for finance" without a human clicking through the UI.
5. The output includes name, description, owner, tags, and certification. In practice, the analyst clicks through to the top hit, verifies the description matches, and copies the fully-qualified name into the dbt or SQL editor.

Output.

Step	Query state	Hits	Action
Full string	revenue term:"quarterly revenue" owner:finance tag:enterprise certified:true	1	pick winner
Drop certified	revenue term:"quarterly revenue" owner:finance tag:enterprise	3	pick best
Drop tag	revenue term:"quarterly revenue" owner:finance	8	narrow further
Drop term	revenue owner:finance	15	still workable
Drop owner	revenue	40	too broad; try again

Rule of thumb. Teach the operator vocabulary in a 30-minute onboarding session. Ship a "search cheat sheet" wiki page on day one of the rollout. Every analyst who learns two operators pays back the training in a week.

Worked example — column-level lineage across dbt + Snowflake + Looker

Detailed explanation. Column-level lineage is the axis that changes lineage from decorative to operational. When the orders.customer_id column is renamed in the raw ingested table, the on-call needs to know every downstream column that transitively depends on it — the dbt models, the mart tables, the Looker measures, the dashboards. Atlan's column-level lineage answers this with a single graph query.

• The trigger. Schema change on raw.orders.customer_id.
• The naive workflow. Grep dbt SQL for customer_id; hope Looker LookML is greppable; guess at the BI impact.
• The Atlan workflow. Open the lineage panel on raw.orders.customer_id; expand column-level downstream; see every dependent column, model, and dashboard in one view.

Question. For a rename raw.orders.customer_id → raw.orders.buyer_id, produce the Atlan lineage query and the downstream impact list. Then propose the migration order.

Input.

Layer	Object	Depends on
Raw	raw.orders.customer_id	(source)
Staging	dbt.stg_orders.customer_id	raw.orders.customer_id
Intermediate	dbt.int_orders.customer_id	dbt.stg_orders.customer_id
Mart	dbt.mart_finance.customer_id	dbt.int_orders.customer_id
BI	looker.finance.explore.orders.customer_id	dbt.mart_finance.customer_id
Dashboard	looker.dashboards.cfo_dashboard.chart_37	looker.finance.explore.orders.customer_id

Code.

# Atlan GraphQL — column-level downstream lineage
query DownstreamLineage($guid: String!) {
  columnLineage(guid: $guid, direction: "downstream", depth: 10) {
    nodes {
      guid
      typeName
      attributes {
        name
        qualifiedName
        connectorName
        certificateStatus
        ownerGroups
      }
    }
    edges {
      source { guid }
      target { guid }
      transformation
    }
  }
}

# Result summary — 6 dependent assets across 3 systems
downstream_impact:
  - qualifiedName: dbt.stg_orders.customer_id
    system: dbt
    owner: platform
    action: rename in stg_orders.sql; ship dbt PR
  - qualifiedName: dbt.int_orders.customer_id
    system: dbt
    owner: platform
    action: rename in int_orders.sql; ship dbt PR
  - qualifiedName: dbt.mart_finance.customer_id
    system: dbt
    owner: finance
    action: rename in mart_finance.sql; ship dbt PR; announce in #data-changes
  - qualifiedName: looker.finance.explore.orders.customer_id
    system: looker
    owner: finance-analytics
    action: rename in LookML view; ship Looker PR
  - qualifiedName: looker.dashboards.cfo_dashboard.chart_37
    system: looker
    owner: finance-analytics
    action: no change required (LookML abstracts field name)
  - qualifiedName: raw.orders.customer_id
    system: snowflake
    owner: platform
    action: apply rename; source of truth

Step-by-step explanation.

1. The GraphQL query asks Atlan for column-level downstream lineage starting from raw.orders.customer_id, up to 10 hops deep. The response is a graph — nodes are columns, edges are transformations (dbt SELECT, Looker LookML measure, etc.).
2. The result summary is the operational artefact — a per-asset action list ordered by migration order. The source-of-truth column ships first, then dbt (leaf → root), then Looker (LookML → dashboards). The order matters because a mid-migration state where dbt has been renamed but Snowflake hasn't leaves the pipeline broken.
3. The transformation edge tells the on-call how the column was derived. A pass-through (SELECT customer_id AS customer_id) is safe to rename mechanically. A calculated column (SELECT COALESCE(customer_id, alt_id) AS customer_id) needs review.
4. Owner groups on every node let the platform lead route the migration to the right team without a Slack search. The dbt models under dbt.mart_finance.* are owned by the finance team; the platform team can open the PRs but must get sign-off from finance.
5. The dashboard edge (chart_37) shows the LookML abstraction win — because Looker views abstract field names, the dashboard doesn't need a change; only the LookML view does. Without column-level lineage, the on-call would have manually opened every dashboard.

Output.

Layer	Downstream nodes	Migration order	Action
Raw	1	1	Rename source
dbt staging	1	2	Rename model
dbt intermediate	1	3	Rename model
dbt mart	1	4	Rename + coordinate with finance
Looker LookML	1	5	Rename LookML view field
Looker dashboards	1	(no change)	Abstract via LookML

Rule of thumb. Column-level lineage is the axis that turns "hope no dashboards break" into a deterministic checklist. If the catalog does not give you a graph query that returns the downstream columns for a rename, it is not a lineage tool — it is a picture of one.

Worked example — Slack-native workflow for ownership + PII

Detailed explanation. Atlan's active-metadata story is most concrete in the Slack integration. When a steward applies a PII tag to a column, an owner is auto-notified in the corresponding Slack channel, a Jira ticket is opened for the masking-policy work, and the catalog UI shows the tag status in real time. Walk through the specific flow so an interviewer can see how the vendor claims map to concrete pipes.

• The tag. Steward applies PII to customers.email in the Atlan UI.
• The Atlan side. Event is emitted, subscribers listen.
• The Slack side. #pii-owners-finance channel receives a message with the asset link and the recommended action.
• The Jira side. A ticket is filed against the platform-team board with the asset details.
• The Snowflake side. The masking policy is queued for the next platform-team sprint.

Question. Wire the end-to-end workflow: PII tag → Slack notification → Jira ticket → Snowflake masking policy. Show the Atlan-side webhook config and the receiving Slack/Jira/Snowflake side.

Input.

Component	Value
Tag	PII
Asset	snowflake.raw.customers.email
Owner group	finance-data
Slack channel	#pii-owners-finance
Jira project	PLAT (platform team board)

Code.

# Atlan workflow YAML — declaratively wire the tag change event
name: pii-tag-workflow
trigger:
  type: entityChange
  filters:
    - path: attributes.classifications
      op: contains
      value: PII

actions:
  - id: notify-slack
    type: slack
    config:
      channel: "#pii-owners-finance"
      message: |
        :warning: PII tag applied by {{ actor.name }} to
        <{{ asset.url }}|{{ asset.qualifiedName }}>
        Owner: {{ asset.ownerGroups[0] }}
        Please review the masking policy checklist.

  - id: open-jira
    type: jira
    config:
      project: PLAT
      issueType: Task
      summary: "Apply PII masking policy to {{ asset.qualifiedName }}"
      description: |
        Column {{ asset.qualifiedName }} was tagged PII by {{ actor.name }}.
        Steward: {{ actor.email }}
        Please attach the pii_mask policy in Snowflake.

  - id: enqueue-snowflake-job
    type: webhook
    config:
      url: https://ops.internal/webhooks/atlan/masking-policy
      method: POST
      body:
        asset: "{{ asset.qualifiedName }}"
        tag: PII

# Receiving webhook — apply the masking policy
from flask import Flask, request, abort
import snowflake.connector, hmac, hashlib, os

app = Flask(__name__)
SECRET = os.environ["ATLAN_WEBHOOK_SECRET"].encode()

@app.post("/webhooks/atlan/masking-policy")
def apply_policy():
    sig = request.headers.get("X-Atlan-Signature", "")
    body = request.get_data()
    expected = hmac.new(SECRET, body, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(sig, expected):
        abort(401)

    payload = request.get_json()
    fqn = payload["asset"]                    # e.g. snowflake.raw.customers.email
    _, db, schema, table_col = fqn.split(".", 3)
    table, column = table_col.rsplit(".", 1)

    with snowflake.connector.connect(**cfg) as conn:
        conn.cursor().execute(f"""
            ALTER TABLE {db}.{schema}.{table}
              MODIFY COLUMN {column}
              SET MASKING POLICY pii_mask
        """)
    return "", 204

Step-by-step explanation.

1. The Atlan workflow YAML is a declarative event → action mapping. The trigger fires on any entityChange where the classifications include PII; the actions run in order (Slack notify → Jira open → Snowflake webhook).
2. The Slack action posts a message with a link back to the asset, the owner group, and a call-to-action. The channel is chosen by owner group (#pii-owners-finance for finance-owned assets), so the notifications land in the right team's channel without a global broadcast.
3. The Jira action files a ticket on the platform team's board. This is the durable work-tracking layer — Slack notifications get lost; Jira tickets survive rotation. The description includes the asset qualified name, the steward, and the checklist reference.
4. The Snowflake webhook is the active-metadata action. Atlan POSTs the asset qualifier to ops.internal, which validates the HMAC signature, parses the qualified name into (db, schema, table, column), and runs the ALTER TABLE ... SET MASKING POLICY SQL. The masking policy pii_mask is defined once, out of band; the workflow only attaches it.
5. The HMAC signature validation is non-negotiable. Any inbound webhook that touches the warehouse must verify the caller. Without HMAC, any attacker who guesses the endpoint URL can push arbitrary masking policies.

Output.

Step	Action	Latency	Failure mode
Steward tags column PII	Atlan UI	—	—
Atlan emits event	Atlan	< 1 s	Event queue lag
Slack notify	Atlan → Slack API	1–3 s	Channel not found
Jira ticket	Atlan → Jira API	1–3 s	Board misconfigured
Snowflake masking policy	Atlan → webhook → Snowflake	3–5 s	HMAC mismatch
Column masked in warehouse	Snowflake DDL	1 s	Permission denied

Rule of thumb. Ship the Slack + Jira + warehouse write path on day one of the Atlan rollout. The catalog that only shows metadata is a wiki; the catalog that acts on the world is a control plane. The workflow YAML above is the archetype — start narrow (PII), expand from there.

Senior interview question on Atlan for a modern warehouse team

A senior interviewer might ask: "You're the incoming platform lead for a 40-engineer team on Snowflake + dbt + Looker + Slack. Walk me through the Atlan pilot — what you'd crawl first, how you'd measure success in 30 / 60 / 90 days, and what would push you to escalate to Collibra."

Solution Using a phased Atlan pilot with concrete success metrics

# 90-day Atlan pilot — declarative plan
pilot:
  team_size: 40
  stack: [snowflake, dbt, looker, slack, jira]
  budget: $36000/year (40 seats, mid-market illustrative)

phase_1_crawl:
  days: 0-14
  sources:
    - snowflake:
        databases: [raw, dbt_prod, mart_prod]
        objects: tables, views, materialized_views
    - dbt:
        projects: [core, finance, growth, ops, ml]
        artifacts: [manifest.json, catalog.json]
    - looker:
        instances: [prod-1, prod-2, prod-3, prod-4]
        objects: [models, explores, dashboards]
  expected_asset_count: ~2000
  success_metric: 95% of production tables + dashboards indexed

phase_2_ownership:
  days: 14-30
  activity:
    - assign owner groups per source system
    - link business glossary terms to top-100 tables by popularity
    - certify 20 highest-value marts as VERIFIED
  success_metric: median asset has 1+ owner + description

phase_3_workflow:
  days: 30-60
  workflows:
    - pii_tag_to_snowflake_masking_policy
    - deprecated_tag_to_slack_channel
    - ownership_change_to_jira_ticket
  success_metric: 5+ workflows live; 10+ active-metadata write events per week

phase_4_adoption:
  days: 60-90
  activity:
    - all-hands training + operator cheat sheet
    - Slack `/atlan` command deployed
    - onboarding checklist replaces "ask in Slack"
  success_metric: 60% of new-hire questions resolved via catalog search

escalate_to_collibra_if:
  - regulatory audit mandates a policy engine + workflow chain
  - stewardship function grows beyond 5 dedicated stewards
  - RBAC needs deep row-level and column-level policy modelling

Step-by-step trace.

Phase	Day range	Deliverable	Success gate
Crawl	0–14	Snowflake + dbt + Looker indexed	95% production coverage
Ownership	14–30	Owners + terms on top 100 assets	Median asset owned
Workflow	30–60	5+ workflows live	10+ writes / week
Adoption	60–90	Cheat sheet + Slack command + onboarding	60% questions self-serve
Escalate	if audit / stewards / RBAC bloom	Collibra RFP	—

After 90 days the pilot has 2000 indexed assets, 5 active-metadata workflows shipping writes into Snowflake and Slack, and a documented "when to escalate" criterion. The interview signal is that the candidate treats the rollout as a phased pilot with metrics, not a "we bought Atlan, now what?" scramble.

Output:

Surface	30-day metric	60-day metric	90-day metric
Assets crawled	~1500	~2000	~2200
Owned assets (%)	40	65	80
Active workflows	0	3	5+
Weekly active-metadata writes	0	20	50+
Self-serve question resolution	20%	45%	60%

Why this works — concept by concept:

• Phase gates — crawl before ownership before workflow before adoption. Skipping a phase (e.g. workflow before ownership) means the workflows have no target audience. The phase order is the pilot's operational spine.
• Success metrics per phase — every phase has one measurable outcome that the platform lead reports at the phase-review meeting. "60% self-serve" is a defensible pilot success; "the catalog is working" is not.
• Escalation criteria — pre-declare the conditions that flip the recommendation to Collibra (audit, steward growth, RBAC). This is the senior discipline that separates "we're locked in" from "we're operating a bounded pilot with an exit strategy."
• Slack /atlan command — the highest-leverage adoption trick. Every question that lands in #data-help gets a /atlan search {q} reply from a bot. The catalog meets users where they already are.
• Cost — 40 seats at $900/seat-year = $36k. Amortised across the $88k annualised waste, the pilot pays back in six months. The 90-day sprint is one senior-DE half-time; the pay-off is the four full-time-equivalent hours per week the team stops losing to "which orders is real?"

SQL
Topic — sql
SQL search, lineage, and discovery problems

Practice →

Optimization Topic — optimization Optimization problems on catalog rollout planning

Practice →

---

3. Collibra — the governance-heavy enterprise

Collibra's bet — business glossary as the system of record, policy engine as a workflow layer, stewardship as a first-class function

The mental model in one line: Collibra is what happens when you build a catalog for a regulated bank or hospital in 2015 and never stop investing in the workflow engine, the business glossary, and the deep RBAC surface that lets a Chief Data Officer sign off on data usage as an audit line item. Where Atlan optimises for the analyst finding a table and Alation optimises for the query-log-driven discovery layer, Collibra optimises for the Chief Data Officer who needs a documented policy chain from "column-level PII classification" to "who approved this quarter's DSAR." The catalog is the system of record for governance, not just a mirror of the warehouse.

The four axes for Collibra.

• Discovery UX. Navigation-first (browse the glossary hierarchy) plus keyword search. Modern versions have improved search, but the primary metaphor is "you know the term, find the assets that instantiate it." Less immediate than Atlan; more principled for governance-first workloads.
• Lineage depth. Column-level lineage across warehouses; BI-side coverage is deep for the top vendors (Tableau, Power BI) and workable elsewhere. The differentiator is not depth but ownership-traceable lineage — every edge has a documented owner and a review cadence.
• Governance workflow. The Collibra Workflow Engine (BPMN-based) is the killer feature. Approvals, stewardship handoffs, DSAR queries, policy propagation — all modelled as declarative workflows that produce audit-loggable steps.
• Extensibility — closed SaaS with deep customisation surface. No self-host, but the workflow engine, the operating model, and the metadata types are all customisable through the UI + API. Vendor-engagement threshold is higher than Atlan for deep model changes; day-to-day workflow authoring is self-serve.

The Collibra operating model — the vocabulary interviewers probe.

• Business Asset. Terms (business concepts), Policies (rules), Data Domains (organisational buckets), KPIs. The glossary is the top layer.
• Data Asset. Tables, columns, files, reports. The warehouse-level surface.
• Governance Asset. Roles, responsibilities, workflows, communities. The org layer.
• The link. Business Assets connect to Data Assets via "represents" relationships. Governance Assets connect to both via "owns" and "steward" relationships. The three layers give you a defensible policy chain: policy → term → column → dashboard.

The workflow engine in concrete terms.

• BPMN-based. Each workflow is a directed graph of tasks (user tasks, service tasks, timer tasks) modelled in Business Process Model and Notation.
• User tasks. Show up as work items in the assigned user's Collibra inbox. Approvals, reviews, definition changes.
• Service tasks. Automated — REST calls to external systems, DDL against the warehouse, notifications.
• Timer tasks. SLA enforcement — if a step isn't done in N days, escalate.
• Audit log. Every step is timestamped, actor-tagged, and reviewable. This is the compliance win.

Pricing and packaging in 2026.

• Enterprise seat + module. Editor seats + governance modules (business glossary, policy manager, data helpdesk, data privacy). Not mid-market friendly; typical entry point is 200+ seats.
• Rollout friction. Higher than Atlan. Expect 3–6 months to first production workflow; 12+ months to full stewardship program.
• Sweet spot. Regulated industries (finance, healthcare, insurance, pharma), large enterprises with dedicated data-governance teams, organisations where "the catalog" and "the compliance system" must be the same thing.

The Collibra interview probes.

• "Explain the Collibra operating model." — required: name Business Asset / Data Asset / Governance Asset and the connecting relationships.
• "How would you model a DSAR workflow in Collibra?" — senior signal: BPMN with user task (verify identity) → service task (query catalog for PII tables) → user task (approve deletion) → service task (execute deletion) → user task (sign-off).
• "When is Collibra the wrong answer?" — required: mid-market teams with no dedicated stewardship function, engineering-heavy teams that want to script their metadata layer, teams outside regulated industries.
• "How does Collibra handle a business term change?" — senior signal: change proposal → steward review → approval workflow → propagation to Data Assets → audit-log entry.

Worked example — modelling a PII policy workflow in Collibra

Detailed explanation. The archetypal Collibra workflow: a steward proposes a PII classification on a column; the workflow routes the proposal through a review chain, applies the classification on approval, propagates a masking policy to the warehouse, and closes with an audit-log entry. Walk through the BPMN structure so an interviewer can see the workflow-engine advantage over Atlan's lighter Slack + Jira model.

• The trigger. Steward submits a "propose PII classification" request in Collibra.
• The workflow. Steward proposal → Data owner review → Data protection officer sign-off → Warehouse masking → Audit-log entry.
• The SLA. Each step has a timer; missed timers escalate to the next-up manager.

Question. Model the PII classification workflow as a BPMN process. Show the tasks, the routing, and the audit-log fields.

Input.

Task	Type	Actor	SLA
Propose classification	User task	Steward	—
Review classification	User task	Data owner	3 days
Sign-off	User task	DPO	5 days
Apply masking policy	Service task	Warehouse connector	1 hour
Notify stakeholders	Service task	Slack + email	1 minute
Close and audit	Service task	Audit-log writer	1 minute

Code.

<!-- BPMN 2.0 — PII classification workflow (excerpt) -->
<bpmn:process id="PIIClassificationWorkflow" isExecutable="true">

  <bpmn:startEvent id="Start" name="Steward proposes PII"/>

  <bpmn:userTask id="Review" name="Data owner review"
                 collibra:assignee="${asset.owner}"
                 collibra:dueDate="${today + 3d}">
    <bpmn:documentation>
      Review the proposed PII classification for {{ asset.qualifiedName }}.
      Approve, reject, or request more information.
    </bpmn:documentation>
  </bpmn:userTask>

  <bpmn:userTask id="Signoff" name="DPO sign-off"
                 collibra:candidateGroup="data-protection-office"
                 collibra:dueDate="${today + 5d}"/>

  <bpmn:serviceTask id="ApplyMask" name="Apply Snowflake masking"
                    collibra:type="rest"
                    collibra:url="https://ops.internal/webhooks/collibra/masking"
                    collibra:body='{"asset":"${asset.qualifiedName}"}'/>

  <bpmn:serviceTask id="Notify" name="Slack + email notify"
                    collibra:type="notification"
                    collibra:channels="slack,email"/>

  <bpmn:serviceTask id="Audit" name="Write audit entry"
                    collibra:type="audit"
                    collibra:fields="actor,timestamp,asset,decision"/>

  <bpmn:endEvent id="End" name="Classification live"/>

  <!-- Sequence flows (simplified) -->
  <bpmn:sequenceFlow sourceRef="Start"    targetRef="Review"/>
  <bpmn:sequenceFlow sourceRef="Review"   targetRef="Signoff"    conditionExpression="${approved}"/>
  <bpmn:sequenceFlow sourceRef="Signoff"  targetRef="ApplyMask"  conditionExpression="${approved}"/>
  <bpmn:sequenceFlow sourceRef="ApplyMask" targetRef="Notify"/>
  <bpmn:sequenceFlow sourceRef="Notify"   targetRef="Audit"/>
  <bpmn:sequenceFlow sourceRef="Audit"    targetRef="End"/>

  <!-- Escalations on SLA miss -->
  <bpmn:boundaryEvent attachedToRef="Review" cancelActivity="false">
    <bpmn:timerEventDefinition>
      <bpmn:timeDuration>P3D</bpmn:timeDuration>
    </bpmn:timerEventDefinition>
  </bpmn:boundaryEvent>

</bpmn:process>

# Audit-log record produced at Close step
audit_entry:
  workflow: PIIClassificationWorkflow
  workflow_instance: 4c7f-abd1-...
  asset: snowflake.raw.customers.email
  steward: [e-mail]
  owner_review:
    actor: [e-mail]
    decision: approved
    timestamp: 2026-07-04T10:33:00Z
    comment: "Confirmed PII per handbook §4.2"
  dpo_signoff:
    actor: [e-mail]
    decision: approved
    timestamp: 2026-07-05T09:12:00Z
  masking_policy_applied:
    policy: pii_mask
    warehouse: snowflake
    executed_at: 2026-07-05T09:14:00Z
  final_state: LIVE

Step-by-step explanation.

1. The BPMN diagram encodes the workflow as a sequence of tasks. Start events kick off; user tasks route to a person or group; service tasks call external systems; end events close the instance. Every task is timestamped and auditable.
2. The user tasks (Review, Signoff) show up in the assignee's Collibra inbox. The assignee sees the asset context, a decision panel (approve / reject / request more info), and the due date. Missed SLAs trigger the boundary timer event, which escalates to the next-up manager.
3. The service tasks (ApplyMask, Notify, Audit) run without human intervention. ApplyMask POSTs to the same ops-internal webhook that the Atlan example used — Collibra is not replacing the warehouse connector, it is orchestrating the write with an audit-trailed workflow.
4. The audit entry produced at the Close step is the compliance payoff. Every actor, every decision, every timestamp is captured and queryable. When the annual audit asks "who approved the PII classification on customers.email in Q3?", the answer is one query against the audit log.
5. Compared to Atlan's YAML workflow (which fires Slack + Jira + webhook and forgets), Collibra's BPMN workflow persists the entire decision chain. The trade-off is authoring cost — BPMN takes real modelling — vs audit-defensibility. Regulated industries need the latter; mid-market teams often do not.

Output.

Field	Value	Purpose
workflow	PIIClassificationWorkflow	Which template
asset	snowflake.raw.customers.email	Target asset
owner_review.actor	[e-mail]	First approval
dpo_signoff.actor	[e-mail]	Compliance approval
masking_policy_applied.policy	pii_mask	Active-metadata write
final_state	LIVE	Terminal state

Rule of thumb. If the interviewer asks "how would you model this workflow?", model it. BPMN in the answer is a senior signal; hand-waving at "we'd wire it up" is not. The audit-log field list is the second-order signal — the workflow that doesn't produce an audit entry is not a governance workflow, it is a Slack bot.

Worked example — DSAR (data subject access request) end-to-end

Detailed explanation. A DSAR is the GDPR-defined request from a user to see or delete every piece of data an organisation holds about them. The catalog role is find every table that contains PII for this user, coordinate the response, and audit-log the actions. Walk through how Collibra's catalog + workflow engine handles a DSAR end-to-end.

• The trigger. User submits a "delete my data" request through the company website.
• The catalog role. Given the user's ID, find every table in the warehouse tagged PII that contains a row for this user.
• The workflow role. Route the deletion through legal review, execute the deletions, notify the user of completion.

Question. Design the DSAR workflow. Show the catalog query that finds the impacted tables and the BPMN skeleton that drives the response.

Input.

Component	Value
DSAR type	Deletion
User identifier	user_id = 4247
SLA (GDPR)	30 days
Catalog tag driving discovery	PII

Code.

# Collibra GraphQL — find every asset tagged PII across the warehouse
query PIIAssets {
  assets(
    filter: {
      typeName: "Column"
      tags: { contains: "PII" }
    }
    first: 500
  ) {
    edges {
      node {
        qualifiedName
        parent { qualifiedName }
        tags
        owner { name }
      }
    }
  }
}

-- Warehouse query — driven by catalog result
-- For each PII-tagged column, generate the deletion SQL
-- (real Collibra Data Privacy module emits parameterized deletions)

DELETE FROM raw.customers      WHERE customer_id = 4247;
DELETE FROM raw.orders         WHERE customer_id = 4247;
DELETE FROM raw.support_tickets WHERE customer_id = 4247;
DELETE FROM raw.marketing_events WHERE user_id    = 4247;
-- ... one per PII-tagged parent table ...

<!-- BPMN skeleton — DSAR deletion workflow -->
<bpmn:process id="DSARDeletion">
  <bpmn:startEvent id="Start" name="DSAR submitted"/>

  <bpmn:userTask id="VerifyIdentity" name="Legal verifies identity"
                 collibra:candidateGroup="legal"
                 collibra:dueDate="${today + 3d}"/>

  <bpmn:serviceTask id="QueryCatalog" name="Query catalog for PII tables"
                    collibra:type="graphql"
                    collibra:query="PIIAssets"/>

  <bpmn:userTask id="Approve" name="DPO approves deletion plan"
                 collibra:candidateGroup="data-protection-office"
                 collibra:dueDate="${today + 5d}"/>

  <bpmn:serviceTask id="Execute" name="Run deletions"
                    collibra:type="rest"
                    collibra:url="https://ops.internal/webhooks/dsar/execute"/>

  <bpmn:serviceTask id="NotifyUser" name="Email user"
                    collibra:type="email"
                    collibra:template="dsar_complete"/>

  <bpmn:serviceTask id="Audit" name="Write audit"
                    collibra:type="audit"/>

  <bpmn:endEvent id="Done" name="DSAR closed"/>
</bpmn:process>

Step-by-step explanation.

1. The catalog query drives the discovery. Instead of grepping the warehouse for tables with customer_id, the workflow queries Collibra's GraphQL for every column tagged PII. The catalog is authoritative because the stewardship workflow (previous worked example) is the process that applied every PII tag.
2. The BPMN process starts with a user task — Legal verifies the identity of the requesting user. This is a compliance requirement, not a technical one; the catalog is not the source of truth for identity verification.
3. The service task QueryCatalog runs the GraphQL query, receives the list of impacted parent tables, and stores it as workflow variables. The next user task shows the DPO the deletion plan as a summary; the DPO can approve or ask for exceptions (e.g., legally-required retention).
4. The service task Execute calls the ops webhook with the approved deletion plan. The webhook (implemented as a separate service — Collibra does not run DDL directly) runs the parameterised deletions against the warehouse, one per PII-tagged parent table.
5. The workflow closes with an email to the user and an audit entry. The audit entry contains every actor, every decision, and every SQL statement executed. When the regulator later asks "prove you deleted this user's data," the answer is one audit-log query.

Output.

Step	Actor	Latency	Audit fields
DSAR submitted	User via website	0	—
Verify identity	Legal	1–3 days	actor, ticket ID
Query catalog	Automated	seconds	GraphQL response, N tables
DPO approval	DPO	1–5 days	actor, decision, exceptions
Execute deletions	Automated	1–10 min	SQL, affected rows
Notify user	Automated	seconds	email delivered
Audit written	Automated	seconds	complete decision chain

Rule of thumb. DSAR is the interview scenario that makes Collibra's workflow engine shine. If the interviewer's role is regulatory-adjacent, walk through DSAR before walking through PII tagging — the workflow chain across identity + catalog + warehouse + audit is exactly what Collibra was designed to model.

Worked example — business glossary rollout in a regulated bank

Detailed explanation. The business glossary is Collibra's front door for the non-technical stakeholder. Terms like revenue, churn, active user, restricted party need one authoritative definition, one owner, and one propagation path down to the physical tables. Walk through the rollout in a regulated bank so the interviewer can see the operating model in production.

• The scope. 400 business terms spanning finance, risk, compliance, customer, product.
• The owners. A Term Owner (business), a Steward (technical), a Working Group (approvers).
• The propagation. Each term links to N Data Assets (physical columns) via a "represents" relationship.

Question. Rollout plan for the 400-term glossary. Show the operating model and one worked term end-to-end.

Input.

Element	Value
Term count	400
Domains	5 (finance, risk, compliance, customer, product)
Term owner role	Business, per domain
Steward role	Technical, per domain
Working group	Approves term definitions

Code.

# Collibra operating model — YAML export for one term
term:
  name: "Net Revenue"
  domain: Finance
  definition: >
    Gross revenue less returns, discounts, and taxes. Recognised
    at the point of shipment for physical goods and at the point of
    subscription-renewal for SaaS. Aligned with the finance policy
    document FIN-2025-014.
  synonyms:
    - Net Sales
    - Recognised Revenue
  owner:
    business: cfo-office
    steward: platform-finance
    working_group: finance-glossary-wg
  status: APPROVED
  approved_at: 2026-05-12
  represents:
    - snowflake.mart_finance.fct_revenue_daily.net_revenue
    - snowflake.mart_finance.fct_revenue_monthly.net_revenue
    - looker.finance.explore.revenue.net_revenue
  related_terms:
    - Gross Revenue
    - Deferred Revenue
    - Recognised Revenue Adjustment
  regulatory_refs:
    - ASC 606 (revenue recognition)
    - policy: FIN-2025-014

<!-- Term change workflow — every edit to the term goes through here -->
<bpmn:process id="TermChangeWorkflow">
  <bpmn:startEvent id="Start" name="Proposed edit"/>

  <bpmn:userTask id="StewardReview" name="Steward review"
                 collibra:assignee="${term.owner.steward}"/>

  <bpmn:userTask id="WGVote" name="Working group vote"
                 collibra:candidateGroup="${term.owner.working_group}"
                 collibra:votingRule="majority"/>

  <bpmn:serviceTask id="Publish" name="Publish term"
                    collibra:type="publishTerm"/>

  <bpmn:serviceTask id="Propagate" name="Propagate to represents"
                    collibra:type="propagateDefinition"/>

  <bpmn:serviceTask id="Audit" name="Write audit"/>
  <bpmn:endEvent id="Done"/>
</bpmn:process>

Step-by-step explanation.

1. Each term has three ownership roles — a Business Owner (the CFO office for finance terms), a Steward (the platform-finance engineering team), and a Working Group (the finance glossary working group that votes on definition changes). This tri-owner model is the compliance signature; it prevents any single person from unilaterally changing an authoritative definition.
2. The represents field is the propagation path. When "Net Revenue" is approved, the same definition surfaces on every linked physical asset — fct_revenue_daily.net_revenue, fct_revenue_monthly.net_revenue, and the Looker measure. An analyst looking at the physical asset sees the authoritative business definition without leaving the catalog.
3. The workflow for a term change is BPMN-modelled: proposed edit → steward review → working-group vote → publish → propagate → audit. The vote step is what an unregulated team never needs and a bank always does; a single steward cannot change "Net Revenue" without a documented working-group majority.
4. The rollout for the 400-term glossary is phased: 50 terms in month 1 (per domain), 100 in months 2–3, 200 more in months 4–6. Owner assignment happens first (every term must have a Business Owner + Steward + Working Group before it enters the workflow). Definitions come next; propagation last.
5. The regulatory refs are the audit-defence field. When the auditor asks "which policy authorises this definition of Net Revenue?", the answer is FIN-2025-014, ASC 606. Without this field, every definition is an unsupported assertion.

Output.

Rollout phase	Terms in glossary	Owners assigned	Approved	Propagated
Month 1	50	50	40	30
Month 3	150	150	130	110
Month 6	400	400	380	350

Rule of thumb. In regulated industries, the business glossary is not a nice-to-have on top of a catalog — it is the reason the catalog exists. The tri-owner model + BPMN change workflow is the operating rhythm. Ship the operating model before the terms; the terms are worthless without the ownership chain.

Senior interview question on Collibra in a regulated enterprise

A senior interviewer might ask: "You're the platform architect at a mid-size bank. Legal + Compliance are mandating a policy engine + DSAR workflow + business-glossary system of record within 12 months. Walk me through the Collibra rollout, the operating model, and the failure modes you'd guard against."

Solution Using a phased Collibra rollout with the tri-owner operating model

# 12-month Collibra rollout — declarative plan for a regulated bank
rollout:
  team_size: 200 seats (bank-wide governance function)
  domains: [finance, risk, compliance, customer, product]
  budget: enterprise (six-figure annual; assume $250k-$500k range)

phase_0_operating_model:
  months: 0-1
  activity:
    - stand up Data Governance Council (co-chairs: CDO, CCO)
    - define Business Owner / Steward / Working Group triad per domain
    - publish RACI matrix
  gate: council chartered; RACI signed off by legal

phase_1_glossary:
  months: 1-3
  activity:
    - seed 50 top-of-mind business terms per domain (250 total)
    - assign owners on every term
    - author BPMN term-change workflow
  gate: 250 terms approved; workflow live

phase_2_data_asset_link:
  months: 3-6
  activity:
    - crawl warehouse (Snowflake, Teradata legacy, mainframe extracts)
    - link every glossary term to represents Data Assets
    - assign steward on every Data Asset
  gate: 90% of production tables have owner + linked term

phase_3_policy_engine:
  months: 6-9
  activity:
    - author PII classification workflow (BPMN)
    - author DSAR deletion workflow (BPMN)
    - author retention policy workflow (BPMN)
  gate: 3 workflows in production; audit logs verified end-to-end

phase_4_active_metadata:
  months: 9-12
  activity:
    - wire Snowflake masking policy write path from workflow
    - wire warehouse retention DDL from workflow
    - wire Slack + email notify from workflow
  gate: 100+ active-metadata writes per month

failure_modes:
  - "Skip Phase 0" — no council, no operating model, workflow becomes a wiki
  - "Underspec workflow SLA" — no timer events, workflows stall forever
  - "Steward pool too shallow" — 200 tables per steward is a burnout recipe
  - "No audit-log query practice" — auditor arrives, audit log never exercised

Step-by-step trace.

Phase	Months	Gate	Risk if skipped
Operating model	0–1	Council + RACI	Workflow ownership vacuum
Glossary	1–3	250 terms	No system of record
Data asset link	3–6	90% linked	Discovery still ad-hoc
Policy engine	6–9	3 workflows live	No audit-defensible chain
Active metadata	9–12	100 writes/mo	Catalog is a wiki, not control plane

After 12 months the bank has a working data-governance council, a 400-term glossary with tri-owner backing, warehouse coverage with linked terms + stewards, three BPMN workflows in production, and active-metadata writes into the warehouse. The interview signal is that the candidate treats the Collibra rollout as an operating-model rollout with a catalog attached, not the reverse.

Output:

Deliverable	Month 3	Month 6	Month 9	Month 12
Terms approved	250	400	400	400
Stewards assigned	5	20	25	30
Data Assets linked	—	90%	95%	98%
BPMN workflows live	1	1	3	3
Monthly active-metadata writes	0	0	20	100+
DSARs processed via catalog	0	0	5	25

Why this works — concept by concept:

• Operating model first — Collibra without the tri-owner operating model is a wiki. The council + RACI + working groups are the reason the workflows have meaning. Skipping this phase is the single most common failure mode.
• Tri-owner triad — Business Owner + Steward + Working Group ensures every change goes through a business-technical-approver chain. This is the compliance signature; regulators recognise it as best practice.
• BPMN workflow engine — the audit-loggable, timer-enforced workflow is the axis Collibra wins on. Atlan's Slack + Jira model doesn't produce a defensible audit chain; Collibra's BPMN does.
• Active metadata as the last phase — writing masking policies is the pay-off; it comes after the operating model, the glossary, the data-asset link, and the workflow are stable. Reversing this order ships policies with no defensible provenance.
• Cost — enterprise pricing, 12-month rollout, dedicated governance function. The pay-off is audit defensibility — a regulator can query the catalog and receive a documented chain from policy to column. That is the pay-off, and the reason regulated industries pay for Collibra.

ETL
Topic — etl
ETL problems on governance, PII, and DSAR patterns

Practice →

SQL Topic — sql SQL policy, glossary, and stewardship problems

Practice →

---

4. Alation + DataHub — the veteran and the OSS challenger

Alation's bet — query-log-driven catalog, mature enterprise deployments, a stewardship model that predates the modern catalog wave

The mental model in one line: Alation is the catalog that decided in 2012 that the warehouse's own query log is the best signal for what data actually matters — who queries what, how often, in which shapes — and built the catalog UX on top of that popularity signal, then spent a decade hardening the stewardship model in enterprise finance and healthcare shops. Where Atlan is search-first modern and Collibra is workflow-first regulated, Alation is query-log-first. The catalog's front page is not "here are all your tables" — it's "here are the top 20 tables in your warehouse this week, sorted by query count, annotated by top queriers."

Alation's four axes.

• Discovery UX. Query-log-seeded popularity + keyword search + curated navigation. The "Top Queried" tab is the killer feature; every new user finds the important tables in their first session because the catalog surfaces them by usage, not alphabetically.
• Lineage depth. Column-level lineage across warehouse; BI-side lineage is workable but often via a paid connector. Historical strength is table-level lineage across warehouse + reporting.
• Governance workflow. Mature stewardship model — a "Steward" role, an "Article" system for authoring context, and workflow templates for common approvals. Not BPMN-deep like Collibra; not Slack-light like Atlan.
• Extensibility — closed SaaS with plugin surface. Java-based plugin SDK; several vendor-supported ingestion connectors. Extension threshold is higher than Atlan; lower than Collibra.

DataHub's bet — OSS-first, pluggable ingestion, GraphQL API, and a metadata model that engineering-heavy teams can extend without vendor tickets.

The mental model in one line: DataHub is what LinkedIn open-sourced in 2019 when the metadata problem outgrew the internal WhereHows catalog, and Acryl (the commercial company behind DataHub) has spent the years since polishing the OSS core into a production-grade catalog with a first-class Python SDK, a GraphQL API, and a metadata model that a platform team can extend by writing a new "aspect" in an afternoon. Where the closed-SaaS trio (Atlan, Collibra, Alation) trades extensibility for polish, DataHub trades polish for extensibility.

DataHub's four axes.

• Discovery UX. Solid — keyword, tag, term, owner search; column facets; deprecated filter. The UX is engineer-flavoured (dense information layout) more than analyst-flavoured (curated cards). Improves rapidly release over release.
• Lineage depth. Column-level via pluggable parsers — dbt manifest, sqllineage for raw SQL, LookML for Looker, custom parsers for anything else. The parser story is the differentiator; the OSS community ships new parsers regularly.
• Governance workflow. Bring your own. The OSS action framework (see the Section 1 worked example) lets you wire Slack + Jira + warehouse writes, but nothing is pre-built. Acryl Cloud adds hosted workflows.
• Extensibility — OSS with SDK. Python SDK for ingestion; GraphQL API for reads and mutations; custom metadata "aspects" via schema definition; DataHub Actions framework for reactive writes. This is the axis DataHub wins on absolutely.

Where each of Alation and DataHub wins.

• Alation wins. Mature enterprise governance shops with existing steward function; query-log-heavy warehouses (Teradata, Snowflake, BigQuery); teams valuing 10-year deployment references; RFP-driven procurement processes.
• DataHub wins. Engineering-heavy platform teams; OSS-first orgs; data-mesh implementations that need custom metadata aspects (data contract status, cost score, freshness SLA); teams comfortable running open-source infrastructure; roadmaps that include self-hosted metadata for compliance reasons.

The Alation + DataHub interview probes.

• "What's Alation's differentiator?" — required: query-log-driven popularity signal + mature stewardship.
• "When do you pick DataHub over the closed-SaaS trio?" — senior signal: OSS requirement, custom metadata model, engineering-heavy team, data-mesh with per-domain ownership.
• "How does DataHub's aspect model work?" — senior signal: entities have aspects (chunks of metadata); aspects are versioned; the Python SDK ships helpers per aspect; custom aspects are declared in PDL/protobuf.
• "How does Alation seed the catalog?" — required: query-log crawler + warehouse metadata crawler + manual authoring via Articles.

Worked example — Alation query-log ingestion for a Snowflake warehouse

Detailed explanation. Alation's query-log-driven catalog needs a periodic ingestion of the warehouse's query history. In Snowflake, this is the query_history view in the SNOWFLAKE shared database. Alation crawls it, extracts the queried objects, counts per-object query volumes, and surfaces the "Top Queried Tables This Week" panel that new users see on login.

• The signal. Query volume per table per week.
• The extract. SQL over snowflake.account_usage.query_history filtered to the last 7 days.
• The surface. Alation "Top Queried" tab; per-table "popular queriers" panel.

Question. Design the Alation Snowflake query-log ingestion. Show the SQL Alation runs, the object-extraction step, and the resulting catalog surface.

Input.

Component	Value
Warehouse	Snowflake
Query-log source	snowflake.account_usage.query_history
Lookback	7 days
Refresh cadence	Hourly

Code.

-- Alation-issued extract query — batch every hour
WITH parsed AS (
  SELECT
    q.query_id,
    q.database_name,
    q.schema_name,
    q.warehouse_name,
    q.user_name,
    q.query_type,
    q.start_time,
    q.total_elapsed_time,
    q.query_text,
    q.rows_produced
  FROM   snowflake.account_usage.query_history q
  WHERE  q.start_time >= DATEADD(day, -7, CURRENT_TIMESTAMP())
    AND  q.execution_status = 'SUCCESS'
    AND  q.query_type = 'SELECT'
)
SELECT
  query_id,
  database_name,
  schema_name,
  user_name,
  start_time,
  total_elapsed_time,
  query_text
FROM   parsed
ORDER  BY start_time DESC
LIMIT  1000000;

# Alation-side parser — extract queried tables per query
# (illustrative; Alation ships this internally)

import sqllineage
from collections import Counter

def extract_tables(query_text: str) -> list[str]:
    """Return list of fully-qualified tables referenced in the query."""
    try:
        parser = sqllineage.LineageRunner(query_text)
        sources = [str(t) for t in parser.source_tables()]
        return sources
    except Exception:
        return []

def build_popularity_index(rows) -> dict:
    counter = Counter()
    queriers = {}
    for row in rows:
        tables = extract_tables(row["query_text"])
        for t in tables:
            counter[t] += 1
            queriers.setdefault(t, Counter())[row["user_name"]] += 1
    return {
        t: {
            "query_count":   counter[t],
            "top_queriers":  queriers[t].most_common(5),
        }
        for t in counter
    }

# Result — the raw material for the "Top Queried" tab
# {
#   "prod.mart_finance.fct_revenue_daily": {
#     "query_count": 1247,
#     "top_queriers": [("alice", 320), ("bob", 190), ...]
#   },
#   ...
# }

Step-by-step explanation.

1. The extract query pulls the last 7 days of successful SELECT queries from Snowflake's account_usage.query_history. The 1M row limit is conservative; production Alation may stream through pagination.
2. The Python parser runs each query text through sqllineage (or Alation's proprietary parser) to extract the fully-qualified tables referenced. This is the moment where table-level popularity is materialised.
3. The counter aggregates per-table query counts; the queriers dict tracks per-user counts per table. Both feed the "Top Queried" panel and the per-table "who queries this?" affordance.
4. The Alation UI surfaces this as three panels: (a) a per-warehouse "Top 20 tables this week" ordered by query_count; (b) a per-table "top queriers" panel with user profiles; (c) a per-user "you might be interested in" recommendation based on tables the user's teammates query.
5. The competitive advantage is subtle: the analyst who joins the team on Monday morning does not need to ask which tables matter. The catalog surfaces them by usage. Discovery becomes a passive act, not an active search.

Output.

Panel	Signal	Refresh
Top Queried Tables	count(query_id) per table, 7d	Hourly
Top Queriers per Table	count(query_id) per (table, user), 7d	Hourly
Team Recommendations	tables queried by teammates	Daily
Query volume trend	count per day	Daily

Rule of thumb. The query-log signal is Alation's differentiator. If your warehouse tracks query history, Alation gives you an automatic usage-driven discovery layer that requires no human curation. In shops without a mature steward function, this alone justifies the cost.

Worked example — DataHub GraphQL query for column-level lineage

Detailed explanation. DataHub's GraphQL API is the SDK for engineers. When the platform team needs to answer "what's downstream of customers.email?" or "list every asset owned by the finance team with a PII tag" the API is the primary interface. Walk through a column-level downstream lineage query so an interviewer can see the API story.

• The API. DataHub GraphQL at https://datahub-host/api/graphql.
• The query. Downstream column-level lineage starting from a single column URN.
• The consumer. A CI job in the dbt repo that flags PRs which change a column with downstream dependencies.

Question. Write the GraphQL query for downstream lineage and the CI helper that consumes it.

Input.

Component	Value
Column URN	urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:snowflake,raw.customers,PROD),email)
Depth	5
Direction	downstream

Code.

# DataHub GraphQL — column downstream lineage
query DownstreamLineage($urn: String!, $depth: Int!) {
  searchAcrossLineage(
    input: {
      urn: $urn
      direction: DOWNSTREAM
      types: [SCHEMA_FIELD, DATASET, DASHBOARD]
      query: "*"
      start: 0
      count: 500
      lineageFlags: {
        entitiesExploredPerHopLimit: 200
        maxHops: 5
      }
    }
  ) {
    total
    searchResults {
      degree
      entity {
        urn
        type
        ... on SchemaFieldEntity {
          fieldPath
          parent {
            urn
            ... on Dataset {
              name
              platform { name }
              tags { tags { tag { name } } }
              ownership { owners { owner { urn } } }
            }
          }
        }
        ... on Dataset { name platform { name } }
        ... on Dashboard { title }
      }
    }
  }
}

# CI helper — flag PRs that change columns with downstream dependencies
import requests, sys, json, os

DATAHUB    = os.environ["DATAHUB_URL"]           # https://datahub.internal
TOKEN      = os.environ["DATAHUB_TOKEN"]
COLUMN_URN = sys.argv[1]                          # passed by dbt-changed-columns hook

QUERY = """query DownstreamLineage($urn: String!, $depth: Int!) { ... }"""  # elided

def downstream_count(urn: str) -> dict:
    resp = requests.post(
        f"{DATAHUB}/api/graphql",
        json={"query": QUERY, "variables": {"urn": urn, "depth": 5}},
        headers={"Authorization": f"Bearer {TOKEN}"},
        timeout=10,
    )
    resp.raise_for_status()
    data = resp.json()["data"]["searchAcrossLineage"]
    by_type = {"SCHEMA_FIELD": 0, "DATASET": 0, "DASHBOARD": 0}
    for r in data["searchResults"]:
        by_type[r["entity"]["type"]] = by_type.get(r["entity"]["type"], 0) + 1
    return by_type

impact = downstream_count(COLUMN_URN)
if impact["DASHBOARD"] > 0:
    print(f"WARN: change to {COLUMN_URN} impacts {impact['DASHBOARD']} dashboards")
    print("Request review from data-analytics team before merging.")
    sys.exit(1)
sys.exit(0)

Step-by-step explanation.

1. The searchAcrossLineage GraphQL query is DataHub's canonical downstream / upstream traversal. The direction: DOWNSTREAM flag walks the graph outward from the input URN; maxHops: 5 bounds the traversal.
2. The types filter restricts the returned nodes to columns (SCHEMA_FIELD), tables (DATASET), and dashboards (DASHBOARD). This is what "column-level lineage" means in practice — the query returns not just downstream columns but the tables and dashboards that transitively depend on them.
3. The Python CI helper wraps the GraphQL call. It's called by a dbt-changed-columns hook that fires on every PR touching a *.sql file. The hook extracts the changed columns and calls downstream_count for each.
4. The exit-code contract is the CI signal: exit 0 → PR merges cleanly; exit 1 → GitHub check fails with the impact summary. A change that impacts any dashboard forces a review from the data-analytics team before merge.
5. This is the engineer-flavoured catalog use case that DataHub was designed for. The catalog is not just a UI; it is an API that engineers script against. The equivalent CI check in Atlan or Collibra is doable but often via a vendor connector; in DataHub it's ten lines of Python + a GraphQL query.

Output.

Downstream type	Count	CI action
SCHEMA_FIELD	12	inform
DATASET	4	inform
DASHBOARD	2	require review
Total	18	comment on PR

Rule of thumb. DataHub's GraphQL + Python SDK is the "catalog as an engineering primitive" story. If the platform team wants to script the catalog into CI, into observability, into cost dashboards, DataHub is the low-friction choice. Closed-SaaS catalogs offer APIs, but the DataHub story is API-first from day one.

Worked example — DataHub custom aspect for data-contract status

Detailed explanation. Custom aspects are DataHub's extensibility story. When the platform team needs to track "does this table have a signed data contract?" as a first-class piece of metadata, they define a new aspect (chunk of metadata attached to an entity) in PDL/protobuf, register it with the metadata service, and populate it via ingestion or the Python SDK. This is the axis where DataHub decisively wins over any closed-SaaS catalog.

• The need. Track data-contract signed/expired/breached status per table.
• The DataHub answer. Custom aspect dataContractStatus on Dataset entities.
• The alternative in a closed catalog. File a vendor ticket, wait 6–12 weeks, receive an "attribute" that isn't quite what you needed.

Question. Design a dataContractStatus custom aspect. Show the schema, the ingestion helper, and a GraphQL read.

Input.

Component	Value
Aspect name	dataContractStatus
Fields	status (enum), signed_at, expires_at, breach_history
Entity	Dataset

Code.

// dataContractStatus.pdl — DataHub custom aspect schema
{
  "type": "record",
  "name": "DataContractStatus",
  "namespace": "com.linkedin.pipecode.metadata",
  "@Aspect": { "name": "dataContractStatus" },
  "fields": [
    {
      "name": "status",
      "type": {
        "type": "enum",
        "name": "DataContractStatusType",
        "symbols": ["SIGNED", "EXPIRED", "BREACHED", "PENDING"]
      }
    },
    { "name": "signedAt",  "type": ["null","long"], "default": null },
    { "name": "expiresAt", "type": ["null","long"], "default": null },
    {
      "name": "breachHistory",
      "type": { "type": "array", "items": {
        "type": "record",
        "name": "BreachEvent",
        "fields": [
          {"name": "at",         "type": "long"},
          {"name": "reason",     "type": "string"},
          {"name": "reporter",   "type": "string"}
        ]
      }}
    }
  ]
}

# Populate the aspect via the DataHub Python emitter
from datahub.emitter.rest_emitter import DatahubRestEmitter
from datahub.emitter.mcp import MetadataChangeProposalWrapper
from datahub.metadata.schema_classes import DataContractStatusClass
from time import time

emitter = DatahubRestEmitter(gms_server="https://datahub.internal:8080")

emitter.emit_mcp(MetadataChangeProposalWrapper(
    entityUrn="urn:li:dataset:(urn:li:dataPlatform:snowflake,mart_finance.fct_revenue_daily,PROD)",
    aspect=DataContractStatusClass(
        status="SIGNED",
        signedAt=int(time() * 1000),
        expiresAt=int((time() + 365 * 86400) * 1000),
        breachHistory=[],
    ),
))

# GraphQL read — surface contract status in the UI + CI checks
query ContractStatus($urn: String!) {
  dataset(urn: $urn) {
    dataContractStatus {
      status
      signedAt
      expiresAt
      breachHistory { at reason reporter }
    }
  }
}

Step-by-step explanation.

1. The .pdl file declares the aspect schema — a record type with an enum status, two nullable timestamps, and an array of breach events. The @Aspect annotation tells DataHub this is an aspect (attachable to an entity) rather than a standalone type.
2. The metadata service reads the .pdl, generates the Java/Python bindings, and registers the aspect in the GraphQL schema. This is the moment "custom aspect" becomes a first-class piece of the catalog — every entity type that supports the aspect exposes it in the API.
3. The Python emitter writes an aspect value for a specific dataset. The MetadataChangeProposalWrapper is the DataHub-native change event; the emitter serialises it and POSTs to the metadata service. From this moment the aspect is queryable.
4. The GraphQL read returns the aspect on a dataset query. A CI job that wants to gate deployments on contract status reads this endpoint before allowing a schema-change PR to merge; the check is one line of GraphQL.
5. The full loop — schema, register, write, read — takes an afternoon of engineering time on DataHub. In a closed-SaaS catalog, the same feature requires either a vendor engagement (6–12 weeks) or a hack (misuse an existing "custom attribute" field). This is the DataHub extensibility bet.

Output.

Step	Time	Artifact
Author .pdl	30 min	dataContractStatus.pdl
Register with metadata service	15 min	schema regenerated
Write ingestion	1 hr	Python emitter integration
Add UI surface	2 hr	React aspect renderer
Wire CI check	30 min	GraphQL query in Actions
Total	half-day	production feature

Rule of thumb. If the platform team's roadmap includes any custom metadata — data contracts, freshness SLAs, cost scores, PII risk levels — DataHub's custom aspect story is the axis that pays. Every ticket-to-the-vendor you avoid is a week of roadmap regained.

Senior interview question on Alation + DataHub trade-offs

A senior interviewer might ask: "You have a 60-engineer platform team with a mature Snowflake warehouse and a data-mesh roadmap. Walk me through whether you'd start with Alation (mature enterprise catalog) or DataHub (OSS pluggable). Score the fit and defend the pick."

Solution Using DataHub for the data-mesh roadmap with a phased Alation exit criterion

# Decision framework — Alation vs DataHub for a 60-engineer team on Snowflake + data-mesh
scoring:
  weights:
    discovery_ux:      0.20
    lineage_depth:     0.25
    governance_flow:   0.15
    extensibility:     0.25   # elevated because of data-mesh roadmap
    cost:              0.15

  alation:
    discovery_ux:    3   # query-log signal is the winner here
    lineage_depth:   2   # column-level workable; BI a paid connector
    governance_flow: 3   # mature stewardship
    extensibility:   1   # plugin SDK but closed
    cost:            2   # enterprise pricing; mid-market feasible
    total:           0.20*3 + 0.25*2 + 0.15*3 + 0.25*1 + 0.15*2 = 2.10

  datahub:
    discovery_ux:    2   # engineer-flavoured UI
    lineage_depth:   3   # column-level + pluggable parsers
    governance_flow: 2   # bring your own via actions framework
    extensibility:   3   # custom aspects; GraphQL; Python SDK
    cost:            3   # OSS core; Acryl Cloud tiered
    total:           0.20*2 + 0.25*3 + 0.15*2 + 0.25*3 + 0.15*3 = 2.65

decision: datahub
tie_breakers:
  - data-mesh roadmap requires per-domain custom aspects
  - engineering team comfortable with self-hosted metadata service
  - GraphQL + Python SDK map to CI + observability tooling

alation_exit_criteria:
  - if regulatory audit mandates BPMN workflow engine → escalate to Collibra
  - if steward function grows > 15 dedicated stewards → re-evaluate Alation
  - if query-log-driven discovery becomes the primary UX ask → re-evaluate

Step-by-step trace.

Question	DataHub answer	Alation answer	Winner
Q1 — Discovery UX for engineers	GraphQL + dense UI	Query-log surface	Alation on UX, DataHub on API
Q2 — Column-level lineage across dbt + BI	Pluggable parsers	Column-level warehouse; BI paid	DataHub
Q3 — Custom metadata for mesh (contracts, SLAs)	Custom aspects in an afternoon	Plugin SDK + longer cycles	DataHub
Q4 — Cost	OSS core + Acryl Cloud tiered	Enterprise seat + module	DataHub
Q5 — Enterprise references / RFP	Growing	Mature	Alation

Four of five push to DataHub; one (references) to Alation. For a mid-size engineering-heavy platform team with a mesh roadmap, DataHub is the defensible pick. The Alation-exit criteria are pre-declared so the platform lead can re-evaluate cleanly if the org profile changes.

Output:

Tool	When it wins
Alation	Mature governance shops; query-log-driven UX priority; RFP-heavy enterprise procurement
DataHub	Engineering-heavy platform; data-mesh; custom metadata roadmap; OSS preference
Neither	Deep BPMN workflow + regulatory audit → Collibra; low-friction mid-market → Atlan

Why this works — concept by concept:

• Weighting the extensibility axis higher — for a data-mesh roadmap, custom aspects are not a nice-to-have; they are the reason mesh implementations succeed. Elevating the weight to 0.25 encodes this priority; leaving it at 0.15 would produce a different (and arguably wrong-for-mesh) answer.
• Pluggable parsers as the lineage differentiator — DataHub's community-contributed parsers cover more source systems than any closed vendor's engineering pipeline can service. This is the OSS-scale effect in action.
• CI + observability integration — the DataHub GraphQL API + Python SDK make the catalog a natural first-class citizen in the engineering stack. Alation's plugin SDK is workable but higher-friction; the "an afternoon of custom aspect work" story is the DataHub-only offering.
• Pre-declared exit criteria — mature platform leads pre-declare the conditions that flip the recommendation. This is a senior interview signal: "we picked DataHub and here are the four conditions under which we'd re-evaluate." Locking-in without an exit criterion is a junior move.
• Cost — OSS core removes the seat-license drag from the ROI model. Acryl Cloud is the paid tier for teams that want hosted operations; the migration between OSS and Cloud is minimal. The cost story lets the team ramp cheaply and pay when the ops burden becomes real.

SQL
Topic — sql
SQL query-log and popularity analysis problems

Practice →

Optimization Topic — optimization Optimization problems on GraphQL lineage and custom aspects

Practice →

---

5. Picking a catalog + rollout patterns

The five patterns every senior data engineer ships — decision matrix, 90-day plan, catalog-rot mitigation, governance-over-usage guard, and the "no owner" cleanup

The mental model in one line: a healthy catalog in production is the cumulative effect of a defensible decision matrix, a phased 90-day rollout, an active anti-catalog-rot regimen, an intentional balance between governance and usage, and a systematic sweep of "no owner" columns — none of these is exotic, and missing any one is the difference between a catalog that outlives your tenure and a catalog that quietly becomes irrelevant six months after the launch email. The catalog is not a product; it is a practice.

Pattern 1 — the decision matrix in full.

• Axes. UX × lineage × governance × cost × OSS. Five columns.
• Rows. Every candidate vendor (typically Atlan + Collibra + Alation + DataHub).
• Weights. Team-profile driven. Mid-market Snowflake shops emphasise UX + lineage; regulated banks emphasise governance + audit-defence; engineering-heavy mesh teams emphasise extensibility.
• Output. A ranked list with tied results broken by a pre-declared tie-break criterion (extensibility roadmap, ecosystem fit, procurement constraints).
• The senior signal. The matrix is shared before the vendor conversation. Vendors see their own scores; the platform lead defends the weights.

Pattern 2 — the 90-day rollout plan.

• Day 0. Crawl the top 3 sources (warehouse + dbt + a BI tool). This is the fastest path to first-value.
• Day 30. Publish the top-100 assets with owners + business-glossary terms. The catalog now answers "who owns this?" for the highest-value assets.
• Day 60. Open the first Slack-native workflow (ownership request, PII tag propagation, or dashboard-broken alert). The catalog now acts on the world.
• Day 90. Onboard the first non-platform team by inviting them to author metadata for their own domain. Adoption starts spreading.
• The failure mode. Skipping any milestone. Rollouts that skip the Slack workflow (day 60) plateau at "read-only wiki" and never justify their cost.

Pattern 3 — catalog rot and how to prevent it.

• The symptom. Descriptions stale; owners have left the company; certifications expired; lineage links broken.
• The root cause. Metadata is authored once at rollout and never re-verified.
• The pattern. Ship a quarterly "catalog freshness" review — every asset must have last_reviewed_at < 90 days ago. Assets that fail get flagged with a stale tag and routed to the current owner (or, if the owner has left, to the team's manager).
• The automation. A DataHub Actions job (or Atlan workflow, or Collibra BPMN) that scans for stale assets weekly and files Jira tickets against owning teams.

Pattern 4 — the governance-over-usage tilt.

• The symptom. Every column has a tag, an owner, a certification, a business term, a policy reference — but nobody uses the catalog to find anything, because search returns 40 hits for every query.
• The root cause. The rollout emphasised governance completeness before usage feedback.
• The pattern. Measure catalog usage alongside governance coverage. Every quarterly review reports both: "80% of assets have owners" (governance) and "60% of new-hire questions resolved via catalog search" (usage). Only shipping both metrics forces the team to keep the two in balance.
• The re-balance move. If governance leads usage, invest in adoption (training, /catalog Slack command, onboarding checklist). If usage leads governance, invest in stewardship (steward hires, glossary sprints, ownership sweeps).

Pattern 5 — the "no owner" cleanup.

• The symptom. 30% of tables have no owner. On-call cannot escalate. Users cannot ask questions. Catalog value drops.
• The pattern. Ship a weekly "no owner" digest: every table without an owner is listed, grouped by database, and posted to #data-platform. Each table gets a needs owner tag. Every week the digest shrinks; adopting teams claim tables.
• The automation. A GraphQL query that returns unowned tables + a Slack scheduled message.
• The escalation. If a table stays unowned for 90 days, mark it deprecated and open a ticket to archive.

Common interview probes on rollout patterns.

• "How do you keep a catalog fresh?" — required: quarterly review + staleness tag + automated Jira tickets.
• "How do you balance governance and usage?" — senior signal: measure both; re-balance quarterly.
• "How do you handle the last-mile adoption problem?" — required: onboarding checklist + Slack command + power-user program.
• "What kills a catalog rollout?" — senior signal: skipping the workflow phase (day 60); over-emphasising governance completeness before adoption; letting stale metadata rot.

Worked example — applying the decision matrix end-to-end for three team profiles

Detailed explanation. The same decision matrix produces different winners depending on the team profile. Walk through three profiles side by side so the interviewer can see how the weights determine the outcome. This is the highest-signal move in a catalog interview — showing that the framework is stable but the answer is team-conditional.

• Profile A. Mid-market Snowflake shop, 40 engineers, dbt-heavy.
• Profile B. Regulated bank, 500 engineers, compliance-driven.
• Profile C. Engineering-heavy mesh, 80 engineers, contracts on the roadmap.

Question. Score all four catalogs against each profile. Show the weights and defend the winner.

Input.

Profile	UX weight	Lineage weight	Governance weight	Extensibility weight	Cost weight
A — mid-market	0.30	0.25	0.15	0.15	0.15
B — regulated bank	0.15	0.20	0.40	0.10	0.15
C — mesh engineering	0.15	0.25	0.15	0.30	0.15

Code.

# Three-profile scorer
PROFILES = {
    "A_mid_market":     {"ux":0.30,"lineage":0.25,"gov":0.15,"ext":0.15,"cost":0.15},
    "B_regulated_bank": {"ux":0.15,"lineage":0.20,"gov":0.40,"ext":0.10,"cost":0.15},
    "C_mesh":           {"ux":0.15,"lineage":0.25,"gov":0.15,"ext":0.30,"cost":0.15},
}

# Base scores per vendor per axis — 0..3
SCORES = {
    "atlan":    {"ux":3,"lineage":3,"gov":2,"ext":1,"cost":2},
    "collibra": {"ux":2,"lineage":2,"gov":3,"ext":1,"cost":1},
    "alation":  {"ux":2,"lineage":2,"gov":3,"ext":1,"cost":2},
    "datahub":  {"ux":2,"lineage":3,"gov":2,"ext":3,"cost":3},
}

def rank(profile_weights):
    totals = {}
    for v, axes in SCORES.items():
        totals[v] = sum(profile_weights[a] * axes[a] for a in profile_weights)
    return sorted(totals.items(), key=lambda kv: -kv[1])

for name, weights in PROFILES.items():
    print(name, rank(weights))

# A_mid_market:     [('atlan', 2.30), ('datahub', 2.60), ...] → DataHub wins on cost + lineage
# B_regulated_bank: [('collibra', 2.35), ('alation', 2.20), ...] → Collibra wins on governance
# C_mesh:           [('datahub', 2.70), ('atlan', 2.15), ...]   → DataHub wins on extensibility

# Interview-ready output — one paragraph per profile
verdicts:
  A_mid_market:
    winner: datahub (or atlan, ~tie depending on ecosystem fit)
    rationale: >
      Cost + lineage carry the day; ecosystem fit tie-breaker.
      Atlan wins if 90% surface is Snowflake+dbt+Looker; DataHub wins if
      the team plans custom metadata (freshness SLA, cost score).

  B_regulated_bank:
    winner: collibra
    rationale: >
      Governance weight (0.40) dominates. BPMN workflow, tri-owner
      operating model, and audit-loggable decision chains are Collibra's
      home turf. Alation is second; Atlan and DataHub are non-starters
      for BPMN-grade audit compliance.

  C_mesh:
    winner: datahub
    rationale: >
      Extensibility weight (0.30) is dispositive. Custom aspects for
      data contracts, freshness SLAs, cost scores — DataHub ships them
      in an afternoon; closed-SaaS vendors ship them in a quarter (if at all).

Step-by-step explanation.

1. The Python scorer takes three profiles and four vendors, computes weighted totals, and ranks. Notice that the same base scores produce different winners across profiles — the weights are the interview-defensible part.
2. Profile A (mid-market Snowflake shop) is the closest race. Atlan and DataHub finish nearly tied; the tie-breaker is ecosystem fit (Atlan wins if the stack is Snowflake+dbt+Looker; DataHub wins if the roadmap includes custom metadata).
3. Profile B (regulated bank) is a walk for Collibra. Governance at 0.40 weight dominates every other axis. The senior signal is naming why Collibra wins (BPMN + tri-owner + audit log) rather than just "Collibra."
4. Profile C (mesh engineering team) is a walk for DataHub. Extensibility at 0.30 weight carries the decision. Custom aspects for data contracts, per-domain metadata models, and CI integration are the exact features mesh implementations need.
5. The interview-ready verdicts encode both the winner and the reason. Walking an interviewer through three profiles in sequence demonstrates that the candidate treats catalog selection as a bounded framework, not a "gut feel" vendor pick.

Output.

Profile	Winner	Runner-up	Delta	Tie-break axis
A — mid-market	DataHub (or Atlan)	Alation	0.15	Ecosystem fit
B — regulated bank	Collibra	Alation	0.15	(none — clear)
C — mesh engineering	DataHub	Atlan	0.55	(none — clear)

Rule of thumb. Ship the matrix before the vendor conversation. Ship it with the team profile explicitly stated. The rubric is stable; the answer is profile-conditional. Vendors who complain about the weights are the vendors you should not buy.

Worked example — the 90-day rollout in fine-grained detail

Detailed explanation. The 90-day plan is the operational spine of a successful catalog rollout. Walk through each phase with concrete outputs, gate criteria, and failure modes so an interviewer can see the sequencing discipline.

• Day 0–14. Crawl the top 3 sources (warehouse + dbt + BI tool).
• Day 14–30. Publish glossary + ownership on the top 100 assets.
• Day 30–60. Open the first Slack-native workflow.
• Day 60–90. Onboard the first non-platform team.

Question. Produce the 90-day plan for the mid-market Snowflake shop from the previous example. Show the outputs, gates, and failure modes at each milestone.

Input.

Milestone	Day	Owner	Output	Gate
Crawl top 3 sources	14	Platform lead	Snowflake + dbt + Looker indexed	95% coverage
Publish glossary + ownership	30	Steward	100 assets certified	Median asset owned
Open first workflow	60	Platform lead	1 active-metadata workflow	10+ writes / week
Onboard first team	90	Team lead	Finance team authoring metadata	Team-lead sign-off

Code.

# 90-day rollout — declarative plan with gates and failure modes
plan:
  team: mid-market Snowflake shop
  stack: [snowflake, dbt, looker, slack, jira]
  budget: seat-based (~$36k first-year)

phase_1_crawl:
  window: day 0 to day 14
  activities:
    - stand up catalog SaaS tenant
    - install Snowflake connector; crawl raw + mart schemas
    - install dbt connector; ingest manifest.json + catalog.json
    - install Looker connector; ingest models + explores + dashboards
  gate:
    - 95% of production warehouse tables indexed
    - 95% of dbt models indexed
    - 95% of top-100 dashboards indexed
  failure_modes:
    - "Connector auth misconfigured" → falls back to manual export; slower
    - "Slow crawl" → tune concurrency; usually a same-day fix
    - "PII in metadata" → apply auto-classification before external users see catalog

phase_2_ownership:
  window: day 14 to day 30
  activities:
    - assign owner groups per source system
    - author 100 business terms across finance, growth, product, ops
    - link terms to top 100 assets
    - certify top 20 marts as VERIFIED
  gate:
    - 100% of top-100 assets have an owner + a business term
    - 20 assets certified VERIFIED
  failure_modes:
    - "Ownership vacuum on legacy tables" → escalate to source-system managers
    - "Term definitions contested" → open working-group review
    - "Certification bar too high" → relax to 20-asset target for first quarter

phase_3_workflow:
  window: day 30 to day 60
  activities:
    - pick the first workflow (PII propagation is the most-common)
    - author workflow YAML / BPMN / actions config
    - integrate Snowflake write path (masking policy or DDL)
    - integrate Slack notification
    - smoke test end-to-end with a test asset
  gate:
    - 1 workflow live in production
    - 10+ workflow-triggered writes in the past week
  failure_modes:
    - "Warehouse write auth denied" → fix role grants; usually a same-day fix
    - "Slack channel not found" → verify channel naming convention
    - "HMAC signature mismatch" → verify webhook secret

phase_4_adoption:
  window: day 60 to day 90
  activities:
    - onboard finance team as first non-platform team
    - deliver "author your own domain metadata" training
    - deploy Slack `/catalog search` command
    - ship "no owner" digest + weekly steward stand-up
  gate:
    - finance team has 3+ authored terms
    - Slack command has 20+ uses in the past week
    - 60% of new-hire questions self-resolve via catalog
  failure_modes:
    - "Team refuses to author" → escalate to the team's VP; usually a training gap
    - "Search bad UX" → iterate on operator cheat sheet
    - "Slack command low usage" → add to onboarding checklist

Step-by-step explanation.

1. Each phase has a window, activities, a gate, and a failure modes list. Missing any of these four turns the plan from a rollout into a Gantt chart. The gate is the pass/fail; the failure modes are the pre-declared risks.
2. Phase 1 (crawl) is the fastest wall-clock phase — 14 days is a conservative window; most modern catalogs can crawl a mid-market warehouse in a day. The gate (95% coverage) is intentional; 100% is unattainable because legacy tables always resist.
3. Phase 2 (ownership) is the social phase — assigning owners and authoring terms requires the platform lead to negotiate with source-system managers. The gate (100% of top-100 owned) is achievable; the trap is spending all quarter on the "top-500 owned" long tail.
4. Phase 3 (workflow) is the phase most rollouts skip. The catalog is now populated; the temptation is to declare victory. The senior discipline is shipping one workflow (PII → masking policy is the most-common) as proof the catalog can act, not just describe.
5. Phase 4 (adoption) is the phase where the rollout compounds. Onboarding the first non-platform team is the flywheel — that team's contributions become social proof for the next team, and the rollout spreads by internal word-of-mouth rather than by top-down mandate.

Output.

Milestone	Day	Deliverable	Gate met?	Failure mode
Crawl top 3 sources	14	2000 assets indexed	95% coverage	(mitigated)
Publish glossary + ownership	30	100 assets owned + certified	100% top-100	(mitigated)
Open first workflow	60	PII → masking policy live	20+ writes	(mitigated)
Onboard first team	90	Finance authoring metadata	3+ terms	(mitigated)

Rule of thumb. Every rollout is the same four phases; only the specific vendor differs. The pattern is transferable. Ship the plan on day zero; review at each phase gate; measure adoption metrics from day one.

Worked example — quarterly catalog freshness sweep

Detailed explanation. The single most-common catalog failure mode is rot — descriptions go stale, owners leave, certifications expire, and the catalog quietly loses trust. The quarterly freshness sweep is the discipline that prevents rot. Walk through the automation and the operational rhythm.

• The signal. last_reviewed_at on every asset.
• The check. Any asset with last_reviewed_at > 90 days ago is stale.
• The action. Automated Jira ticket to the owner; if the owner has left, escalate to the owning team's manager.

Question. Design the freshness sweep. Show the GraphQL query, the Jira integration, and the operational rhythm.

Input.

Component	Value
Freshness threshold	90 days
Sweep cadence	Weekly (rolling sweep; quarterly summary)
Escalation cadence	If unresolved for 30 days after ticket
Report audience	Platform lead + steward leads

Code.

# DataHub GraphQL — find stale assets
query StaleAssets($threshold_ms: Long!) {
  searchAcrossEntities(
    input: {
      types: [DATASET]
      query: "*"
      filters: [
        { field: "lastReviewedAt", condition: LESS_THAN, values: [$threshold_ms] }
      ]
      count: 500
    }
  ) {
    total
    searchResults {
      entity {
        urn
        ... on Dataset {
          name
          lastReviewedAt
          ownership { owners { owner { urn } } }
        }
      }
    }
  }
}

# Weekly freshness sweep — GraphQL + Jira + Slack digest
import requests, os, datetime as dt
from jira import JIRA

DATAHUB   = os.environ["DATAHUB_URL"]
DATAHUB_T = os.environ["DATAHUB_TOKEN"]
JIRA_CFG  = { "server": os.environ["JIRA_URL"] }
JIRA_AUTH = (os.environ["JIRA_USER"], os.environ["JIRA_TOKEN"])

threshold_ms = int((dt.datetime.utcnow() - dt.timedelta(days=90)).timestamp() * 1000)

resp = requests.post(
    f"{DATAHUB}/api/graphql",
    json={"query": STALE_QUERY, "variables": {"threshold_ms": threshold_ms}},
    headers={"Authorization": f"Bearer {DATAHUB_T}"},
    timeout=30,
).json()

stale = resp["data"]["searchAcrossEntities"]["searchResults"]

jira = JIRA(options=JIRA_CFG, basic_auth=JIRA_AUTH)
digest_lines = []
for hit in stale:
    ent = hit["entity"]
    owner_urn = ent["ownership"]["owners"][0]["owner"]["urn"] if ent["ownership"]["owners"] else None
    if owner_urn:
        assignee = owner_urn.split(":")[-1]
        jira.create_issue(
            project="PLAT",
            issuetype={"name": "Task"},
            summary=f"Refresh catalog metadata for {ent['name']}",
            description=f"Asset {ent['urn']} has not been reviewed in > 90 days.",
            assignee={"name": assignee},
        )
    digest_lines.append(f"- {ent['name']} — owner {owner_urn or 'NONE'}")

# Slack digest — one message per week to #data-platform
slack_message = f"Weekly stale-asset digest ({len(stale)} assets):\n" + "\n".join(digest_lines[:25])
requests.post(os.environ["SLACK_WEBHOOK"], json={"text": slack_message})

Step-by-step explanation.

1. The GraphQL query returns every dataset whose lastReviewedAt is older than 90 days. The threshold_ms is computed each run — this is a rolling window, not a fixed date.
2. The Python job iterates the results, files a Jira ticket per stale asset, and assigns it to the current owner. If the owner has left (URN not resolvable), the ticket is filed unassigned and surfaces in the Slack digest for manual triage.
3. The Slack digest is the visibility layer. Once a week, the platform lead sees a single message summarising the freshness state — how many assets went stale this week, who owns them, and how many are unassigned.
4. The escalation loop: any Jira ticket unresolved after 30 days is auto-escalated to the assignee's manager. The escalation prevents "stale ticket rot" from compounding "stale asset rot."
5. The quarterly summary rolls up the weekly digests into a governance-council report: median staleness, worst-offender teams, top-N staleness reasons ("owner left the company" is nearly always #1).

Output.

Week	Stale count	Owner unassigned	Tickets filed	Median resolution time
Week 1	120	30	90	12 days
Week 8	80	15	65	8 days
Week 20	40	5	35	5 days
Week 52	20	2	18	3 days

Rule of thumb. Every catalog needs a freshness sweep from day one. Skipping this ships the catalog into a slow-motion rot. The sweep is 50 lines of Python; not having it is the cause of every "our catalog is a graveyard" incident.

Senior interview question on picking and rolling out a catalog

A senior interviewer might ask: "You're the platform lead at a 60-engineer scale-up on Snowflake + dbt + Looker with a data-mesh roadmap. Walk me through the entire catalog decision — matrix, winner, 90-day plan, and the year-one operational rhythm."

Solution Using the full five-pattern playbook for a mesh-oriented scale-up

# End-to-end catalog decision + rollout — 60-engineer mesh-oriented scale-up

matrix:
  team_profile: mesh_engineering
  weights:
    ux:       0.15
    lineage:  0.25
    gov:      0.15
    ext:      0.30
    cost:     0.15
  ranking:
    - datahub: 2.70    # winner
    - atlan:   2.15
    - alation: 2.05
    - collibra: 1.90
  tie_break: "extensibility drives the mesh roadmap; DataHub is unambiguous"

rollout_90d:
  phase_1_crawl:
    output: snowflake + dbt + looker indexed
    gate: 95% coverage
  phase_2_ownership:
    output: 100 top assets owned + termed
    gate: median owned
  phase_3_workflow:
    output: pii → masking policy live
    gate: 20+ writes/week
  phase_4_adoption:
    output: finance team authoring
    gate: 3 terms + 20 slack uses

year_one_ops:
  quarterly_freshness_sweep:
    threshold: 90d
    output: jira tickets + slack digest
  monthly_governance_vs_usage_scorecard:
    metrics: [asset_owned_pct, slash_catalog_uses, self_serve_resolve_pct]
    action: rebalance investment if drift > 20%
  weekly_no_owner_digest:
    output: unowned tables → #data-platform
  mesh_domain_authoring:
    output: each new domain onboarded with steward + terms + workflow
  custom_aspect_roadmap:
    quarter_1: dataContractStatus
    quarter_2: freshnessSLA
    quarter_3: costScore
    quarter_4: piiRiskLevel

Step-by-step trace.

Step	Action	Outcome
Matrix	Weighted vote across five axes	DataHub wins by 0.55
Rollout Phase 1	Crawl 3 sources	2000 assets indexed
Rollout Phase 2	Own + term top 100	Median asset owned
Rollout Phase 3	First workflow (PII)	Active-metadata proven
Rollout Phase 4	Onboard first team	Adoption flywheel starts
Year 1 — freshness	Quarterly sweep	Rot prevented
Year 1 — balance	Monthly scorecard	Gov / usage tracked together
Year 1 — no-owner	Weekly digest	Ownership gaps close
Year 1 — mesh	Per-domain rollout	Mesh flywheel expands
Year 1 — aspects	Custom aspect per quarter	Extensibility roadmap ships

After year one the platform team has a defensible vendor decision, a completed rollout, four operational disciplines running on cadence, and four custom aspects shipped. The catalog has moved from "we bought this thing" to "this is how we operate." The interview signal is treating the catalog as a practice with cadences and metrics, not as a product install.

Output:

Milestone	State at year 1
Vendor	DataHub (self-hosted core + Acryl Cloud)
Assets crawled	3500+
Owned assets (%)	85%
Active-metadata workflows	6
Custom aspects	4 (contract, SLA, cost, PII-risk)
Self-serve question resolution	70%
Mesh domains onboarded	5

Why this works — concept by concept:

• Matrix as artifact — the decision is codified, defensible, and shareable. Six months later, when the team debates re-evaluating, the matrix is the base to update — not a memory of "we picked this because."
• Rollout as spine — the four-phase plan is the operational skeleton. Each phase's gate ensures the next phase has a foundation; skipping any phase (especially workflow) leaves the rollout brittle.
• Year-one cadences as insurance — freshness sweep + governance/usage balance + no-owner digest + mesh domain authoring + custom-aspect roadmap. Five cadences, each 1–2 hours a week of platform-lead time; together they prevent the catalog from decaying into a graveyard.
• Custom aspect roadmap — four quarters of first-class metadata that the closed-SaaS trio cannot ship in a year. This is the DataHub extensibility bet compounding into a mesh-native metadata model.
• Cost — self-hosted OSS core removes the seat-license drag; Acryl Cloud pays for hosted ops if and when needed. The year-one platform-lead time budget is roughly one FTE-quarter; the pay-off is a catalog that supports the mesh roadmap without vendor bottlenecks.

SQL
Topic — sql
SQL rollout planning and freshness-sweep problems

Practice →

ETL
Topic — etl
ETL problems on catalog crawl, ingestion, and ownership sweeps

Practice →

---

Cheat sheet — data catalog recipes

• When to reach for a catalog. The moment the platform crosses 200–500 tables, or the first time an analyst asks "which orders table is real?" in Slack, or the first quarter you have to answer a DSAR — one of those triggers, and the catalog is now a hard requirement. Direct discovery-via-Slack-search stops working around 200 tables and becomes an existential drag by 500.
• The four axes for every catalog interview. Discovery UX (search primitives, popularity signals), lineage depth (column-level or table-level, warehouse-only or BI-included), governance workflow (Slack-light vs BPMN-deep), extensibility (OSS + custom aspects vs closed SaaS). Score every vendor against these axes with team-profile weights.
• Atlan search operator vocabulary. owner:group_slug, tag:tag_slug, type:Table|View|Column|Dashboard|BusinessTerm, source:snowflake|bigquery|redshift|looker|tableau, term:"business term", certified:true|false, updated:>YYYY-MM-DD, popularity:high, deprecated:false. Teach in a 30-min workshop; ship a wiki cheat sheet.
• Collibra BPMN workflow skeleton. Start event → user task (review, dueDate: today+3d) → user task (sign-off, dueDate: today+5d) → service task (REST to warehouse) → service task (Slack + email) → service task (audit) → end event. Boundary timer events on user tasks enforce SLAs; missed SLAs escalate to the next-up manager.
• Alation query-log config. Extract snowflake.account_usage.query_history for the last 7 days, filter to execution_status = 'SUCCESS' AND query_type = 'SELECT'. Parse each query_text for referenced tables via sqllineage. Aggregate counts per (table, user). Refresh hourly. Surface as "Top Queried Tables" + "Top Queriers" panels.
• DataHub GraphQL — column downstream lineage. searchAcrossLineage with direction: DOWNSTREAM, types: [SCHEMA_FIELD, DATASET, DASHBOARD], lineageFlags: { maxHops: 5, entitiesExploredPerHopLimit: 200 }. Wrap in a CI helper that fails the PR if DASHBOARD count > 0 without a data-analytics reviewer.
• DataHub custom aspect skeleton. Author a .pdl file with @Aspect: { name: "yourAspect" }, register it with the metadata service, generate bindings, emit values via MetadataChangeProposalWrapper, read via GraphQL. Complete workflow: an afternoon of engineering time for a first-class metadata dimension.
• 90-day rollout plan. Day 0–14 crawl top 3 sources → Day 14–30 own + term top 100 assets → Day 30–60 open first Slack workflow → Day 60–90 onboard first non-platform team. Skip Day 30–60 (the workflow phase) and the rollout plateaus at "read-only wiki."
• Active vs static metadata. Static: catalog reflects the warehouse (crawlers, UI). Active: catalog writes into the warehouse (masking policy from PII tag, retention from lifecycle tag, DDL from schema change). Atlan and Collibra ship this natively; Alation via a module; DataHub via the OSS Actions framework.
• Governance vs usage balance. Measure both — % of top-100 assets owned (governance) and % of new-hire questions self-resolved via catalog (usage). If governance leads by 20+ points, invest in adoption (training, Slack command, onboarding checklist). If usage leads, invest in stewardship (steward hires, glossary sprints).
• Catalog rot mitigation. Quarterly freshness sweep: query all assets with last_reviewed_at > 90 days ago, file Jira tickets against current owners, escalate unassigned tickets to team managers. Ship a weekly Slack digest of stale-asset counts to #data-platform.
• No-owner cleanup. Weekly GraphQL query for unowned tables, one Slack digest per week to #data-platform, needs owner tag applied automatically. 90-day escalation: unowned tables get marked deprecated, ticket opened to archive.
• PII → masking policy write path. Steward tags column PII → catalog fires event → webhook (with HMAC signature) → warehouse service runs ALTER TABLE ... SET MASKING POLICY. The masking policy is defined once, out of band; the workflow only attaches it. HMAC validation is non-negotiable.
• DSAR skeleton. User submits DSAR → legal verifies identity (user task) → automated GraphQL query for every PII-tagged column (service task) → DPO approves deletion plan (user task) → warehouse deletion job runs (service task) → user notified (service task) → audit entry written (service task). 30-day SLA per GDPR; each step timer-enforced.
• Vendor selection tie-breakers. Ecosystem fit (Atlan wins on Snowflake+dbt+Looker), extensibility roadmap (DataHub wins on custom aspects), audit-defensibility (Collibra wins on BPMN + tri-owner), stewardship maturity (Alation wins on query-log-driven adoption). Pre-declare tie-breakers in the matrix before the vendor conversation.

Frequently asked questions

What is a data catalog and why can't we just improve dbt docs?

A data catalog is the discovery + governance + lineage layer for the whole data surface — not just dbt models. dbt docs cover the dbt project itself (typically 30–40% of a modern warehouse); the catalog covers the other 60–70%: raw ingested tables (Fivetran, Airbyte, Kafka sinks), warehouse views not authored in dbt, BI dashboards (Looker, Tableau, Mode, Power BI), downstream ML feature stores, and the business glossary that gives every asset a definition. Beyond scope, catalogs offer three capabilities dbt docs cannot: cross-source search (find "revenue" across dbt models and Looker measures and Snowflake views in one query), active metadata (push a PII classification to a Snowflake masking policy or a dbt exposure tag), and governance workflow (route ownership requests to Slack, DSARs to legal, retention decisions to compliance). The right architecture usually keeps dbt docs — the catalog links back to them — and layers the catalog on top of the 60–70% dbt docs cannot see. Never argue "we don't need a catalog" past 500 tables; the argument is really "we don't want to pay for one," and the annualised engineer-hour cost of not having one nearly always exceeds the seat cost.

Atlan vs Collibra in one line — how do I remember which is which?

Atlan is search-first + Slack-native + modern-warehouse-friendly; think "the catalog analysts want to use, wired for Snowflake + dbt + Looker + Slack." Collibra is workflow-first + business-glossary-heavy + regulated-industry-safe; think "the catalog Compliance and the CDO require an enterprise to use, wired for BPMN workflows + tri-owner stewardship + audit-loggable decision chains." The one-line pick: pick Atlan for a mid-market platform team that wants low-friction discovery without a governance council; pick Collibra for a regulated bank, hospital, or insurer where the catalog must double as the system of record for policy and lineage sign-off. The trap is buying Collibra for a team that will never staff the governance function to run it (the tool becomes a wiki with a login screen) or Atlan for a bank that will fail its next audit because the workflow chain isn't audit-defensible. Match the tool to the operating model, not to the marketing site.

Is DataHub production-ready, or is it "OSS-only, run at your own risk"?

DataHub is definitively production-ready in 2026 — it powers LinkedIn's internal catalog at massive scale, runs in production at hundreds of engineering-heavy companies (Netflix, Zendesk, Pinterest, Grofers, and many more), and Acryl Cloud offers a fully-hosted commercial tier that removes the ops burden. The OSS core is a Java/Kotlin metadata service (GMS), an Elasticsearch-backed search index, a Kafka-based event bus, and a React UI; self-hosting is a real commitment (typically one dedicated engineer for setup + ongoing ops), but the software itself is battle-tested. Where DataHub does still trail the closed-SaaS trio is (a) UX polish for non-engineering users — the UI is dense and engineer-flavoured, (b) hosted-workflow bench depth — you build governance workflows via the Actions framework rather than clicking through a workflow designer, and (c) enterprise procurement ecosystem — RFP responses, SOC 2 reports, and vendor-management relationships are newer than 15-year-old Collibra. For engineering-heavy platform teams comfortable with self-hosted infrastructure, DataHub is a first-tier choice; for governance-heavy regulated shops, Collibra or Acryl Cloud is the safer procurement narrative.

Do I need Alation if I already have dbt docs and a query-history dashboard?

Probably not — Alation's core value proposition (query-log-driven popularity + curated stewardship layer) overlaps substantially with what a mature dbt + Looker + custom-analytics-dashboard stack already delivers. Alation shines when (a) your warehouse is not mostly-dbt (heavy raw ingestion, legacy SQL views, mainframe extracts), (b) your team has a dedicated steward function that will author Articles and curate top-queried tables, or (c) your governance posture demands the mature stewardship model Alation ships (an established Alation deployment has 10 years of accumulated best practice). If you're a 40-engineer scale-up on Snowflake + dbt + Looker with no dedicated stewards, the more-defensible spend is Atlan (better discovery UX for your stack) or DataHub (extensibility for your roadmap). Alation is not a wrong choice — it's a fit choice, and the fit is regulated enterprise + mature stewardship, not modern-warehouse mid-market.

How do I compute ROI on a catalog before I've bought one?

The defensible model is anchor-in-engineer-hours, not vendor-slide-in-a-deck. Enumerate five failure modes with a per-event time cost: (1) duplicated pipelines (~3 per quarter × 1 senior-week each ≈ $36k/year), (2) wrong-source dashboards (~2 per quarter × 3 senior-days each ≈ $14k/year), (3) on-call lineage grep (~20 incidents per quarter × 30 min each ≈ $4k/year), (4) onboarding drag (~8 hires per year × 2 extra weeks each ≈ $32k/year), (5) trust-loss meetings (~4 per year × 4 senior-hours each ≈ $1.6k/year). For a 40-engineer team the annualised waste totals ~$88k. Compare against the fully-loaded catalog spend: Atlan seat pricing (illustrative $900/seat-year × 40 seats = $36k → 1.44× first-year ROI), DataHub Acryl at ~$24k (2.67× ROI), Collibra at ~$200k+ (0.44× — negative first-year unless the regulatory value is priced in), Alation somewhere between Atlan and Collibra. The catalog that passes the ROI test is the one to pick on qualitative axes; the catalog that fails is the one you cannot afford at any price. Ship this model on a single slide before the vendor conversation.

Which catalog fits a data-mesh organisation?

For data mesh, DataHub is the near-inevitable answer, and the reasoning is structural: mesh implementations require per-domain custom metadata (data contracts, freshness SLAs, cost scores, domain-specific PII policies), and custom aspects are DataHub's headline extensibility feature. In a closed-SaaS catalog, every new mesh-domain metadata field is a vendor engagement (6–12 weeks per aspect); in DataHub, it's an afternoon of Python + .pdl schema authoring. Beyond custom aspects, mesh benefits from DataHub's GraphQL API (each domain team can script the catalog into their own CI + observability), the OSS action framework (per-domain workflows without a shared vendor workflow engine), and the pluggable ingestion model (each domain runs its own ingest recipes). The closed-SaaS trio can work for mesh, but you'll pay in extensibility ceiling and vendor-ticket cycle time. Where the closed-SaaS story does win in mesh contexts is centralised governance — the mesh federation layer (usually a data platform team) still needs a governance workflow, and Collibra's BPMN engine is the most-defensible if the federation is compliance-driven. The 2026 mesh pattern most commonly ships DataHub per-domain + Collibra federation for the compliance-heavy cases; DataHub alone for engineering-heavy cases.

Practice on PipeCode

• Drill the SQL practice library → for the discovery, lineage-traversal, and metadata-modelling problems senior interviewers love to probe on the catalog axis.
• Rehearse on the ETL practice library → for the catalog-crawler, ingestion-pattern, and freshness-sweep problems that turn a vendor pilot into a rollout.
• Sharpen the tuning axis with the optimization practice library → for the decision-matrix, ROI-modelling, and governance-vs-usage-balance problems that separate senior data engineers from mid.
• Stack the prerequisites against PipeCode's broader 450+ data-engineering catalogue to anchor the catalog decision + rollout intuition against real graded inputs.

Lock in catalog decision muscle memory

Vendor docs explain features. PipeCode drills explain the decision — when Atlan wins the modern-warehouse mid-market, when Collibra owns the regulated-bank workflow, when DataHub is the only catalog that survives a data-mesh roadmap, and when Alation's query-log signal is the difference between a discovery layer that pays for itself and one that decays. Pipecode.ai is Leetcode for Data Engineering — pattern-first practice tuned for the production trade-offs senior data engineers actually face.

Practice SQL problems →
Practice optimization problems →