A data governance operating model is the answer to a question every governance program eventually fails to answer on paper: when a metric is wrong, an access grant is too broad, or a PII column ships untagged, who is accountable, who does the work, and what actually stops it from happening again? A policy PDF names none of those people and blocks none of those changes — it sits on a wiki, unread, while the pipeline keeps shipping. The operating model is the part that makes governance real: a spine of roles with named decision rights, a cadence for arbitrating conflicts, and an enforcement mechanism that runs whether or not anyone remembers the policy exists. Get the operating model right and the policy document becomes a formality; get it wrong and no amount of policy writing will save you.
This guide walks the operating model the way a senior interviewer probes it: start with the accountability spine — the data owner who is answerable for a domain versus the data steward who executes the owner's intent day to day — then the RACI decision rights that stop two people (or nobody) from owning a call, the governance council that arbitrates cross-domain conflicts without becoming a bottleneck, the federated model that scales the whole thing across dozens of teams, and finally policy as code — the shift from a written rule to an executable check that blocks a non-compliant change inside the pull request instead of flagging it six months later in an audit. Each section pairs a teaching block with a Solution-Tail interview answer: code, a step-by-step trace, an output table, then a concept-by-concept breakdown of why it works.
When you want hands-on reps immediately after reading, drill the design practice library →, rehearse the query mechanics on the SQL practice library →, and pressure-test the enforcement checks on the data validation practice library →.
On this page
- Why the operating model — not the policy PDF — decides whether governance survives
- Data owners and the accountability spine
- Data stewards and the stewardship workflow
- The governance council and the federated operating model
- Policy-as-code — encoding governance as executable checks
- Cheat sheet — operating model recipes
- Frequently asked questions
- Practice on PipeCode
1. Why the operating model — not the policy PDF — decides whether governance survives
An operating model is roles plus decision rights plus cadence plus enforcement — the document is the least important part
The one-sentence invariant: a data governance operating model is the standing arrangement of who is accountable for each data domain, who does the stewardship work, how conflicts are arbitrated, and how policy is enforced automatically — and a program dies not from a missing policy but from a missing owner, an ambiguous decision right, or a rule with no enforcement hook. Every mature program converges on the same four load-bearing components; the org chart, the tooling, and the policy wording are downstream of them. When an interviewer asks "how would you stand up data governance," the weak answer describes a document ("we'd write a data classification policy"); the senior answer describes an operating model ("we'd assign an accountable owner per domain, staff stewards, convene a council on a monthly cadence, and encode the non-negotiable rules as policy-as-code that blocks in CI").
The four components of any operating model.
-
Roles and the accountability spine. The
data owneris accountable for a domain's data (fitness, access, classification); thedata stewardis responsible for the day-to-day work; the custodian / platform team operates the storage and pipelines; thegovernance councilarbitrates cross-domain calls. One accountable party per decision — never zero, never two. - Decision rights (RACI). For every recurring decision — schema change, PII classification, access grant, retention exception — a RACI row names exactly one Accountable, one or more Responsible, the Consulted, and the Informed. Ambiguous decision rights are the single most common reason governance stalls.
- Cadence and forums. The council meets on a fixed cadence (monthly is typical) with a charter, a standing agenda, and a decision log. Cadence turns governance from a one-time project into a running process.
-
Enforcement (
policy as code). The rules that must always hold are encoded as executable checks — SQL policy queries, dbt tests, OPA/Rego rules — wired into CI so a non-compliant change is blocked at the pull request, not discovered at audit. A rule with no enforcement hook is a suggestion.
Centralized vs federated vs hybrid — pick by org shape, not fashion.
- Centralized. One central team owns standards, tooling, and execution. Works at small scale and for highly regulated single-domain shops; becomes a bottleneck past ~10 producing teams because every decision funnels through one group.
-
Federated (data-mesh style). Domains own their data products and most decisions; a thin central function owns only the global standards (
federated governance). Scales to dozens of teams but fails if the central function standardises nothing, producing forty incompatible glossaries. - Hybrid. The common real-world answer: central team owns the platform, the global standards, and policy-as-code; domains own their owners, stewards, and local rules. The council is the seam between the two.
What interviewers listen for.
- Do you distinguish owner (accountable) from steward (responsible) without prompting? — required answer.
- Do you frame governance as an operating model rather than a document? — senior signal.
- Do you name RACI and insist on exactly one Accountable per decision? — senior signal.
- Do you name policy-as-code / enforcement rather than "we'd train people on the policy"? — senior signal.
- Do you pick centralized vs federated by team count and regulatory load, not by preference? — required answer.
Worked example — the four-role accountability spine
Detailed explanation. The most useful artifact for a governance interview is a memorised four-role spine mapped to a concrete decision. Every governance discussion converges on "who is accountable versus who does the work" within the first ten minutes; having the spine in your head is what separates a fluent answer from an org-chart ramble. Walk through mapping the spine for a customer data domain feeding a marketing warehouse.
-
Domain.
customer— profile, contact, consent, and lifecycle data on Postgres, replicated to Snowflake. -
Decision under test. "A new analyst requests read access to
customer.emailfor a campaign." Who decides? - Anti-pattern to avoid. "Whoever built the pipeline owns it" — engineers operate the data; they are not accountable for who may see PII.
Question. Map the four roles to the customer domain and assign each role's part in the access-grant decision.
Input.
| Role | Accountability | Example person |
|---|---|---|
| Data owner | Accountable for the domain's fitness, access policy, classification | VP of Customer / domain lead |
| Data steward | Responsible for glossary, quality rules, executing access decisions | Analytics engineer embedded in the domain |
| Custodian / platform | Operates storage, pipelines, IAM plumbing | Data platform team |
| Governance council | Arbitrates cross-domain conflicts and exceptions | Cross-functional standing body |
Code.
Access-grant decision for customer.email (PII)
===============================================
Owner (Accountable) → sets the policy: "email is PII; access requires
business justification + owner sign-off."
Steward (Responsible) → executes: validates the justification, records
the grant in the access registry, sets an expiry.
Custodian (Operates) → applies the IAM/GRANT change in the warehouse.
Council (Informed) → sees the grant in the monthly decision-log review;
escalated only if the request crosses domains.
Step-by-step explanation.
- The owner is accountable, not hands-on: they set the standing policy ("email is PII, access needs justification + sign-off") once, so every future grant follows a rule instead of a debate. Accountability is about the outcome, not the keystrokes.
- The steward is where the daily work lives: they validate the justification against the owner's policy, record the grant with an expiry, and own the quality of the access registry. The steward has authority delegated by the owner — this is the difference between a steward and a note-taker.
- The custodian applies the mechanical change (
GRANT SELECT, an IAM role). They must not make the policy call — separating "decides" from "operates" is what prevents a platform engineer from silently widening PII access. - The council is merely Informed for a routine same-domain grant; it only becomes involved if the request is an exception or crosses domains (e.g. marketing wants finance's data). Routing routine work away from the council is what keeps it from becoming a bottleneck.
- The failure mode this prevents: with no named owner, the access request lands on whoever built the table, who says "sure" to be helpful, and PII leaks with no accountable party. The spine makes the accountable party explicit.
Output.
| Decision step | Role | RACI letter |
|---|---|---|
| Set the access policy | Data owner | A |
| Validate + record the grant | Data steward | R |
| Apply the GRANT / IAM change | Custodian | R (execution) |
| Review in decision log | Council | I |
Rule of thumb. For any governance decision, name the single Accountable party first (the owner), then the Responsible executor (the steward), then who merely operates or is informed. If you can't name exactly one Accountable, you don't have an operating model — you have a policy document.
Worked example — centralized vs federated vs hybrid
Detailed explanation. The operating-model shape is an org decision, and interviewers test whether you pick it from constraints rather than fashion. The three shapes trade central control against domain autonomy; the right one is a function of team count and regulatory load. Walk through choosing a shape for three organisations.
- Small regulated fintech. 3 data-producing teams, heavy compliance (PCI, SOC 2).
- Scaling marketplace. 25 data-producing teams, moderate regulation, speed-critical.
- Large enterprise mid-migration. 60 teams, mixed regulation, moving to a data mesh.
Question. Pick centralized, federated, or hybrid for each org and justify from constraints.
Input.
| Org | Teams | Regulation | Speed need |
|---|---|---|---|
| Regulated fintech | 3 | high (PCI, SOC 2) | moderate |
| Scaling marketplace | 25 | moderate | high |
| Enterprise migration | 60 | mixed | high |
Code.
def pick_operating_model(teams: int, high_regulation: bool) -> str:
"""Choose the governance operating-model shape from org constraints."""
if teams <= 5 and high_regulation:
return "centralized" # one team owns standards + execution
if teams >= 20:
return "hybrid or federated" # central owns global standards only
return "hybrid" # default seam: central platform + domain owners
print(pick_operating_model(3, True)) # → centralized
print(pick_operating_model(25, False)) # → hybrid or federated
print(pick_operating_model(60, True)) # → hybrid or federated
Step-by-step explanation.
- The regulated fintech has only 3 teams and heavy compliance, so a centralized model concentrates scarce compliance expertise in one place and keeps the audit surface small. The bottleneck risk is negligible at 3 teams.
- The scaling marketplace at 25 teams cannot funnel every schema change through a central team without killing velocity, so it goes federated / hybrid — domains own their data products and most decisions, and a thin central function owns only the handful of global standards (PII classification, retention, naming).
- The enterprise mid-migration at 60 teams is the canonical data-mesh case: federated computational governance, with the council owning the global minimum and each domain owning the rest. Attempting centralized here guarantees a multi-quarter backlog at the central team.
- The
pick_operating_modelhelper is illustrative, not gospel — the real decision also weighs data-product maturity and executive sponsorship — but "few teams + high regulation ⇒ centralized; many teams ⇒ federated/hybrid" is the defensible interview heuristic. - The trap the heuristic avoids: choosing federated for a 3-team shop (nobody to federate to; wasted overhead) or centralized for a 60-team shop (instant bottleneck). Shape follows constraints.
Output.
| Org | Chosen shape | Primary reason |
|---|---|---|
| Regulated fintech | centralized | few teams + concentrate compliance expertise |
| Scaling marketplace | hybrid / federated | 25 teams; central bottleneck unacceptable |
| Enterprise migration | federated | 60 teams; data-mesh computational governance |
Rule of thumb. Centralized fits few teams and heavy regulation; federated fits many autonomous teams; hybrid is the pragmatic default where a central platform owns global standards and policy-as-code while domains own owners, stewards, and local rules.
Senior interview question on governance operating models
A senior interviewer often opens with: "You join a 30-team company whose 'data governance' is a Confluence page nobody follows — metrics disagree across dashboards, PII has leaked into a public schema twice, and access grants are permanent and untracked. Walk me through the operating model you'd stand up in the first 90 days, who you'd make accountable, and how you'd make any of it stick."
Solution Using a hybrid operating model with an ownership registry and policy-as-code enforcement
-- Step 1 — make ownership explicit: an ownership registry table
CREATE TABLE governance.domain_ownership (
domain TEXT PRIMARY KEY, -- 'customer', 'orders', 'finance'
owner_email TEXT NOT NULL, -- the ACCOUNTABLE party
steward_email TEXT NOT NULL, -- the RESPONSIBLE executor
classification TEXT NOT NULL, -- 'public' | 'internal' | 'pii' | 'restricted'
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- Every governed dataset maps to exactly one owned domain
CREATE TABLE governance.dataset_domain (
dataset TEXT PRIMARY KEY, -- 'analytics.customer_email'
domain TEXT NOT NULL REFERENCES governance.domain_ownership(domain)
);
-- Step 2 — the "no orphan datasets" policy check (runs in CI nightly)
SELECT d.dataset
FROM information_schema.tables t
LEFT JOIN governance.dataset_domain d
ON d.dataset = t.table_schema || '.' || t.table_name
WHERE t.table_schema NOT IN ('pg_catalog', 'information_schema')
AND d.dataset IS NULL; -- any row here = a dataset with NO owner → fail CI
# Step 3 — council cadence + decision log (charter excerpt, checked into git)
council:
cadence: monthly
quorum: [head_of_data, security_lead, 2_domain_owners]
standing_agenda:
- new_domain_onboarding
- exception_requests # e.g. temporary broad access
- policy_changes
decision_log: governance/decisions/ # one markdown file per decision
escalation_sla_days: 5 # exceptions answered within 5 business days
Step-by-step trace.
| Step | Action | Effect |
|---|---|---|
| Registry created |
domain_ownership + dataset_domain
|
every dataset now maps to one accountable owner |
| Orphan check runs | the "no orphan datasets" SQL | CI fails if any table lacks an owner |
| Council chartered | monthly cadence + decision log | conflicts arbitrated on a fixed rhythm, not ad hoc |
| Policy encoded | orphan check + PII checks in CI | enforcement runs whether or not anyone reads the wiki |
| Access made temporary | grants carry an expiry (added in §2) | permanent-untracked-access failure mode closed |
By day 90, every dataset has a named owner and steward, the council has a running decision log, and the two enforcement checks (no orphan datasets, no untagged PII) block non-compliant changes in CI — turning a shelf document into a standing process.
Output:
| Metric | Before (Confluence page) | After (operating model) |
|---|---|---|
| Datasets with a named owner | ~0 (implicit) | 100% (orphan check enforces) |
| Access grant tracking | none (permanent) | registry + expiry |
| Conflict arbitration | ad hoc / never | monthly council + decision log |
| Policy enforcement | manual, at audit | automated, in CI (pre-merge) |
| PII leak recurrence | twice in prior year | blocked pre-merge by PII check |
Why this works — concept by concept:
- Ownership registry — a single source of truth that maps every dataset to exactly one accountable owner and one steward. Without it, "who owns this?" has no answer and every decision stalls; with it, accountability is a JOIN, not a debate.
- No-orphan-datasets check — the SQL anti-join between real tables and the registry surfaces any dataset with no owner. Wiring it into CI converts "we should assign owners" from a good intention into a merge-blocking gate.
- Council cadence + decision log — a fixed monthly rhythm with a git-tracked decision log turns governance into a process with memory; the log means a decision made once is not relitigated every quarter.
- Hybrid split — the central function owns the registry, the council, and policy-as-code; domains own their owners, stewards, and local rules. This is the seam that scales to 30 teams without a central bottleneck.
- Cost — one small registry (two tables), a handful of CI checks (O(datasets) per run), and one recurring meeting. The eliminated cost is the recurring PII-leak incident, the cross-dashboard metric disputes, and the untracked-access audit finding. Net: O(1) standing overhead versus O(incidents) firefighting.
Design
Topic — design
Design problems on data governance operating models
2. Data owners and the accountability spine
A data owner is accountable for a domain's outcomes — the person who signs off, not the person who writes the pipeline
The mental model in one line: a data owner is the single accountable party for a data domain — its fitness for use, its access policy, and its classification — and the whole point of naming one is that when something goes wrong there is exactly one person who is answerable, one signature that gates the risky decisions, and one entry in a registry that makes accountability queryable rather than folkloric. Owners rarely touch the data; they set the rules the steward executes and the checks the platform enforces. The most common and most damaging mistake is conflating the owner (accountable for the outcome) with the engineer (operates the pipeline).
What the data owner is accountable for.
- Fitness for use. The domain's data is correct, complete, and timely enough for its documented purpose. The owner defines "enough" via SLAs; the steward measures against them.
- Access policy. Who may see the data and under what justification — especially for PII and restricted classifications. The owner sets the standing policy; grants follow it.
-
Classification. Every dataset in the domain carries a classification (
public/internal/pii/restricted) that drives downstream enforcement. The owner is accountable for the classification being right. - Decision rights. The owner holds the Accountable letter on the domain's recurring decisions — schema changes, retention exceptions, new consumers — even when a steward does the work.
RACI — the decision-rights matrix that ends "who signs off".
- R — Responsible. Does the work. Can be several people (steward, engineer). Example: the steward validates and records an access grant.
- A — Accountable. Answerable for the outcome. Exactly one per decision — the cardinal rule. Usually the owner.
- C — Consulted. Two-way input before the decision (security, legal, a downstream domain).
- I — Informed. One-way notification after the fact (the council's decision log, downstream consumers).
The ownership registry — accountability you can query.
- What it is. A table mapping each domain to its owner and steward, and each dataset to its domain. Turns "who owns this?" into a JOIN.
- Why a table, not a wiki. A wiki drifts and can't gate a pull request. A registry table is queryable, diffable, and can back a CI check that fails when a dataset has no owner.
-
Provenance. Add
updated_atand, ideally, who changed the owner and when — governance itself needs an audit trail.
Common beginner mistakes.
- Two Accountable parties on one decision. If both the owner and a security lead are "A", nobody is — decisions deadlock. Exactly one A.
- Owner = whoever built the table. Engineers operate; they are not accountable for who may see PII. Separate "operates" from "decides".
- Ownership gaps. New datasets ship with no owner and quietly become orphans. Enforce ownership with a no-orphan CI check (see §1).
- RACI as decoration. A matrix nobody consults is shelfware. It must gate real decisions (the sign-off, the grant, the schema change).
Worked example — building the RACI matrix
Detailed explanation. The RACI matrix is the operating model's decision layer. Interviewers hand you a set of recurring decisions and expect you to assign exactly one Accountable to each without deadlocking. Walk through building the matrix for the customer domain's four recurring decisions.
- Decisions. Schema change; PII classification; access grant; retention exception.
- Roles. Owner, Steward, Custodian (platform), Security, Council.
- Hard rule. Exactly one A per row.
Question. Fill the RACI matrix for the four decisions and verify the one-Accountable invariant.
Input.
| Decision | Owner | Steward | Custodian | Security | Council |
|---|---|---|---|---|---|
| Schema change | A | R | R | C | I |
| PII classification | A | R | — | C | I |
| Access grant | A | R | R | C | I |
| Retention exception | C | R | — | C | A |
Code.
# Validate the one-Accountable-per-decision invariant
raci = {
"schema_change": {"owner": "A", "steward": "R", "custodian": "R", "security": "C", "council": "I"},
"pii_classification": {"owner": "A", "steward": "R", "security": "C", "council": "I"},
"access_grant": {"owner": "A", "steward": "R", "custodian": "R", "security": "C", "council": "I"},
"retention_exception": {"owner": "C", "steward": "R", "security": "C", "council": "A"},
}
for decision, roles in raci.items():
accountable = [r for r, letter in roles.items() if letter == "A"]
assert len(accountable) == 1, f"{decision}: needs exactly 1 Accountable, got {accountable}"
print(f"{decision:22} → Accountable: {accountable[0]}")
Step-by-step explanation.
- Each decision gets exactly one A. For schema change, PII classification, and access grant the owner is Accountable — these are domain-outcome decisions. The assertion
len(accountable) == 1is the automated guard against the deadlock anti-pattern. - The steward is R (Responsible) on every row: they do the work — draft the schema migration, apply the classification, validate the grant. Responsibility can be shared (schema change is R for both steward and custodian) but accountability cannot.
- Security is C (Consulted) on the risk-bearing decisions: they give input on PII classification and broad access but do not hold the sign-off. Consulted ≠ Accountable; keeping security out of the A column prevents every grant from queueing on the security team.
- The retention exception flips: the council is Accountable because a retention exception (e.g. keeping PII past the standard window for a legal hold) is a cross-cutting policy call, not a single domain's to make. The owner drops to Consulted.
- Running the validator in CI means a malformed matrix (two A's, or none) fails before it becomes the source of a real deadlock. The RACI is itself governed by policy-as-code.
Output.
| Decision | Accountable | Why that role |
|---|---|---|
| Schema change | Owner | domain-fitness outcome |
| PII classification | Owner | domain-classification outcome |
| Access grant | Owner | domain-access policy |
| Retention exception | Council | cross-cutting policy call |
Rule of thumb. Build the RACI decision-by-decision, assign exactly one Accountable per row, keep Security/Legal as Consulted (not Accountable) so they inform rather than bottleneck, and escalate genuinely cross-cutting calls (retention, cross-domain access) to the council's A column.
Worked example — the ownership registry table
Detailed explanation. The registry makes accountability queryable. The most useful query it unlocks is "for a given dataset, who is accountable, who is the steward, and what is the classification" — the exact lookup an incident responder or an access approver needs. Walk through the registry and its lookup.
-
Tables.
domain_ownership(domain → owner, steward, classification) anddataset_domain(dataset → domain). -
Lookup. Given
analytics.customer_email, return owner, steward, classification. -
Enforcement. A dataset not in
dataset_domainis an orphan → CI fails.
Question. Write the lookup that resolves a dataset to its accountable owner, steward, and classification.
Input.
| Dataset | Domain | Owner | Steward | Classification |
|---|---|---|---|---|
| analytics.customer_email | customer | vp.customer@co | ae.customer@co | pii |
| analytics.orders | orders | vp.orders@co | ae.orders@co | internal |
| analytics.gl_entries | finance | vp.finance@co | ae.finance@co | restricted |
Code.
-- Resolve a dataset to its accountable owner, steward, and classification
SELECT dd.dataset,
o.domain,
o.owner_email,
o.steward_email,
o.classification
FROM governance.dataset_domain dd
JOIN governance.domain_ownership o ON o.domain = dd.domain
WHERE dd.dataset = 'analytics.customer_email';
Step-by-step explanation.
- The query JOINs
dataset_domain(which dataset belongs to which domain) withdomain_ownership(who owns each domain). The two-table split keeps ownership at the domain grain — you assign an owner once per domain, not once per table. - Filtering on
dd.dataset = 'analytics.customer_email'resolves the incident-time question "who do I page and who signs off on access?" in one lookup. The answer is the owner (accountable) and the steward (does the work). - Returning
classificationalongside the people drives the downstream PII enforcement:piihere is what a policy-as-code check (§5) reads to require tagging and masking. - Because it's a JOIN over two small tables, the lookup is O(1) with a primary-key index on
dataset— fast enough to run inside an access-approval workflow or an incident bot. - If the JOIN returns zero rows, the dataset is an orphan (in no domain) — which is exactly the condition the no-orphan CI check in §1 flags and blocks.
Output.
| dataset | domain | owner_email | steward_email | classification |
|---|---|---|---|---|
| analytics.customer_email | customer | vp.customer@co | ae.customer@co | pii |
Rule of thumb. Keep ownership at the domain grain (owner + steward per domain), map datasets to domains in a separate table, and make the "dataset → owner/steward/classification" lookup a single indexed JOIN so incident response and access approval never have to guess.
Senior interview question on data ownership and decision rights
A senior interviewer might ask: "Two teams both claim to own the revenue metric and their dashboards disagree by 4%. Finance says it's theirs; the growth team says they defined it first. Design the ownership and decision-rights model that resolves this permanently, and show the registry and RACI entries that encode the resolution."
Solution Using a single accountable owner, a RACI escalation, and a certified-metric registry
-- Step 1 — one metric, one accountable owner (finance), recorded in the registry
INSERT INTO governance.domain_ownership (domain, owner_email, steward_email, classification)
VALUES ('revenue', 'vp.finance@co', 'ae.finance@co', 'restricted')
ON CONFLICT (domain) DO UPDATE
SET owner_email = EXCLUDED.owner_email,
steward_email = EXCLUDED.steward_email,
updated_at = now();
-- Step 2 — a certified-metric definition table: one blessed SQL definition
CREATE TABLE governance.certified_metric (
metric_name TEXT PRIMARY KEY, -- 'net_revenue'
domain TEXT NOT NULL REFERENCES governance.domain_ownership(domain),
definition_sql TEXT NOT NULL, -- the single source-of-truth SQL
approved_by TEXT NOT NULL, -- the accountable owner's sign-off
approved_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
INSERT INTO governance.certified_metric (metric_name, domain, definition_sql, approved_by)
VALUES (
'net_revenue', 'revenue',
'SUM(gross_amount) - SUM(refund_amount) - SUM(discount_amount)',
'vp.finance@co'
);
# Step 3 — RACI + escalation for the metric-definition dispute
decision: metric_definition_change
raci:
accountable: revenue_owner # finance VP — the single tie-breaker
responsible: [finance_steward]
consulted: [growth_team, analytics_council]
informed: [all_dashboard_consumers]
escalation:
on_conflict: governance_council # unresolved cross-team disputes go here
sla_days: 5
Step-by-step trace.
| Step | Before | After |
|---|---|---|
Ownership of revenue
|
disputed (2 claimants) | single owner: finance VP (registry) |
| Metric definition | two divergent SQLs (4% gap) | one certified net_revenue SQL |
| Dispute resolution | endless meetings | RACI: finance A, growth Consulted |
| Dashboard source | each team's own query | both point at certified_metric
|
| Future changes | anyone edits | owner sign-off + decision log |
Once revenue has a single accountable owner and one certified metric definition, both dashboards are rebuilt to read the certified SQL; the 4% gap disappears because there is now exactly one definition. Future change requests route through the RACI: growth is Consulted, finance is Accountable, and any deadlock escalates to the council within five days.
Output:
| Metric | Before | After |
|---|---|---|
Accountable owners for revenue
|
2 (deadlock) | 1 (finance VP) |
| Certified metric definitions | 0 | 1 (net_revenue) |
| Dashboard variance | 4% | 0% (same source SQL) |
| Dispute resolution path | none | RACI + 5-day council escalation |
| Change control | none | owner sign-off + decision log |
Why this works — concept by concept:
-
Single accountable owner — assigning
revenueto exactly one owner (finance) breaks the two-claimant deadlock. The registryON CONFLICT DO UPDATEmakes the assignment idempotent and auditable viaupdated_at. -
Certified-metric registry — one blessed
definition_sqlper metric is the source of truth both dashboards read from; two divergent definitions cannot coexist, so the 4% gap has nowhere to hide. - RACI escalation — growth stays Consulted (their input is heard) but finance holds the Accountable letter (the tie-breaker), and genuine deadlocks escalate to the council with a 5-day SLA — turning an endless debate into a bounded process.
- Sign-off + decision log — every future definition change requires the owner's approval and lands in the decision log, so the resolution sticks instead of silently drifting apart again next quarter.
- Cost — one registry row, one certified-metric row, and a RACI entry — O(1) standing state. The eliminated cost is the recurring cross-team dispute and the executive escalations it triggered. Net: a single JOIN replaces an unbounded series of alignment meetings.
Design
Topic — design
Design problems on ownership, RACI, and decision rights
3. Data stewards and the stewardship workflow
A data steward turns the owner's policy into daily practice — glossary, quality rules, issue triage, and a measurable scorecard
The invariant in one line: a data steward is the delegated executor of the owner's intent — they maintain the business glossary and metadata, author and run the quality rules, triage data issues against SLAs, and report a quality scorecard — and the role only works when the steward has real authority delegated by the owner, not just a queue of tickets and no power to say no. Where the owner is accountable for outcomes, the steward is responsible for the work that produces them. Stewardship is the operating model's engine room: it is where governance stops being a policy and becomes a Tuesday-morning routine.
What a steward actually does day to day.
- Issue triage. A quality alert fires, an access request lands, a glossary gap is reported — the steward triages by priority against an SLA and either resolves it or routes it. This is the bulk of the daily work.
-
Business glossary. The steward owns the definitions:
active_customer,net_revenue,churned. One agreed definition per term, with the owner's sign-off, so dashboards stop disagreeing. - Metadata and data contracts. Column descriptions, PII flags, freshness SLAs, and the schema contract a downstream consumer can rely on. The steward keeps metadata current as schemas evolve.
- Quality rules. The steward authors the checks — completeness, freshness, validity, uniqueness — and owns their results. Failing checks become triage tickets.
- Scorecard. The steward reports a domain quality scorecard on a cadence so the owner and council can see whether stewardship is working.
Stewardship SLAs — the measurable contract.
-
Freshness SLA. "The
orderstable is at most 6 hours behind source." Breach → P1 ticket. - Triage SLA. "P1 data incidents acknowledged in 30 minutes, resolved or escalated in 4 hours."
- Glossary SLA. "New terms defined within 5 business days of first use."
- Why SLAs matter. Unmeasured stewardship is invisible; the owner cannot tell a working steward from a struggling one. SLAs turn stewardship into a scorecard.
Authority, not just responsibility.
- The failure mode. A steward with a ticket queue but no authority to block a bad merge or reject a broken schema change is a note-taker; problems route around them.
- The fix. The owner delegates authority explicitly: the steward can fail a CI check, reject an access request that lacks justification, and require a fix before a merge. Responsibility without authority is the most common reason stewardship fails.
Common beginner mistakes.
- Steward as a passive ticket queue. No authority to say no → the role is decorative.
- Glossary drift. Definitions set once and never maintained; terms fork again within a quarter.
- Unmeasured stewardship. No scorecard → nobody knows if quality is improving or rotting.
- Steward owns everything. Overloading one steward across ten domains guarantees SLA breaches. Staff to the domain count.
Worked example — the steward's daily issue-triage queue
Detailed explanation. The steward's morning is a triage queue: a mix of quality alerts, access requests, and glossary gaps, each with a priority and an SLA clock. The skill is triaging by priority and SLA breach risk, not first-in-first-out. Walk through triaging a queue of four items.
-
Items. A freshness breach on
orders, an access request forcustomer.email, a glossary gap onactive_customer, a nullability warning onorders.discount. - Priority. P1 (SLA breach / PII), P2 (blocks a consumer), P3 (hygiene).
- Rule. Sort by (priority, SLA time remaining); PII and freshness breaches jump the queue.
Question. Triage the four-item queue into resolution order and assign each an SLA action.
Input.
| Item | Type | Priority | SLA clock |
|---|---|---|---|
| orders freshness > 6h | quality alert | P1 | breached |
| access to customer.email | access request | P2 | 8h remaining |
| active_customer undefined | glossary gap | P3 | 5 days |
| orders.discount nulls | validity warning | P2 | 1 day |
Code.
from dataclasses import dataclass
@dataclass
class Ticket:
item: str
priority: int # 1 = P1 (highest)
sla_hours_left: float # negative = already breached
queue = [
Ticket("orders freshness > 6h", 1, -0.5),
Ticket("access to customer.email", 2, 8.0),
Ticket("active_customer undefined", 3, 120.0),
Ticket("orders.discount nulls", 2, 24.0),
]
# Triage: highest priority first, then least SLA time remaining (breaches first)
order = sorted(queue, key=lambda t: (t.priority, t.sla_hours_left))
for rank, t in enumerate(order, 1):
state = "BREACHED" if t.sla_hours_left < 0 else f"{t.sla_hours_left:.0f}h left"
print(f"{rank}. P{t.priority} {t.item:28} ({state})")
Step-by-step explanation.
- The sort key
(priority, sla_hours_left)encodes the triage policy: priority dominates, and within a priority the ticket closest to (or past) its SLA deadline goes first. First-in-first-out would let the freshness breach rot while the steward answers a low-priority glossary question. - The
ordersfreshness breach is P1 with a negative SLA clock — already breached — so it sorts first. The steward acknowledges, pages the platform team (custodian) to unblock the pipeline, and posts status to consumers. - The two P2 items sort by SLA remaining:
orders.discountnulls (24h) versus the access request (8h). The access request is closer to breach, so it goes next — the steward validates the business justification against the owner's PII policy before recording the grant. - The
orders.discountvalidity warning follows: the steward files a fix ticket to the engineer and adds a data-contract test so the null pattern is caught earlier next time. - The P3 glossary gap (
active_customer) is last — five days of SLA — but it is not ignored; leaving glossary gaps unresolved is exactly how definitions fork. The steward schedules the definition with the owner's sign-off within the SLA.
Output.
| Rank | Ticket | Priority | Action |
|---|---|---|---|
| 1 | orders freshness > 6h | P1 (breached) | page custodian, notify consumers |
| 2 | access to customer.email | P2 (8h) | validate justification, record grant + expiry |
| 3 | orders.discount nulls | P2 (24h) | file fix + add contract test |
| 4 | active_customer undefined | P3 (5d) | define with owner sign-off |
Rule of thumb. Triage by (priority, SLA time remaining), never FIFO; let PII and freshness breaches jump the queue; and convert every recurring issue into a quality rule so the same ticket does not reappear next week.
Worked example — the business glossary and metadata contract
Detailed explanation. The glossary is where dashboards stop disagreeing: one agreed definition per term, owned by the steward, signed off by the owner, and expressed precisely enough to be executable. The metadata contract attaches machine-readable governance facts (owner, PII flag, SLA) to each dataset. Walk through defining active_customer and its contract.
-
Term.
active_customer— used by three teams with three different meanings. -
Agreed definition. A customer with at least one order in the trailing 90 days and no
deleted_at. - Contract. owner, steward, PII flag, freshness SLA, the definition SQL.
Question. Encode active_customer as a glossary entry plus a machine-readable metadata contract.
Input.
| Field | Value |
|---|---|
| term | active_customer |
| definition | ≥1 order in trailing 90 days AND deleted_at IS NULL |
| owner | vp.customer@co |
| steward | ae.customer@co |
| PII | no |
| freshness SLA | 6h |
Code.
# Business glossary + metadata contract (checked into git, read by the catalog)
term: active_customer
definition: >-
A customer with at least one order in the trailing 90 days
and no deleted_at (not soft-deleted).
owner: vp.customer@co # accountable
steward: ae.customer@co # responsible
classification: internal
pii: false
freshness_sla_hours: 6
definition_sql: |
SELECT c.customer_id
FROM customers c
WHERE c.deleted_at IS NULL
AND EXISTS (
SELECT 1 FROM orders o
WHERE o.customer_id = c.customer_id
AND o.created_at >= now() - INTERVAL '90 days'
)
Step-by-step explanation.
- The
definitionis prose for humans; thedefinition_sqlis the executable truth. Shipping both means the glossary is not just documentation — the same SQL can back a certified view so every dashboard computesactive_customeridentically. -
ownerandstewardembed the accountability spine directly into the metadata: the catalog can show "who to ask" on the term's page, and an access or change workflow can route to the right people automatically. -
classification: internalandpii: falseare the fields policy-as-code reads in §5 — ifpiiweretrue, the PII check would require masking and tagging before any dataset backing this term could ship. -
freshness_sla_hours: 6makes the stewardship SLA machine-readable: a monitoring job compares the backing table's max timestamp tonow()and fires a P1 if it exceeds 6 hours — closing the loop back to the triage queue. - Checking the contract into git means definition changes are reviewed, diffed, and signed off (owner approval) — the glossary cannot silently drift because every change is a pull request.
Output.
| Contract field | Resolved value | Downstream use |
|---|---|---|
| definition_sql | 90-day order + not-deleted | backs the certified view |
| owner / steward | vp.customer / ae.customer | catalog "who to ask" + routing |
| pii | false | read by PII policy check (§5) |
| freshness_sla_hours | 6 | drives freshness monitor → P1 alert |
Rule of thumb. Every glossary term ships prose and executable definition_sql, embeds owner/steward/classification/PII/SLA as machine-readable metadata, and lives in git so changes are reviewed and signed off — a glossary that cannot back a query and cannot gate a change is just a wiki page waiting to drift.
Senior interview question on the stewardship workflow
A senior interviewer might ask: "Your domain's freshness and quality are silently degrading — consumers complain but you have no numbers. Design the stewardship scorecard that makes quality visible, the SLA definitions behind it, and the query that computes the domain's daily quality score for the owner and council."
Solution Using a quality-rule registry, an SLA scorecard query, and a council-facing rollup
-- Step 1 — a quality-rule registry: each rule is a named, owned, SLA-bound check
CREATE TABLE governance.quality_rule (
rule_id TEXT PRIMARY KEY, -- 'orders_freshness'
dataset TEXT NOT NULL, -- 'analytics.orders'
dimension TEXT NOT NULL, -- 'freshness' | 'completeness' | 'validity'
threshold NUMERIC NOT NULL, -- e.g. 6 (hours) or 0.99 (ratio)
steward TEXT NOT NULL
);
-- Step 2 — each rule run records a result (fed by the nightly check job)
CREATE TABLE governance.quality_result (
rule_id TEXT NOT NULL REFERENCES governance.quality_rule(rule_id),
run_at TIMESTAMPTZ NOT NULL DEFAULT now(),
passed BOOLEAN NOT NULL,
observed NUMERIC,
PRIMARY KEY (rule_id, run_at)
);
-- Step 3 — the daily domain quality scorecard (one score per dataset)
WITH latest AS (
SELECT DISTINCT ON (rule_id) rule_id, passed
FROM governance.quality_result
ORDER BY rule_id, run_at DESC -- most recent run per rule
)
SELECT r.dataset,
COUNT(*) AS rules_total,
COUNT(*) FILTER (WHERE l.passed) AS rules_passed,
ROUND(100.0 * COUNT(*) FILTER (WHERE l.passed)
/ COUNT(*), 1) AS quality_score_pct
FROM governance.quality_rule r
JOIN latest l ON l.rule_id = r.rule_id
GROUP BY r.dataset
ORDER BY quality_score_pct ASC; -- worst datasets first for the council
Step-by-step trace.
| Step | Data | Effect |
|---|---|---|
| Rules registered |
orders_freshness, orders_completeness, orders_validity
|
quality made explicit + owned |
| Nightly job runs | writes pass/fail rows into quality_result
|
every rule has a time series |
latest CTE |
DISTINCT ON (rule_id) ... ORDER BY run_at DESC |
keeps only each rule's newest result |
| Scorecard aggregates | FILTER (WHERE passed) / COUNT(*) |
passed-rule ratio per dataset |
| Ordered worst-first | ORDER BY quality_score_pct ASC |
council sees the riskiest datasets on top |
The scorecard turns "quality is degrading, I think" into "analytics.orders is at 67% (2 of 3 rules passing), down from 100% last week." The owner sees a number; the council sees the worst datasets first; the steward has a prioritised fix list.
Output:
| dataset | rules_total | rules_passed | quality_score_pct |
|---|---|---|---|
| analytics.orders | 3 | 2 | 66.7 |
| analytics.customer_email | 4 | 4 | 100.0 |
| analytics.gl_entries | 5 | 5 | 100.0 |
Why this works — concept by concept:
- Quality-rule registry — naming each rule with a dataset, dimension, threshold, and steward makes quality explicit and owned; an unnamed check nobody owns is exactly how quality rots invisibly.
-
DISTINCT ON (rule_id)— Postgres'sDISTINCT ONwithORDER BY rule_id, run_at DESCkeeps only the most recent result per rule, so the scorecard reflects current state rather than double-counting a rule's history. -
FILTER (WHERE passed)— the aggregate filter computes the passed-rule ratio in one pass, giving a per-dataset quality score without a self-join or subquery. -
Worst-first ordering —
ORDER BY quality_score_pct ASCsurfaces the riskiest datasets at the top of the council's rollup, so scarce attention goes where it matters instead of scrolling a hundred green rows. - Cost — two small tables and one grouped query, O(rules) per scorecard run. The eliminated cost is the "we think quality is fine" blind spot and the consumer-reported incidents it hides. Net: a nightly O(rules) scan replaces reactive firefighting with a leading indicator.
Data validation
Topic — data-validation
Data validation and quality-scorecard problems
4. The governance council and the federated operating model
The council standardises only what must be global — everything else stays with the domains, or the council becomes the bottleneck
The invariant in one line: a governance council is a standing cross-functional body that owns the small set of decisions that must be global — PII classification rules, retention policy, naming standards, cross-domain access, exceptions — while federated governance pushes everything else down to the domains that own the data, and the entire model lives or dies on whether the council resists the temptation to approve routine work it should have delegated. A council that reviews every schema change is a bottleneck; a council that standardises nothing produces forty incompatible glossaries. The art is drawing the global-versus-local line and holding it.
What the council owns (and what it does not).
- Owns — the global minimum. PII/classification taxonomy, retention policy, naming and metadata standards, cross-domain access rules, and exception approvals. These must be identical everywhere or the org fragments.
- Does not own — domain-local decisions. A domain's schema design, its internal quality rules, its glossary terms, its same-domain access grants. Pushing these down is what keeps the council fast.
- The charter. A written charter defines membership, quorum, cadence, the standing agenda, and the escalation SLA. Without a charter the council drifts into either rubber-stamping or bottlenecking.
- The decision log. Every decision is recorded (git-tracked markdown, one file per decision) so a call made once is not relitigated and new domains can read the precedent.
Cadence and membership.
- Cadence. Monthly is typical; the standing agenda (new-domain onboarding, exception requests, policy changes) keeps meetings bounded. Urgent exceptions use an async path with an SLA, not an emergency meeting.
- Membership. Head of data (or platform lead), a security/privacy lead, and a rotating pair of domain owners — small enough to decide, broad enough to be legitimate.
- Quorum. Define it so decisions are not blocked by one absentee, and so no single function can unilaterally set global policy.
Federated / data-mesh computational governance.
- Global standards, local ownership. The council sets the standards; domains implement them in their own data products. This is the data-mesh principle of federated computational governance.
- Computational, not manual. "Federated computational governance" means the global standards are encoded as policy-as-code (§5) and run automatically in every domain's pipeline — the standard is a shared check, not a shared PDF.
- The self-service platform. The central platform team provides the paved road (catalog, policy checks, templates) so domains comply by default rather than by heroics.
Common beginner mistakes.
- Council as approval bottleneck. Reviewing routine schema changes → every team waits on the monthly meeting. Delegate ruthlessly.
- Federated with no global standards. Total autonomy → forty glossaries, no interoperability. Standardise the few global things.
- No decision log. Decisions evaporate; the same debate recurs quarterly. Log every call.
- Council with no teeth. Decisions that are not encoded as policy-as-code are advisory and ignored. Wire the standard into CI.
Worked example — the council charter and decision log
Detailed explanation. The charter is the council's operating contract; the decision log is its memory. Interviewers probe whether you can keep a council fast (delegating routine work) while making its global decisions durable. Walk through a charter excerpt and a single decision-log entry.
- Charter. cadence, quorum, standing agenda, escalation SLA, and — critically — an explicit "delegated to domains" list.
-
Decision log entry. one decision: "PII columns must be tagged and masked before reaching any
analytics.*schema." - Enforcement hook. the decision references the policy-as-code check that enforces it.
Question. Draft the charter's global-vs-delegated split and one decision-log entry with its enforcement hook.
Input.
| Concern | Council owns? | Enforced by |
|---|---|---|
| PII tagging + masking | yes (global) | policy-as-code PII check |
| Retention policy | yes (global) | retention job + check |
| Schema design | no (domain) | domain steward review |
| Same-domain access grant | no (domain) | steward + registry |
| Cross-domain access | yes (global) | council exception approval |
Code.
# governance/council/charter.yaml (checked into git)
council:
cadence: monthly
quorum: [head_of_data, security_lead, ">=2 domain_owners"]
standing_agenda: [new_domain_onboarding, exception_requests, policy_changes]
escalation_sla_days: 5
global_decisions: # ONLY these need council sign-off
- pii_classification_taxonomy
- retention_policy
- naming_and_metadata_standards
- cross_domain_access
delegated_to_domains: # explicitly NOT the council's to approve
- schema_design
- same_domain_access_grants
- domain_glossary_terms
- domain_internal_quality_rules
<!-- governance/decisions/2026-08-18-pii-tagging.md -->
# Decision: PII columns must be tagged and masked before analytics.*
- **Date:** 2026-08-18
- **Accountable:** governance council (security_lead sponsor)
- **Scope:** global — applies to every domain
- **Decision:** Any column classified `pii` must carry a `pii` tag and a masking
policy before it may land in an `analytics.*` schema.
- **Enforcement:** policy-as-code check `pii_columns_tagged` (blocks in CI).
- **Consulted:** all domain owners (async, 5-day window)
- **Supersedes:** none
Step-by-step explanation.
- The charter's
global_decisionslist is deliberately short — four items. Everything a domain can decide for itself is indelegated_to_domains. This explicit split is the single most important line in the document: it is what stops the council from creeping into routine approvals. -
quorumincludes ">=2 domain_owners" so global policy is never set by the central function alone — legitimacy comes from domain representation.escalation_sla_days: 5bounds how long an exception request can sit. - The decision-log entry names an Accountable (the council), a scope (global), and — crucially — an enforcement hook (
pii_columns_tagged). A decision without an enforcement hook is advisory; naming the check makes it real. - Recording
Consulted: all domain owners (async, 5-day window)shows the decision was federated-legitimate without an emergency meeting — the async consultation path keeps cadence monthly while still gathering input. -
Supersedes: noneand the dated filename make the log a durable, ordered record; a future "didn't we already decide this?" is answered by reading the log, not by relitigating.
Output.
| Charter element | Value | Effect |
|---|---|---|
| global_decisions | 4 items | council stays focused |
| delegated_to_domains | 4 items | routine work never queues on council |
| decision enforcement |
pii_columns_tagged check |
decision is enforced, not advisory |
| escalation SLA | 5 days | exceptions bounded |
Rule of thumb. Write the charter's delegated-to-domains list before its global-decisions list — governance succeeds by making the global set as small as possible and pushing everything else down, and every council decision must name the policy-as-code check that enforces it or it is just a meeting note.
Worked example — federated governance in a data mesh
Detailed explanation. In a data mesh, the council sets global standards and each domain implements them in its own pipeline. The mechanism that makes this scale is a shared policy-as-code bundle every domain runs — the standard is the same code everywhere, not a re-implementation per team. Walk through how a global retention standard is federated across three domains.
- Global standard. "PII data is retained at most 400 days unless a legal-hold exception is approved."
-
Federation mechanism. a shared
governance-policiespackage every domain's CI imports and runs. - Local ownership. each domain owns how it deletes (partition drop, soft-delete GC) but not whether it complies.
Question. Show how one global retention standard is enforced identically across three independently-owned domains.
Input.
| Domain | Storage | Local deletion mechanism | Complies with 400-day rule? |
|---|---|---|---|
| customer | Postgres | partition drop | yes (shared check) |
| orders | Snowflake | DELETE + time-travel purge | yes (shared check) |
| finance | BigQuery | partition expiration | yes (shared check) |
Code.
# governance_policies/retention.py — shared package imported by EVERY domain's CI
MAX_PII_RETENTION_DAYS = 400
def check_retention(dataset_meta: dict) -> list[str]:
"""Return a list of violations; empty list = pass. Same code in every domain."""
violations = []
if dataset_meta.get("pii") and not dataset_meta.get("legal_hold"):
retention = dataset_meta.get("retention_days")
if retention is None:
violations.append(f"{dataset_meta['dataset']}: PII with no retention_days set")
elif retention > MAX_PII_RETENTION_DAYS:
violations.append(
f"{dataset_meta['dataset']}: retention {retention}d > {MAX_PII_RETENTION_DAYS}d cap"
)
return violations
# Each domain's CI calls the SAME function against its own dataset metadata
for meta in load_domain_datasets(): # domain-local metadata, global check
for v in check_retention(meta):
fail_ci(v)
Step-by-step explanation.
-
check_retentionlives in a sharedgovernance_policiespackage the council owns. Every domain's CI imports and runs the identical function — this is the "computational" in federated computational governance: the standard is executable code shared across domains, not a document each team interprets differently. - Each domain feeds its own dataset metadata into the same check. The customer domain (Postgres partition drop), orders (Snowflake time-travel), and finance (BigQuery expiration) all delete differently — local ownership of the how — but all must pass the identical retention cap.
- The
legal_holdescape hatch is the council-approved exception: a dataset under legal hold is exempt, but only because the council recorded that exception in the decision log. The code encodes the exception so it is auditable, not a silent override. - A domain that sets
retention_days = 500on a PII dataset fails its own CI with the shared check's message — the global standard is enforced in the domain's own pipeline, before merge, without the council reviewing anything. - When the council changes the cap (say to 365 days), they bump
MAX_PII_RETENTION_DAYSin the shared package once; every domain picks it up on the next dependency update. One edit propagates the new global standard to forty teams.
Output.
| Domain | retention_days | legal_hold | CI result |
|---|---|---|---|
| customer | 365 | false | PASS |
| orders | 500 | false | FAIL (> 400d cap) |
| finance | 730 | true | PASS (legal hold) |
Rule of thumb. Federate governance by shipping the global standard as a shared, executable policy package every domain's CI runs against its own metadata — domains own how they comply, the council owns the one place the rule is defined, and changing the standard everywhere is a single edit plus a dependency bump.
Senior interview question on federated governance
A senior interviewer might ask: "You are moving 40 teams to a data mesh. Leadership wants domain autonomy but the CISO wants guaranteed PII and retention compliance everywhere. Design the council and federated enforcement model that satisfies both, and show how an exception (a legal hold) flows through it without a bottleneck."
Solution Using a thin council, a shared policy bundle, and an auditable exception workflow
# Step 1 — thin council owns ONLY the global minimum
council:
cadence: monthly
global_standards: [pii_taxonomy, retention_cap, cross_domain_access]
everything_else: delegated_to_domains
exception_path: async_pr # exceptions are pull requests, not meetings
exception_sla_days: 5
# Step 2 — shared policy bundle: one definition, every domain runs it in CI
# governance_policies/__init__.py
from .retention import check_retention
from .pii import check_pii_tagged
from .access import check_cross_domain_access
GLOBAL_CHECKS = [check_retention, check_pii_tagged, check_cross_domain_access]
def run_global_checks(dataset_meta: dict) -> list[str]:
violations = []
for check in GLOBAL_CHECKS:
violations.extend(check(dataset_meta))
return violations # non-empty => domain CI fails the merge
-- Step 3 — the exception registry: legal holds are data, not tribal knowledge
CREATE TABLE governance.policy_exception (
exception_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
dataset TEXT NOT NULL,
policy TEXT NOT NULL, -- 'retention_cap'
reason TEXT NOT NULL, -- 'legal hold: case #4471'
approved_by TEXT NOT NULL, -- council sign-off
expires_at TIMESTAMPTZ, -- exceptions must expire
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
Step-by-step trace.
| Step | Actor | Action |
|---|---|---|
| Global standard set | council | defines retention cap + PII taxonomy once |
| Shared bundle shipped | platform | every domain CI imports run_global_checks
|
| Domain builds product | domain team | owns schema, quality, same-domain access |
| Legal hold needed | domain steward | opens exception PR → council (async, 5-day SLA) |
| Exception approved | council | row in policy_exception with expires_at
|
| CI re-runs | domain CI | check reads exception registry → passes with hold |
The CISO gets guaranteed compliance (every domain runs the same global checks in CI, no exceptions slip through silently), and domains get autonomy (they own everything except the four global standards). A legal hold flows as an async pull request approved within five days and recorded as an expiring row — no bottleneck meeting, full audit trail.
Output:
| Requirement | Mechanism | Result |
|---|---|---|
| Domain autonomy | delegate all but 4 global standards | teams move independently |
| Guaranteed PII/retention | shared run_global_checks in every CI |
enforced pre-merge everywhere |
| Exceptions without bottleneck | async PR + 5-day SLA | legal hold approved, no meeting |
| Auditability |
policy_exception with expires_at
|
every override is data, and expires |
| Change a global standard | edit shared bundle once | propagates to all 40 domains |
Why this works — concept by concept:
- Thin council — owning only four global standards and delegating everything else keeps the council off the critical path of routine work; 40 teams do not queue on a monthly meeting.
-
Shared policy bundle — one
run_global_checksdefinition imported by every domain's CI guarantees the CISO's compliance requirement is enforced identically everywhere; there is no per-team reinterpretation of the rule. - Async exception PR — routing exceptions through pull requests with a 5-day SLA replaces the emergency-meeting bottleneck with a bounded, reviewable, federated-legitimate path.
-
Expiring exception registry — storing every legal hold as a row with
expires_atmeans overrides are auditable data, not tribal knowledge, and they cannot quietly become permanent. - Cost — one thin council, one shared package, one exception table — O(1) central state regardless of domain count. The eliminated cost is the per-team compliance drift and the emergency escalations. Net: enforcement cost is O(domains) CI runs (which the domains pay) while central overhead stays flat.
Design
Topic — design
Design problems on federated governance and data mesh
5. Policy-as-code — encoding governance as executable checks
Policy that lives in a PDF is a suggestion — policy-as-code moves enforcement into the pull request, where a non-compliant change is blocked, not flagged
The invariant in one line: policy as code is the practice of expressing governance rules as executable checks — SQL policy queries, dbt tests, OPA/Rego rules — wired into CI so that a change violating a rule fails the build and cannot merge, which converts governance from a periodic audit that catches violations months later into a pre-merge gate that prevents them. The single distinction that matters: a check that warns is still shelfware; a check that blocks is enforcement. Policy-as-code is the mechanism that gives the owner's policy and the council's standards actual teeth.
The policy-as-code stack.
-
SQL policy checks. Query the catalog /
information_schemafor violations — untagged PII columns, orphan datasets, missing owners. Return rows = violations; zero rows = pass. -
dbt tests / data contracts.
not_null,unique,accepted_values,relationships, plus custom generic tests. Failing tests block the dbt run and the merge. -
OPA / Rego. Policy for access and configuration — "no role may grant
SELECTon apiitable without an approved justification." Rego evaluates structured input and returns allow/deny. - Catalog contracts + tags. Governance facts (owner, classification, PII) live as machine-readable tags; checks read them.
Where the checks run — shift left.
- Pre-commit / pre-merge (best). The check runs in CI on the pull request; a violation blocks the merge. This is "shift left" — the cheapest place to catch a violation.
- In the pipeline. dbt tests and contract checks run in the transformation DAG; a failure halts the run before bad data lands.
- Post-hoc (weakest). A nightly scan that reports violations after they shipped. Useful as a backstop, useless as the only line of defense.
Warn vs block — the enforcement mode.
- Warn. Logs a violation but lets the change through. Appropriate only for a grace period while teams remediate a newly-introduced rule.
- Block. Fails CI; the change cannot merge. This is the default for any rule the owner or council has ratified. A permanently-warning check is a rule nobody actually enforces.
Common beginner mistakes.
- Checks that warn forever. No blocking → the rule is decorative. Set a grace-period end date, then flip to block.
- Policy-as-code with no owner mapping. A check fails but nobody knows who must fix it. Wire the ownership registry into the failure message.
- Drift between doc and code. The PDF says 400 days, the check says 500. The code is the source of truth; delete the PDF's number and link to the check.
- Untestable policies. "Data should be high quality" is not enforceable. Every ratified policy must reduce to a check that returns pass/fail.
Worked example — a SQL policy check for untagged PII columns
Detailed explanation. The highest-value policy-as-code check for most orgs is "no PII column ships untagged and unmasked." It reads column metadata and the classification registry, and returns any PII-looking column that lacks a pii tag. Wired into CI, it blocks the merge that would leak PII. Walk through the check.
-
Signal. Column name matches a PII pattern (
email,ssn,phone,dob) OR the dataset's domain is classifiedpii. -
Requirement. Such columns must carry a
piitag in the tag registry. - Result. Any matching column without a tag = a violation row → CI fails.
Question. Write the SQL policy check that returns untagged PII columns, and show it failing on one column.
Input.
| Column | Table | Looks like PII? | Has pii tag? |
|---|---|---|---|
| analytics.customer | yes | no | |
| customer_id | analytics.customer | no | n/a |
| phone | analytics.support | yes | yes |
| ssn | analytics.kyc | yes | no |
Code.
-- Policy check: PII-looking columns must carry a 'pii' tag. Rows returned = violations.
WITH pii_candidates AS (
SELECT c.table_schema || '.' || c.table_name AS dataset,
c.column_name
FROM information_schema.columns c
WHERE c.table_schema = 'analytics'
AND (
c.column_name ~* '(email|ssn|phone|dob|birth|passport)' -- name signal
OR EXISTS ( -- domain signal
SELECT 1
FROM governance.dataset_domain dd
JOIN governance.domain_ownership o ON o.domain = dd.domain
WHERE dd.dataset = c.table_schema || '.' || c.table_name
AND o.classification = 'pii'
)
)
)
SELECT p.dataset, p.column_name
FROM pii_candidates p
LEFT JOIN governance.column_tag t
ON t.dataset = p.dataset
AND t.column_name = p.column_name
AND t.tag = 'pii'
WHERE t.tag IS NULL; -- PII-looking column with NO pii tag → violation
Step-by-step explanation.
- The
pii_candidatesCTE finds columns that look like PII two ways: a name regex (email,ssn,phone, …) and a domain signal (the dataset's domain is classifiedpiiin the ownership registry). Combining both catches PII that a name pattern alone would miss (e.g.contact_1) and PII in explicitly-classified domains. - The outer query LEFT JOINs candidates against the
column_tagregistry, matching on thepiitag specifically. This is the anti-join pattern: keep candidates that have no matching tag row. -
WHERE t.tag IS NULLkeeps exactly the violations — PII-looking columns with nopiitag.emailandssnsurvive (untagged);phoneis filtered out (tagged);customer_idnever entered the candidate set. - Returning rows is the failure signal: the CI wrapper runs this query and fails the build if
rowcount > 0, printing each violation. Zero rows = the check passes and the merge proceeds. - To fix a violation, the steward either adds the
piitag (and the masking policy the tag triggers) or, if it is a false positive (describe_email_template), adds an explicit allow-list exception — recorded, like all exceptions, as data.
Output.
| dataset | column_name |
|---|---|
| analytics.customer | |
| analytics.kyc | ssn |
Rule of thumb. Build PII detection from both a name-pattern signal and a domain-classification signal, use a LEFT JOIN / IS NULL anti-join against the tag registry to surface untagged columns, and wire the query into CI as a blocking check — a policy check that returns rows must fail the build, not just log.
Worked example — an OPA/Rego access policy
Detailed explanation. Access decisions ("may this role read this table?") are structured policy, which is what OPA/Rego is built for. The policy takes the request (role, dataset, justification) plus context (the dataset's classification, approved exceptions) and returns allow/deny. Walk through a Rego policy for PII table access.
-
Rule. A role may
SELECTapiidataset only if it has an approved, unexpired access grant with a justification. - Input. the access request + the dataset classification + the access registry.
-
Output.
allow = true/falsewith a deny reason.
Question. Write the Rego policy that allows PII access only with an approved grant, and evaluate two requests.
Input.
| Request | Role | Dataset | Classification | Approved grant? |
|---|---|---|---|---|
| A | analyst_growth | analytics.customer_email | pii | no |
| B | analyst_finance | analytics.gl_entries | restricted | yes (unexpired) |
Code.
package governance.access
import future.keywords.if
import future.keywords.in
default allow := false
# Allow non-sensitive datasets outright
allow if {
input.dataset.classification == "internal"
}
allow if {
input.dataset.classification == "public"
}
# Sensitive datasets (pii / restricted) require an approved, unexpired grant
allow if {
input.dataset.classification in {"pii", "restricted"}
some grant in input.approved_grants
grant.role == input.request.role
grant.dataset == input.dataset.name
grant.justification != ""
grant.expires_at > input.now
}
# Human-readable deny reason for the CI / access-tool output
deny_reason := "sensitive dataset requires an approved, unexpired grant with justification" if {
not allow
input.dataset.classification in {"pii", "restricted"}
}
Step-by-step explanation.
-
default allow := falseis the safe default — deny unless a rule explicitly allows. For access policy, fail-closed is the only correct default; a missing rule must never grant access. - The first two
allowrules passinternalandpublicdatasets through without a grant — routine data should not require ceremony, which keeps the policy from becoming a bottleneck for low-risk access. - The sensitive-dataset rule is the core: for
pii/restrictedit requires an element ofapproved_grantswhose role and dataset match the request, whose justification is non-empty, and whoseexpires_atis in the future. All four conditions must hold — this is Rego's implicit AND within a rule body. - Request A (growth analyst →
customer_email, PII, no grant) matches noallowrule, soallowstaysfalseanddeny_reasonexplains why. Request B (finance analyst →gl_entries, restricted, unexpired grant) satisfies the sensitive-dataset rule →allowistrue. - Wiring this into the access-provisioning workflow (or CI for infra-as-code grants) means a broad or unjustified PII grant is rejected automatically — the owner's access policy from §2 becomes an executable gate, not a hope.
Output.
| Request | allow | reason |
|---|---|---|
| A (growth → customer_email) | false | no approved grant for PII dataset |
| B (finance → gl_entries) | true | approved unexpired grant with justification |
Rule of thumb. Write access policy fail-closed (default allow := false), let low-classification data through without ceremony, and gate every pii/restricted grant on an approved, justified, unexpired record — then evaluate the policy in the provisioning path so the rule blocks the grant instead of documenting how it should have.
Senior interview question on policy-as-code
A senior interviewer might ask: "Your org keeps shipping PII into public schemas despite a written policy forbidding it. Design the policy-as-code enforcement that makes the violation impossible to merge, show the CI gate, and explain how you roll it out to 40 repos without breaking everyone's builds on day one."
Solution Using a blocking CI gate, a grace-period rollout, and an owner-aware failure message
# Step 1 — the governance CI gate (runs on every pull request)
name: governance-gate
on: [pull_request]
jobs:
policy-as-code:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run SQL policy checks
run: python -m governance_policies.run --mode ${POLICY_MODE:-block}
- name: Run dbt tests (data contracts)
run: dbt test --select tag:governance
# Step 2 — the check runner: block vs warn, with an owner-aware message
# governance_policies/run.py
import sys, argparse
from governance_policies import run_global_checks, load_datasets, owner_of
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--mode", choices=["block", "warn"], default="block")
mode = ap.parse_args().mode
violations = []
for meta in load_datasets():
for v in run_global_checks(meta):
owner = owner_of(meta["dataset"]) # from the ownership registry
violations.append(f"[{meta['dataset']}] {v} → fix owner: {owner}")
for line in violations:
print(("WARN " if mode == "warn" else "FAIL ") + line)
if violations and mode == "block":
return 1 # non-zero exit → CI fails → merge blocked
return 0 # warn mode (grace period) never blocks
# Step 3 — staged rollout plan (no big-bang breakage)
Week 1-2 : POLICY_MODE=warn in all 40 repos. Checks run, print WARN, never block.
Dashboards show violation counts per repo + owner.
Week 3-4 : Stewards remediate; violation count trends to zero per domain.
Week 5 : Flip POLICY_MODE=block for repos at zero violations (most).
Week 6 : Flip the stragglers to block once remediated; warn mode retired.
New rules : always land in warn for 2 weeks, then auto-flip to block.
Step-by-step trace.
| Step | Mode | Effect |
|---|---|---|
| Gate added | warn | checks run on every PR, print WARN, never block |
| Violations surfaced | warn | per-repo, per-owner counts on a dashboard |
| Stewards remediate | warn | tag PII, add masking, fix contracts |
| Repo hits zero | flip to block | further violations now fail CI |
| New rule introduced | warn 2 weeks | teams remediate before it blocks |
The written policy becomes an executable gate: once a repo is in block mode, a pull request that adds an untagged PII column exits non-zero and cannot merge. The two-week warn phase means the rollout never breaks 40 teams' builds on day one — every rule lands soft, then hardens once violations are cleared.
Output:
| Metric | Before (written policy) | After (policy-as-code gate) |
|---|---|---|
| PII-in-public leaks | recurring | impossible to merge (block mode) |
| Time-to-detect a violation | months (audit) | seconds (pull request) |
| Who fixes it | unknown | named in the failure message (owner) |
| Rollout breakage | n/a | zero (2-week warn → block) |
| Source of truth for the rule | PDF (drifts) | the check code (versioned) |
Why this works — concept by concept:
-
Blocking CI gate — running the checks on
pull_requestand exiting non-zero on violation makes the merge itself the enforcement point; a non-compliant change physically cannot land, which is the difference between a policy and a gate. -
Warn-then-block rollout — starting in
warnmode surfaces violations without breaking builds, and flipping toblockper-repo once violations hit zero rolls enforcement out to 40 repos with no big-bang breakage. -
Owner-aware failure message — joining each violation to the ownership registry (
fix owner: …) means a failing check routes itself to the accountable party instead of confusing whoever opened the PR. - Code as source of truth — the rule lives in versioned check code, not a PDF, so doc-versus-code drift is impossible; the check is the policy.
- Cost — one CI job per PR, O(datasets) per run, plus a two-week warn window per new rule. The eliminated cost is the recurring PII-leak incident and the audit that finds it months late. Net: seconds of CI per PR replaces months-late audit findings.
SQL
Topic — sql
SQL problems on metadata policy checks and anti-joins
Data validation
Topic — data-validation
Data validation problems on contract tests and blocking gates
ETL
Topic — etl
ETL problems on in-pipeline policy enforcement
Cheat sheet — operating model recipes
-
The four-role spine in one line.
data owner= accountable for the domain (sets policy, signs off);data steward= responsible for the work (glossary, quality, triage) with delegated authority; custodian / platform = operates storage and pipelines;governance council= arbitrates cross-domain calls and exceptions. Exactly one Accountable per decision — never zero, never two. - RACI template. One row per recurring decision (schema change, PII classification, access grant, retention exception); columns Owner / Steward / Custodian / Security / Council; exactly one A, one or more R, C for input-givers (Security/Legal stay Consulted, not Accountable), I for the decision log and downstream consumers. Validate the "one A per row" invariant in CI.
-
Ownership registry DDL.
domain_ownership(domain PK, owner_email, steward_email, classification, updated_at)+dataset_domain(dataset PK, domain FK). Resolve any dataset to its owner/steward/classification with one indexed JOIN; a dataset absent fromdataset_domainis an orphan → fail CI (no-orphan check). -
Steward SLA scorecard.
quality_rule(rule_id PK, dataset, dimension, threshold, steward)+quality_result(rule_id, run_at, passed, observed); scorecard =DISTINCT ON (rule_id) ... ORDER BY run_at DESCfor latest results, thenCOUNT(*) FILTER (WHERE passed) / COUNT(*)per dataset, ordered worst-first for the council. -
Council charter skeleton. cadence: monthly; quorum: head_of_data + security_lead + ≥2 domain_owners;
global_decisions(short: PII taxonomy, retention, naming, cross-domain access);delegated_to_domains(schema design, same-domain grants, domain glossary, internal quality rules);escalation_sla_days: 5; git-tracked decision log, one file per decision, each naming its enforcement check. - Federated global-vs-local split. Council owns the few things that must be identical everywhere (encoded as a shared policy package); domains own everything else. "Federated computational governance" = the global standard is shared executable code every domain's CI runs against its own metadata — not a shared PDF. Change the standard once; a dependency bump propagates it to all domains.
-
Policy-as-code stack. SQL policy checks (query
information_schema/ catalog; rows = violations); dbt tests + data contracts (not_null,unique,accepted_values,relationships, custom generic tests); OPA/Rego for access (default allow := false, gatepii/restrictedon an approved unexpired justified grant); catalog tags carry owner/classification/PII the checks read. -
Enforcement modes.
warn= logs, never blocks (grace period only);block= fails CI, cannot merge (the default for any ratified rule). A check that warns forever is shelfware. New rules land in warn for ~2 weeks, then auto-flip to block. - Shift-left placement. Pre-merge CI gate (cheapest, catches on the PR) > in-pipeline dbt/contract tests (halts before bad data lands) > nightly post-hoc scan (backstop only). Never make the post-hoc scan your only line of defense.
-
Exceptions as data.
policy_exception(exception_id PK, dataset, policy, reason, approved_by, expires_at, created_at). Every override (legal hold, temporary broad access) is an approved, expiring row routed through an async PR with a 5-day SLA — never tribal knowledge, never permanent. - Maturity ladder. L0 policy PDF nobody follows → L1 named owners + registry → L2 stewards + quality scorecard → L3 chartered council + decision log → L4 policy-as-code blocking in CI → L5 federated computational governance across domains. Interviewers place you by the highest rung you can operate, not the one you can name.
- Anti-patterns. Owner = whoever built the pipeline; two Accountable on one decision; steward with responsibility but no authority; council as approval bottleneck; federated with no global standards; checks that warn forever; policy doc that drifts from the check code.
Frequently asked questions
What is a data governance operating model?
A data governance operating model is the standing arrangement of roles, decision rights, cadence, and enforcement that makes governance an operating process rather than a document. Its four load-bearing components are the accountability spine (a data owner accountable per domain, a data steward responsible for the work, a custodian that operates the platform, and a governance council that arbitrates), the RACI decision rights that assign exactly one Accountable to each recurring decision, the council's cadence and decision log, and policy as code that enforces the ratified rules automatically in CI. Programs fail not from a missing policy but from a missing owner, an ambiguous decision right, or a rule with no enforcement hook — the operating model is precisely the part that supplies those.
Data owner vs data steward — what's the difference?
The data owner is accountable for a domain's outcomes: its fitness for use, access policy, and classification. The owner sets the rules and holds the sign-off but rarely touches the data. The data steward is responsible for the day-to-day work that produces those outcomes: maintaining the business glossary and metadata, authoring and running quality rules, triaging data issues against SLAs, and reporting a quality scorecard. In RACI terms the owner is the A and the steward is the R. The critical nuance is that a steward needs real authority delegated by the owner — the power to fail a check or reject an unjustified access request — or the role degrades into a passive ticket queue that problems route around.
What does a data governance council do?
A governance council is a standing cross-functional body that owns only the decisions that must be global — the PII/classification taxonomy, retention policy, naming and metadata standards, cross-domain access, and exception approvals — and delegates everything else (schema design, same-domain grants, domain glossaries, internal quality rules) to the domains. It runs on a fixed cadence (monthly is typical) with a charter defining membership, quorum, standing agenda, and an escalation SLA, and it records every decision in a git-tracked decision log so calls are durable and not relitigated. The council's central discipline is resisting the urge to approve routine work: a council that reviews every schema change becomes the bottleneck that kills velocity, while a council that standardises nothing lets the org fragment into incompatible silos.
What is policy as code in data governance?
Policy as code is the practice of expressing governance rules as executable checks — SQL policy queries against the catalog, dbt tests and data contracts, OPA/Rego access rules — wired into CI so a change that violates a rule fails the build and cannot merge. It replaces the periodic audit (which catches violations months after they ship) with a pre-merge gate (which prevents them). The distinction that matters most is warn versus block: a check that only warns is still shelfware, while a check that blocks the merge is genuine enforcement. Policy-as-code is what gives the owner's policy and the council's standards teeth, and "federated computational governance" simply means shipping those checks as a shared package every domain's pipeline runs against its own data.
Centralized vs federated governance — which do I pick?
Pick by team count and regulatory load, not by preference. Centralized — one team owns standards, tooling, and execution — fits a small number of data-producing teams (roughly five or fewer) and heavily regulated single-domain shops, where concentrating scarce compliance expertise keeps the audit surface small; past ~10 producing teams it becomes a bottleneck. Federated (federated governance, the data-mesh model) pushes ownership and most decisions to the domains while a thin central function owns only the global standards, and it scales to dozens of teams — but it fails if the central function standardises nothing. Hybrid is the common real-world answer: a central platform owns the global standards and policy-as-code, domains own their owners, stewards, and local rules, and the council is the seam between them.
How do you make governance stick instead of shelfware?
Governance sticks when every layer has teeth. Name a single accountable data owner per domain and make it queryable in an ownership registry (a wiki drifts and cannot gate a merge). Give stewards delegated authority and a measured quality scorecard so stewardship is visible, not invisible. Run the council on a cadence with a decision log so calls are durable. Above all, encode the ratified rules as policy as code that blocks in CI — a policy that only warns, or that lives in a PDF nobody reads, is shelfware by definition. Roll new rules out in warn mode for a couple of weeks so you never break every team's build on day one, then flip them to block once violations reach zero.
Practice on PipeCode
- Drill the design practice library → for the operating-model, ownership, RACI, and federated-governance design problems senior interviewers love.
- Rehearse the query mechanics on the SQL practice library → for the ownership-registry joins, certified-metric reconciliation, and metadata policy-check anti-joins.
- Pressure-test the enforcement checks on the data validation practice library → for quality scorecards, data contracts, and blocking CI gates.
- Wire the pipeline enforcement together on the ETL practice library → for in-pipeline policy checks and shift-left governance, backed by PipeCode's broader 450+ data-engineering catalogue.
Turn the operating model into muscle memory
Docs explain governance roles. PipeCode drills explain the decision — when the owner is accountable but the steward needs authority, when the council must delegate to avoid becoming the bottleneck, when a written policy must become a blocking check, and when federated governance beats a central team. Pipecode.ai is Leetcode for Data Engineering — decision-first practice tuned for the operating-model trade-offs senior data engineers actually defend.
Practice design problems →
Practice data validation problems →





Top comments (0)