Apache Iceberg Governance: Access Control, Policies, and Audit for Open Lakehouses

Governance in an Apache Iceberg lakehouse is not a feature you enable. It is an architecture you assemble.

In a data warehouse — Snowflake, BigQuery, Redshift — governance is built in. The vendor controls access, enforces policies, manages retention, and produces audit trails. You configure roles and trust the system. The trade-off is lock-in: your data lives in a proprietary format, governed by a single vendor's rules.

Iceberg inverts this. The table format is open, engine-agnostic, and deliberately governance-unaware. Any engine can read any table. No single system owns access control. That openness creates a governance vacuum that you have to fill with a stack of purpose-built tools — and filling it correctly is one of the hardest operational problems in production lakehouses.

This guide covers the governance tools, patterns, and architectural layers that production Iceberg lakehouses need — from catalog-level RBAC and credential vending to the new Read Restrictions spec, policy engines, compliance patterns, and the operational governance layer that most teams overlook.

Why Iceberg Governance Works Differently

The Iceberg community made a deliberate choice: the table format is responsible for data portability, not governance. Governance is someone else's job.

Three reasons this is the right boundary:

Portability would be destroyed. If Iceberg metadata included ACLs and masking rules, every engine would need to implement the same governance model — the same RBAC structure, the same masking functions, the same row-filter syntax. Any engine that doesn't implement the full model either breaks compatibility or silently bypasses security.

Evolution would be frozen. Governance requirements change faster than storage formats. GDPR, CCPA, DORA, the AI Act — regulatory regimes evolve, classification taxonomies expand, organizational structures shift. Encoding governance in metadata means every policy change requires metadata migration across potentially thousands of tables.

Enforcement would be unverifiable. When rules live in the format, enforcement depends on the reader. A well-behaved Spark job respects the ACL. A misconfigured Trino deployment ignores it. A custom Parquet reader bypasses it entirely. There is no enforcement authority — only conventions that cooperative engines follow and adversarial engines ignore.

The result: governance in an Iceberg lakehouse must be decomposed into separate, independently evolvable layers. The ecosystem has converged on three.

The Three-Layer Governance Model

Every production Iceberg governance architecture decomposes into three layers:

Layer	Responsibility	Tools
Table format	Data portability and structural integrity	Apache Iceberg (metadata, snapshots, schema)
Catalog	Enforcement and coordination	Apache Polaris, Lakekeeper, Nessie, Gravitino, AWS Glue, Unity Catalog
Policy engine	Rules, classification, and context	OPA, Cedar, OpenFGA, Apache Ranger, cloud IAM

The format defines what the data looks like — how data files, manifests, snapshots, and metadata are organized. It carries structural metadata that governance systems can leverage (column names for classification, snapshot history for audit, table properties for labels), but it never interprets these as rules.

The catalog enforces governance. Every operation — table creation, schema alteration, data read, snapshot commit — routes through the catalog. This makes the catalog the mandatory intermediary between engines and data. If the catalog denies access, the engine never sees the metadata, let alone the data files.

The policy engine defines the rules the catalog enforces. This separation is critical: the catalog is the enforcement mechanism, but it should not be the rule-authoring system. Policy engines are purpose-built for defining, versioning, auditing, and distributing access policies.

The three-layer model means you can swap Ranger for OPA or Cedar without changing the catalog. You can migrate from Glue to Polaris without rewriting your policies. You can upgrade from Iceberg v2 to v3 without touching your governance model. Each layer evolves independently — which is exactly the modularity that makes open lakehouses viable in the first place.

But governance alone does not guarantee that data is actually usable. A table that passes every access control check but takes 45 minutes to query because of 200,000 small files is technically governed but operationally useless. That is where operational governance — table health, maintenance policies, and lifecycle management — becomes essential.

A control plane like LakeOps fills this gap: it handles autonomous table maintenance, structural health monitoring, and compaction across all engines — ensuring that governed data is also operationally sound.

For teams designing governance architectures, understanding how access governance and operational governance interact is a recurring theme — and one we will return to.

Layer 1: The Catalog as Enforcement Point

The catalog is where governance gets teeth. Without a governance-capable catalog, no amount of policy sophistication upstream will matter.

Apache Polaris: The Reference Implementation

Apache Polaris (graduated from incubation in early 2026) is the most fully-realized governance catalog for Iceberg. It implements a four-level RBAC model:

1. Privileges are granted to catalog roles (e.g., finance_reader grants TABLE_READ_DATA on the finance namespace)
2. Catalog roles are granted to principal roles (e.g., data_engineer_role)
3. Principal roles are assigned to service principals (e.g., the Spark ingestion service, the Trino BI cluster)
4. Privileges never attach directly to a service principal — the two-level role indirection means identity management and permission management evolve independently

The privilege model is granular: TABLE_READ_DATA, TABLE_WRITE_DATA, TABLE_CREATE, TABLE_DROP, TABLE_LIST, NAMESPACE_CREATE, NAMESPACE_LIST, CATALOG_MANAGE, plus administrative privileges for managing grants. Privileges scope to the namespace hierarchy — granting TABLE_READ_DATA on analytics implicitly covers analytics.clickstream and analytics.sessions. This enables the principle of least privilege: a data scientist's service principal can read tables in analytics without being able to create tables, alter schemas, or access the raw namespace.

Polaris 1.7.0 also introduced a Policy framework for defining lifecycle and operational rules directly in the catalog:

POST /api/catalog/v1/{catalog}/namespaces/{namespace}/policies
{
  "name": "production-snapshot-retention",
  "type": "system.snapshot-expiry",
  "content": {
    "max-snapshot-age-ms": 604800000,
    "min-snapshots-to-keep": 5
  }
}

Policies attach at catalog, namespace, or table level, with inheritance flowing downward. A table-level policy overrides namespace defaults, which override catalog defaults. This gives teams declarative, auditable lifecycle governance without per-table manual configuration.

Credential Vending: Zero-Trust Storage Access

Credential vending is the foundation of zero-trust lakehouse security, and it solves one of the hardest governance problems.

Without credential vending, engines need direct access to the storage bucket — which means they can read any file in the bucket regardless of table-level access controls. With credential vending, the catalog vends short-lived, scoped storage credentials (AWS STS temporary credentials, GCS signed URLs, Azure SAS tokens) that only allow access to the specific data files in the specific table the engine is authorized to read.

The flow:

1. Engine authenticates to Polaris with OAuth2 credentials
2. Polaris evaluates the request against its RBAC model
3. If authorized, Polaris calls AWS STS AssumeRole with an inline session policy scoped to the exact table locations
4. Engine receives temporary credentials valid for minutes — read-only, specific path prefix, short-lived
5. Engine accesses storage directly with scoped credentials

No long-lived storage credentials are distributed. A compromised engine token is scoped to one table and expires quickly. The catalog is the single chokepoint through which all storage access flows.

Why Catalog-Level Enforcement Matters for Multi-Engine

This is where catalog-level governance pays for itself. When Spark, Trino, Flink, DuckDB, and Snowflake all access the same Iceberg tables, the alternative is configuring access control separately in each engine. Consider what that looks like in practice:

• Spark has its own ACL model via TableCatalog security
• Trino has access-control.type in its catalog configuration, with its own rule syntax
• Flink relies on whatever identity the job was submitted with
• DuckDB has no native access control model at all

Any drift — a permission granted in Spark but not in Trino — creates a bypass path. A data engineer discovers they can read restricted.financial_transactions through DuckDB even though Trino denies the same query. The policy was configured in Trino's access control file but not in the DuckDB deployment, because DuckDB has no mechanism to enforce it. This is not hypothetical — it is the default failure mode in multi-engine lakehouses without catalog-level enforcement.

With catalog-level enforcement, every engine that connects through the REST Catalog protocol inherits the same governance model. A data scientist denied access to financial.transactions is denied regardless of whether they query through Spark, Trino, or DuckDB — because the catalog never returns the metadata, and credential vending never issues scoped storage credentials. A new engine inherits the full governance model the moment it connects. A policy change propagates immediately because the enforcement point is singular.

For a detailed comparison of catalog governance capabilities, see the catalog comparison on LakeOps.

Layer 2: Policy Engines — OPA, Ranger, and Cloud IAM

The catalog enforces decisions. Policy engines make them. This separation matters because production enterprises already have policy systems, and a governance architecture that requires replacing all of them is dead on arrival.

Open Policy Agent (OPA)

OPA is the standard for policy-as-code in cloud-native environments. Policies are written in Rego, versioned in Git, tested with unit tests, and deployed through CI/CD — the same workflows that manage application code. This is governance-as-code: auditable, reviewable, reproducible. In 2026, Polaris's OPA integration is maturing, and Lakekeeper includes a native OPA bridge that translates its OpenFGA permissions into OPA format for engines like Trino.

An OPA policy for Iceberg table access can evaluate rich context beyond identity:

package iceberg.authz

default allow = false

allow {
    input.action == "TABLE_READ_DATA"
    input.resource.classification != "restricted"
    input.principal.team == input.resource.domain
}

allow {
    input.action == "TABLE_READ_DATA"
    input.resource.classification == "restricted"
    input.principal.clearance == "pii_cleared"
    is_business_hours(input.request.timestamp)
}

This is attribute-based access control (ABAC) — policies that evaluate the requesting principal's team membership and clearance, the table's data classification, the columns being accessed, the time window, and the query pattern. ABAC goes where RBAC cannot: purpose limitation under GDPR (restricting access based on declared query purpose), time-restricted production access, classification-conditional column masking.

Lakekeeper: Cedar and OpenFGA

Lakekeeper — a Rust-based Iceberg REST catalog — takes a different approach to authorization by supporting two policy backends:

OpenFGA (open-source default) uses relationship tuples for ReBAC — relationship-based access control. Permissions are expressed as relationships between principals and objects, with bi-directional inheritance flowing through the namespace hierarchy. An object owner can grant access to their own tables through the API or UI, combining RBAC with discretionary access control. Lakekeeper requires OpenFGA v1.11+ for idempotent-write semantics and includes an OPA bridge that translates OpenFGA permissions into OPA format for query engines like Trino.

Cedar (available with Lakekeeper+) embeds authorization directly — no external service required. Policies are versioned, reviewable code with per-decision audit traces. Cedar supports ABAC natively: policies can evaluate resource attributes, tags, time conditions, and request context. A powerful pattern is storing access-control lists as table properties (access-readers, access-owners) that Cedar evaluates at query time — access control metadata lives with the data it governs.

	OpenFGA	Cedar
Extra service	Yes (OpenFGA + database)	No, built into Lakekeeper
Permission model	Relationships stored as data	Policies deployed as code
ABAC support	No	Yes (time, tags, attributes)
Runtime changes	API/UI grants	Redeploy policy source

Apache Ranger

Ranger has been the governance standard in Hadoop ecosystems for over a decade. Organizations with existing Ranger deployments — policies, audit logs, classification taxonomies — need a migration path that preserves their investment. Polaris delegates authorization decisions to Ranger's REST API, passing table identity, namespace, operation type, and requesting principal. Ranger evaluates against its policy store and returns the decision. The enforcement point changes; the rules remain.

Cloud IAM Integration

AWS IAM, Google Cloud IAM, and Azure RBAC are the default authorization systems for cloud infrastructure. The key principle: the catalog does not replace cloud IAM — it layers table-level governance on top. Cloud IAM controls who can reach the catalog. The catalog controls who can access specific tables. The policy engine defines the rules for both.

Iceberg Read Restrictions: Spec-Level Fine-Grained Access Control

The most significant governance development in the Iceberg ecosystem in 2026 is Read Restrictions — an extension to the Iceberg REST Catalog specification that moves column masking and row-level security into the catalog protocol itself.

How It Works

When a client loads a table, the catalog now returns the Iceberg metadata along with two additional governance instructions:

• required-row-filter — an Iceberg predicate (e.g., country = 'USA') that the reader must apply. Any row evaluating to false must not appear in results.
• required-column-projections — a list of columns the reader must mask per a specified action.

Both are scoped to the principal who made the request. The same table returns different restrictions for different callers as policies and roles change.

{
  "read-restrictions": {
    "required-row-filter": {
      "type": "eq",
      "left": { "type": "reference", "id": 3 },
      "right": { "type": "literal", "value": "USA" }
    },
    "required-column-projections": [
      { "field-id": 4, "action": "show-last-4" },
      { "field-id": 7, "action": "sha-256-global" },
      { "field-id": 12, "action": "replace-with-null" }
    ]
  }
}

The spec defines nine standardized masking actions — replace-with-null, show-last-4, sha-256-global, mask-alphanum, truncation, and others. Two engines implementing the spec independently produce identical masked output, making governance truly interoperable.

The Trust Contract

Read Restrictions shift governance from a deny-or-allow model to a nuanced instructions model. The catalog evaluates its policies and returns caller-specific restrictions; the trusted engine is responsible for applying them. Trust is established through mTLS or OAuth delegation — the catalog must know that the engine will actually enforce the restrictions before returning data access.

This is a pragmatic design. It avoids the impossible requirement that every engine implement the same governance model natively. Instead, the catalog speaks a standard governance protocol, and engines that support it participate in fine-grained access control automatically.

The Identity Gap

One practical limitation: Read Restrictions work cleanly for engines with federated identity where the requesting user's identity flows through to the catalog. Spark with connect-style setup and Trino with OAuth pass-through can propagate per-user identity. But the identity models are far from uniform:

• Spark authenticates per-application (one identity per Spark session, shared across all users of a notebook or ETL job)
• Trino supports OAuth pass-through for interactive queries but uses service accounts for scheduled jobs
• Flink connects per-job — a streaming pipeline runs as a single service principal for its entire lifetime
• DuckDB, ClickHouse, Postgres typically connect with a single set of catalog credentials — no per-query identity pass-through at all

For engines without per-user identity, Read Restrictions apply uniformly to the service principal, not to the human who submitted the query. A show-last-4 mask on SSN protects data from the service account — but if that account has full access, every user behind it sees unmasked data. Identity-mapping infrastructure that translates per-engine session identity to catalog-level principals mostly doesn't exist yet.

This is not a flaw in Read Restrictions — it is a gap in the identity fabric that the ecosystem is still building. For production deployments, it means Read Restrictions are most effective when paired with engines that support identity federation, and least effective for the embedded and lightweight engines where governance gaps tend to be largest.

Compliance Patterns: GDPR, Retention, and Audit

Governance in regulated environments goes beyond access control. Compliance requires proving that data lifecycle policies are enforced continuously.

GDPR Right to Be Forgotten

Iceberg's immutability model creates a direct tension with GDPR Article 17. When you run DELETE FROM users WHERE user_id = 48213, Iceberg marks the rows as deleted (via delete files or data file rewrites) but creates a new snapshot while keeping all old snapshots intact. The old data is still accessible through time travel.

Physical erasure requires a three-step pipeline:

Step	Operation	What Happens
1	DELETE	Logically removes the row (creates delete file)
2	Compaction	Rewrites data files without the deleted rows
3	Snapshot expiration + orphan cleanup	Removes old data files that contained the row

Only after all three steps is the personal data physically, irreversibly removed from storage. But the Iceberg table is not the only place the data lives. S3 versioning preserves old object versions even after Iceberg deletes the file. Disaster-recovery replicas, downstream data marts, ML feature stores, BI caches, and backups may all contain copies. A complete erasure pipeline must account for every replication path — not just the source table.

A table with 14-day snapshot retention cannot complete erasure in less than 14 days without expiring snapshots early — which removes the recovery window for every other row. The practical solution: set retention windows shorter than the regulatory deadline on any table holding personal data. GDPR allows one month; a table with seven-day snapshot retention, a seven-day S3 versioning lifecycle, and weekly maintenance completes the sequence in under three weeks.

Cryptographic Shredding: The Alternative

Cryptographic shredding inverts the problem. Instead of finding and destroying every copy of the data across snapshots, replicas, and backups, you encrypt personal data under a key unique to each data subject, store the keys in a small, mutable key store, and delete the key when an erasure request arrives.

Every copy of the ciphertext — in every snapshot, version, replica, backup, and downstream extract — becomes unreadable at the same instant. The ciphertext is still there. The information is gone.

The pattern:

1. Each data subject gets a data encryption key (DEK), wrapped by a KMS master key
2. PII columns are encrypted with the subject's DEK before being written to the lakehouse
3. The key store is a small table (bucketed on user_id) with aggressive retention — one-day snapshot retention, copy-on-write deletes
4. Erasure = delete the subject's row from the key store

This is distinct from Iceberg's native table encryption (which protects entire tables at rest with per-file keys). Crypto-shredding is an application-level pattern for per-subject erasure. The two compose: a table can use Iceberg encryption for at-rest protection and per-subject keys for erasure.

The trade-offs are real: encryption adds read/write overhead to every query touching PII columns, the key store becomes a critical dependency, and AWS KMS enforces a 7–30 day waiting period for key deletion. But for large tables where full compaction is expensive, or environments with many downstream copies, crypto-shredding can be the only way to meet erasure deadlines reliably.

Why Governance and Maintenance Are Inseparable

Without automated, continuous maintenance — compaction running after deletes, snapshot expiration enforced on schedule, orphan cleanup reclaiming storage — GDPR erasure deadlines cannot be met reliably across hundreds of tables. An erasure queue with deadlines and completion tracking is the audit story. Without it, you cannot prove compliance. Manual maintenance means manual compliance, which means eventual non-compliance.

Retention Policies

Different data classifications require different retention periods. Financial data under SOX may need seven years of queryable history. Operational data may need 90 days. PII-containing tables may need the shortest retention possible to limit exposure surface.

Snapshot retention governance should enforce:

• Minimum retention periods per classification level
• Maximum retention periods for PII-containing tables
• Approval workflows for expiring snapshots on compliance-sensitive tables
• Audit logging recording which snapshots were expired, by whom, when

Cross-Engine Audit Trails

With engine-level governance, audit logs are scattered across Spark logs, Trino logs, Flink logs, and engine-specific monitoring systems. Compliance auditors do not want to correlate six different logging systems to reconstruct who accessed what. They want a single, authoritative record.

Catalog-level governance provides this: every access request — regardless of which engine initiated it — is logged in one place. But access audit is only half the story. Maintenance operations — compaction, snapshot expiration, orphan cleanup, manifest rewriting — are also governance-relevant actions. A compaction job rewrites data files. A snapshot expiration permanently removes time-travel capability. An orphan cleanup deletes files from storage.

Most governance systems handle read and write access well. Few handle maintenance operations at all. The result is a gap: the governance system controls who can query and who can ingest, but maintenance operations run outside the governance model, often as overprivileged service accounts with blanket access.

For a deep dive into how the three-layer model addresses this, see the governance separation of concerns guide.

Schema Evolution Governance: Valid Is Not Safe

Iceberg's schema evolution is mechanically safe — field IDs ensure that column renames, reorders, and drops never corrupt data. A renamed column still maps to the same field ID. A dropped column simply disappears from the current schema while old snapshots retain it. No data files are rewritten.

But mechanically safe is not operationally safe. A column rename from ship_date to shipped_at is a metadata-only operation that preserves the field ID — and simultaneously breaks every Spark job, dbt model, Trino query, and BI dashboard that references the old name. The catalog does not know what consumers have open query plans. The Iceberg spec has no built-in mechanism to enumerate downstream impact before a commit lands.

Schema governance requires treating schema changes as releases, not commits:

1. Diff — generate a field-ID-aware diff (not a string diff) that distinguishes renames from drop-then-add
2. Classify — label each change as non-breaking (column add with default), potentially breaking (rename), or definitely breaking (type narrowing, drop of a non-nullable column)
3. Enumerate impact — check against query history, lineage metadata, dbt manifests, and BI connections to identify affected consumers
4. Gate — require approval for potentially breaking and definitely breaking changes
5. Stage rollback — pin a snapshot tag before the change lands so rollback has a named target, not a guessed snapshot ID
6. Verify — run smoke tests against the new schema before promoting to production

Partition evolution carries the same risk profile. A partition-spec change that requires rewrite_data_files should be flagged before publish, not discovered from the next compute bill.

Neither Polaris nor any current catalog provides this release-gate workflow natively. Teams either build custom CI checks against the catalog API, use emerging tools like Kastor for change management, or accept the risk of deploying schema changes without impact analysis.

Operational Governance: The Layer Most Teams Miss

Here is the gap that trips up most teams designing Iceberg governance: access governance tells you who can read and write data. It says nothing about who can maintain it — and maintenance operations have direct governance implications.

Who Can Compact?

Compaction reads existing data files, merges them, and writes new ones. This requires both read and write access. A malicious or buggy compaction job could introduce data corruption, alter sort order, or fail mid-operation leaving the table degraded. Restricting compaction to a trusted maintenance service — and auditing every compaction operation — is a governance requirement.

Who Can Expire Snapshots?

Snapshot expiration permanently removes time-travel capability. In regulated environments, prematurely expiring snapshots on a financial table can violate retention requirements. Governance policies must enforce minimum retention periods per classification, and the expiration service must be auditable.

Who Can Delete Orphan Files?

Aggressive orphan cleanup can delete files still needed by in-progress operations. Orphan cleanup requires a grace period, validation against active snapshots, and restriction to service accounts that understand the table's operational state.

Closing the Gap with a Dedicated Control Plane

Instead of granting maintenance privileges to multiple engines and hoping they coordinate, a dedicated operational control plane runs all maintenance through a single authority that respects table-level policies, avoids conflicts with active writers, sequences operations correctly, and logs every action.

LakeOps fills this role as a control plane for Apache Iceberg lakehouses. Its governance capabilities are specifically designed for operational governance:

Declarative maintenance policies define compaction thresholds, snapshot retention windows, orphan cleanup schedules, and manifest rewrite targets at organization, catalog, namespace, or individual table scope. Policies cascade with clear precedence: a table-level override wins over namespace defaults, which override catalog defaults, which override organization baselines. New tables inherit the correct configuration from their namespace automatically.

Configuration policies enforce consistent Iceberg table settings — format version, file format, write distribution mode, commit retry behavior — across all tables in scope. Without these, table configurations drift as different teams create tables with different defaults, resulting in inconsistent behavior across the lake.

Lake-wide event audit trail logs every maintenance operation across all catalogs and namespaces — what ran, when, duration, files before/after, bytes reclaimed, and the signal that triggered it. The same feed powers compliance evidence for SOC 2 and GDPR: proof that retention policies are enforced continuously, not just configured.

Learn more:

Agentic AI for Iceberg Lakehouse | LakeOps

AI agents query your Iceberg data lakehouse via MCP with read-only, cost-cap, and PII guardrails. Engine routing keeps the lake optimized for AI workloads.

lakeops.dev

Policy versioning tracks every policy change — who modified what, when, and what the previous settings were. The combination of operation logs and policy versioning creates an auditable governance record.

AI agent guardrails address an emerging governance surface: AI agents querying the lakehouse via MCP, Postgres wire protocol, or REST API. LakeOps routes agent queries through layered guardrails — ReadOnly blocks DDL/DML from agent sessions, CostEstimate rejects queries exceeding scan thresholds, PIIMask hashes sensitive columns before results reach the model, and HumanApproval pauses high-stakes operations for review. Agent query telemetry feeds back into compaction and sort-order decisions, closing the loop between agent access patterns and table optimization.

Learn more:

Safe AI Agent Access to Apache Iceberg in Production - LakeOps Blog

How to give AI agents safe access to Apache Iceberg data — guardrails for ReadOnly, CostEstimate, PIIMask, and HumanApproval with governance and routing.

lakeops.dev

For the full operational model, see the LakeOps platform page. For a deep dive into how operational governance intersects with access governance in the three-layer model, see the data lake governance guide.

Building the Governance Stack: A Practical Path

For teams designing governance for a production Iceberg lakehouse, the three-layer model suggests a clear implementation path:

Step 1: Choose a Governance-Capable Catalog

The catalog determines your governance ceiling. If it does not support fine-grained access control, credential vending, and policy engine integration, no amount of policy sophistication upstream will help.

Catalog	RBAC	Credential Vending	Policy Engine Integration	Read Restrictions
Apache Polaris	Full (namespace, table, column)	AWS, GCS, Azure	OPA, Ranger, custom	Yes (spec support)
Project Nessie	No built-in RBAC	No (engines bring own creds)	Pair with Polaris or OPA	No
Lakekeeper	RBAC, ReBAC, ABAC (OpenFGA + Cedar)	AWS, GCS, Azure	OpenFGA, Cedar, OPA bridge	Planned
Apache Gravitino	Namespace/table grants	Via federated catalogs	Custom integration	In progress
AWS Glue	IAM-based	IAM role assumption	IAM policies	Limited
Unity Catalog	Full (row, column, table)	Native	Built-in	Proprietary

Step 2: Establish Namespace and Classification Strategy

Before writing policies, define the organizational structure. A practical starting point:

• Three sensitivity tiers: public, internal, restricted
• One namespace per data domain per tier: e.g., internal.marketing, restricted.finance
• Column-level classification for PII fields

This structure is simple enough to implement in a week and comprehensive enough to cover most governance requirements. It also maps naturally to RBAC grants: data engineers get write access to raw.*, data scientists read curated.*, analysts read published.*.

Step 3: Implement RBAC as the Baseline

Start with role-based access control using the catalog's native privilege model. Define roles mapping to organizational functions, grant namespace-level and table-level privileges, enforce through the catalog. This covers 80% of access control requirements and is auditable, understandable, and manageable without external tooling.

Step 4: Layer ABAC Through a Policy Engine

Once RBAC is in place, add attribute-based policies for requirements roles cannot express — classification-based access, column masking, time restrictions, purpose limitation. Deploy OPA, Cedar, or Ranger depending on your catalog choice (OPA/Ranger for Polaris, Cedar or OpenFGA for Lakekeeper), integrate with the catalog, and version policies in Git alongside application code.

Step 5: Automate Maintenance Governance

Governance without operational health is incomplete. A governed table with millions of small files, thousands of accumulated snapshots, and orphaned data consuming storage passes every access control check but fails every performance expectation. And a table without automated maintenance cannot meet GDPR erasure deadlines — compaction, snapshot expiration, and orphan cleanup must run continuously, not manually.

Close the loop by connecting the governance catalog (who can access) with an operational control plane (ensuring the data is healthy, fast, and compliant). The governance catalog enforces access policies. LakeOps ensures the underlying tables are structurally healthy, compacted, and query-ready — with full audit trails that satisfy the same compliance requirements as your access governance. For tables holding personal data, the same maintenance automation that keeps query performance healthy also drives the three-step erasure pipeline to completion within regulatory deadlines. For a complete production readiness checklist that includes governance alongside maintenance, see the Iceberg production readiness checklist.

The Bottom Line

The open lakehouse disaggregated storage, compute, and metadata into independent layers. Governance must follow the same decomposition — rules separate from enforcement, enforcement separate from the format, operational governance separate from access governance.

The tooling exists. Apache Polaris provides catalog-level RBAC, credential vending, and policy engine integration. Lakekeeper adds Cedar and OpenFGA for policy-as-code and relationship-based access control. Read Restrictions bring interoperable column masking and row filtering into the REST Catalog spec. OPA and Ranger provide external policy engines with full audit trails.

LakeOps provides the operational governance layer — declarative maintenance policies, AI agent guardrails, and lake-wide audit trails that ensure governed data is also structurally healthy.

Start with the catalog choice — it determines your governance ceiling. Layer policies incrementally: RBAC first, ABAC when needed. Treat schema changes as releases, not commits. Build an erasure pipeline that accounts for every replication path — or invest in cryptographic shredding for tables where full compaction cannot meet deadlines. And close the operational loop from day one. The teams that treat access governance, schema governance, and operational governance as three facets of the same architecture — each with its own tooling and audit trail — are the ones running multi-engine lakehouses at scale without governance gaps or compliance drift.