DEV Community

Krishna Tangudu
Krishna Tangudu

Posted on

Mirroring Snowflake Iceberg into Microsoft Fabric: The Gotchas

#microsoft #dbt #snowflake #iceberg

Previously: In Power BI Connectivity with Azure AD, I walked through enabling end-to-end SSO from Power BI to Snowflake.

In this post: A hands-on implementation log for Snowflake mirroring (including Iceberg tables) in Microsoft Fabric — what official docs confirm, what broke in production, and the runbook that now works. If you're seeing differences between Mirroring Status and SQL Analytics Endpoint, this should help.

Our goal was to serve analysts from Fabric's shared OneLake layer instead of having every downstream workflow repeatedly query Snowflake directly.

TL;DR

  • Fabric Mirroring for Snowflake continuously replicates source data into OneLake.
  • For Iceberg tables, Fabric replicates metadata to OneLake via shortcuts to the underlying storage and OneLake exposes data in Delta format for Fabric workloads.
  • SQL Analytics Endpoint is read-only and can show type/metadata limitations even when mirroring is healthy.
  • New or changed tables can take time to appear consistently in SQL Analytics Endpoint because replication and SQL metadata sync are separate steps.
  • Connection configuration accuracy matters (especially warehouse identity and permissions).
  • Not all Snowflake table types are eligible for mirroring.

What I validated against official Microsoft docs

1. How mirroring works for Snowflake in Fabric

From Microsoft documentation, mirroring creates:

  • A mirrored database item in Fabric
  • A SQL Analytics Endpoint for querying the mirrored data

For Snowflake sources specifically:

  • Iceberg table metadata is replicated into OneLake using shortcuts to the storage that contains those Iceberg tables.
  • OneLake automatically converts those Iceberg tables to Delta Lake formatted tables for Fabric workloads.
  • Managed table data is replicated into OneLake and converted to analytics-ready format.

This means consumption in Fabric is against replicated data in OneLake, not a live pass-through to Snowflake. When users say "SQL endpoint looks wrong," I now split diagnosis into replication-plane checks vs SQL metadata-plane checks.

2. SQL Analytics Endpoint is read-only and has data type/materialization limits

Microsoft documents that SQL Analytics Endpoint uses the same engine family and constraints as Fabric Warehouse for persisted/object materialized types. In practice this explains why:

  • Mirroring can be healthy
  • But some columns can be missing/truncated/not surfaced as expected in SQL Analytics Endpoint

This is especially important when source types do not map cleanly to endpoint-supported persisted types.

3. Snowflake mirroring limitations that matter in design

Official Snowflake mirroring limitations include:

  • Regular/native table replication focus (not all table categories are supported)
  • A current mirrored table-count limit per mirrored database
  • Schema-change replication nuances (some schema changes need data changes to trigger propagation)
  • Backoff behavior when source tables are idle

4. Cost and serving model

From Microsoft guidance:

  • Fabric query workloads (SQL, Spark, BI) consume Fabric capacity.
  • Snowflake compute can be incurred for data-change reads during replication.

This is not a one-time load model; mirroring is continuous synchronization. Total cost depends on change volume, replication activity, and Fabric-side query demand.

Why we still chose it: it reduced repeated direct-query pressure from many downstream users and gave us a shared serving layer across SQL, Spark, and BI.

Incident Log: What broke and what fixed it

Simple-looking errors often had multiple causes: connection identity, permissions, metadata timing, and endpoint type handling. The key discipline was separating what docs explicitly confirm from support heuristics.

Ticket #1–2: Connection and warehouse errors

Symptom

Mirroring refresh failed with warehouse-not-found / not-authorized style errors.

What we found

The configured warehouse identity in Fabric connection did not align with the warehouse defined/accessible in Snowflake for the mirroring principal.

Fix

  • Stop replication
  • Recreate the Snowflake connection in Fabric
  • Re-enter server/warehouse carefully
  • Ensure the principal has warehouse usage + required mirroring privileges
  • Rebind mirrored database to the corrected connection
  • Restart replication and monitor

Important precision

Microsoft docs explicitly tell you to normalize server entry format (for example remove protocol and use lowercase server host format in setup guidance). For warehouse value handling, my operational practice is to treat it as an exact identifier and validate both existence and authorization before restart.

Ticket #3: Mirroring successful, but SQL Analytics Endpoint shows column issues

Symptom

Mirroring status looked healthy, but querying via SQL Analytics Endpoint showed warnings or incomplete column exposure for some tables.

Root cause pattern

Endpoint-level SQL metadata and type mapping did not always represent source columns as expected.

Why this happens

This behavior is aligned with documented SQL Analytics Endpoint limitations and data type mapping constraints. Mirroring into OneLake can still be successful while SQL endpoint representation is partially constrained.

Mitigations

  • Validate data in OneLake directly (for example via Spark) when endpoint output looks suspicious
  • Cast unsupported/problematic source columns to endpoint-friendly types in curated tables/views
  • Use SQL endpoint refresh and metadata sync checks where relevant
  • For critical workloads, create curated serving layers specifically for endpoint compatibility

Architecture model that helped me

Think about Snowflake mirroring in Fabric as three planes:

Replication plane

  • Movement/sync of data and metadata into OneLake
  • Includes Iceberg metadata handling + managed table replication

Storage/open format plane

  • Data persisted in OneLake with Delta interoperability for Fabric engines

Serving plane

  • SQL Analytics Endpoint as read-only, T-SQL-friendly surface
  • Useful, but not equivalent to full source-type fidelity

Most confusion in operations comes from mixing plane #1 health with plane #3 usability.

My production runbook now

Before enabling mirroring:

  • Validate Snowflake privileges for mirroring principal (CREATE STREAM, SELECT, SHOW TABLES, DESCRIBE TABLES per Microsoft/Snowflake guidance)
  • Validate server entry format and connection parameters
  • Validate warehouse existence and effective permissions for principal
  • Identify which tables are regular/native and eligible
  • For Iceberg, confirm object store (e.g. S3) permissions up front: the mirroring principal needs read access to the underlying bucket where Iceberg data files are stored, not just Snowflake-level privileges

After enabling mirroring:

  • Monitor table-level replication status and last refresh
  • Distinguish OneLake landing success from SQL endpoint metadata availability
  • Add smoke tests in both Spark and SQL endpoint
  • Track Snowflake compute impact during replication windows

When troubleshooting:

  • If connection/auth errors: recreate connection cleanly, verify identifiers and grants
  • If missing columns in endpoint: check SQL endpoint limitations and data type mappings, then curate/cast
  • If schema changes not reflected: trigger/verify data change propagation and refresh metadata
  • If large-table updates are massive: consider stop/start strategy per documented performance guidance
  • If newly added tables don't appear in SQL endpoint: wait for initial replication to complete, validate data landed in OneLake/Spark first, then trigger endpoint metadata refresh — endpoint availability is near-real-time, not instant

Common misconceptions I had (now corrected)

Misconception 1: "Mirroring success means SQL endpoint is guaranteed complete"

Not always true. Replication and endpoint representation are related but not identical concerns.

Misconception 2: "All Snowflake table variants mirror the same way"

No. Supported categories are explicitly constrained; design with documented limitations first.

Misconception 3: "If it says queried live, Fabric is directly querying Snowflake"

Operationally, Fabric mirroring is a replication model into OneLake with Fabric-side query consumption.

Final takeaways

  • Fabric mirroring for Snowflake Iceberg works well when you separate replication health from SQL endpoint behavior.
  • SQL Analytics Endpoint issues do not always mean replication failure.
  • Connection hygiene (especially warehouse + auth correctness) prevents a surprising number of incidents.
  • For production, design a compatibility layer for endpoint consumption instead of relying on raw source type fidelity.
  • This was genuinely painful to implement end-to-end, but the pain forced a better operating model and a sharper troubleshooting discipline.

This is one of those integrations where knowing the documented constraints upfront saves far more time than reactive firefighting.

Neutrality and scope disclosure

  • This post reflects my hands-on implementation experience and support-ticket learnings.
  • It is not sponsored content and not intended as a blanket endorsement of any product.
  • Platform behavior, latency, and cost outcomes can vary by workload shape, table volume, schema patterns, and operating model.
  • The intent is practical: help others avoid common mistakes by combining official documentation with real incident patterns.

Sources (official)

Top comments (0)