DEV Community

Mintu Ghosh
Mintu Ghosh

Posted on

Why I left Warehouse out of our Fabric deployment scope

#microsoftfabric #datawarehouse #cicd #devops

title: Why I left Warehouse out of our Fabric deployment scope
published: true
tags: microsoftfabric, datawarehouse, cicd, devops

Our Fabric deployment pipeline handles sixteen item types. Warehouse is not one
of them, and that was deliberate.

DEFAULT_ITEM_TYPES = [
    "DataPipeline",
    "Lakehouse",
    "Notebook",
    "SemanticModel",
    # "Warehouse" is intentionally excluded. Warehouse schema deployment must
    # be handled separately to avoid schema reset risk during publish.
    "Environment",
    "Eventhouse",
    ...
]
Enter fullscreen mode Exit fullscreen mode

The reason

Publishing a warehouse through this path can reset its schema.

Not "might behave unexpectedly". The failure mode is that a deployment intended
to be additive removes structure, and the thing that removes it is the same
routine that successfully deploys the other sixteen types.

The choice that follows

Two options once you know that.

Include it and hope nobody deploys a warehouse without reading the docs. The
pipeline supports everything, and one day someone promotes a change on a Friday
and finds out.

Or exclude it, document why, and handle warehouse deployment as its own problem
with its own tooling.

I took the second. An automation that covers most cases and silently corrupts
the rest is worse than one that covers most cases and refuses the rest. The
refusal is visible. The corruption is not.

Making the exclusion loud

An exclusion is only useful if someone notices it. Three things help:

The comment sits inside the list, not in a doc nobody opens. Anyone reading
the item types sees the gap and the reason in the same glance.

It is in the README under known limitations, next to the other things the
framework does not do.

There is a test. It asserts Warehouse is absent from the default scope:

def test_warehouse_stays_excluded(self):
    """Warehouse publish can reset schema, so it is handled separately."""
    self.assertNotIn("Warehouse", deploy.DEFAULT_ITEM_TYPES)
Enter fullscreen mode Exit fullscreen mode

That test looks silly. It is asserting that a string is missing from a list.

It exists because the natural instinct, when someone hits "my warehouse did not
deploy", is to add the string to the list. The test turns that into a failing
build with an explanation attached, instead of a merged one-line change that
nobody reviews closely.

What I would want next

The honest gap: excluding it does not solve warehouse deployment, it defers it.
Teams that need warehouse promotion still need an answer, and mine is currently
"do it separately, carefully".

That is a real limitation rather than a design choice, and worth saying so.

Full context: [https://www.linkedin.com/pulse/building-enterprise-microsoft-fabric-cicd-practical-guide-mintu-ghosh-f3dbf/]
Code: [https://github.com/vedaforge-team/fabric-cicd-reference/tree/v1.0.0]

Top comments (0)