DEV Community

Mintu Ghosh
Mintu Ghosh

Posted on • Originally published at linkedin.com

Deploying Microsoft Fabric items from Azure DevOps with a service principal

#microsoftfabric #azuredevops #cicd #python

Microsoft Fabric workspaces are edited in a browser. That works until you have
more than one environment and need changes to move between them with a review
step and a way back.

This is the smallest thing that does that: one Python script, one Azure DevOps
pipeline, three branches, three workspaces. Runs as a service principal, so no
human session is involved.

Full code: [https://github.com/vedaforge-team/fabric-cicd-reference/tree/v1.0.0].
This post covers the parts worth understanding
before you copy it.

The shape 

Development workspace  --(Fabric Git integration)-->  Azure DevOps repo
                                                            |
                                        develop  ->  validate against DEV
                                        release  ->  deploy to UAT
                                        main     ->  deploy to PROD
Enter fullscreen mode Exit fullscreen mode

Only the Development workspace is Git-connected. UAT and PROD are reached over
the Fabric REST API by the pipeline.

Connect UAT to Git as well and that environment has two sources of truth: the
repository, and whoever last edited the workspace directly. Both will look
current. Nothing errors.

The script is a thin wrapper

Everything environment-specific arrives as an environment variable. The pipeline
supplies them from variable groups.

def build_workspace() -> FabricWorkspace:
    credential = ClientSecretCredential(
        tenant_id=read_env("TENANT_ID"),
        client_id=read_env("CLIENT_ID"),
        client_secret=read_env("CLIENT_SECRET"),
    )
    return FabricWorkspace(
        workspace_id=read_env("TARGET_WORKSPACE_ID"),
        environment=read_env("TARGET_ENVIRONMENT_NAME"),
        repository_directory=repository_directory,
        item_type_in_scope=read_csv("ITEM_TYPES_IN_SCOPE", DEFAULT_ITEM_TYPES),
        token_credential=credential,
        exclude_paths=read_csv("EXCLUDE_FOLDERS"),
    )
Enter fullscreen mode Exit fullscreen mode

Twelve variables in total, covering authentication, the target workspace, the
source directory, and switches for validation and cleanup.

Validate-only is not a no-op

The develop branch runs the same script with VALIDATE_ONLY=true. Ordering
matters more than it looks:

workspace = build_workspace()          # authenticates, resolves, parses

if read_bool("VALIDATE_ONLY"):
    print("Validation-only mode. No workspace changes applied.")
    return

if read_bool("REMOVE_ORPHANS", True):
    unpublish_all_orphan_items(workspace)
publish_all_items(workspace)
Enter fullscreen mode Exit fullscreen mode

Constructing FabricWorkspace is what authenticates the service principal,
resolves the target workspace and parses every item in the repository. It is the
validation. Return before it and every develop build goes green having checked
nothing.

Worth a test that asserts validation actually validates, because a refactor that
moves that line will not otherwise fail anything.

Branch gating

Three stages, each gated only by the branch. No dependsOn between them.

- stage: Deploy_UAT
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/release'))
  variables:
    - group: VG-Fabric-UAT
Enter fullscreen mode Exit fullscreen mode

There is no automatic DEV to UAT to PROD chain. Promotion happens when someone
merges a pull request, which is where review happens. A commit never walks itself
into production.

Warehouse is excluded on purpose

Sixteen item types are in scope. Warehouse is not:

DEFAULT_ITEM_TYPES = [
    "DataPipeline", "Lakehouse", "Notebook", "SemanticModel",
    # "Warehouse" is intentionally excluded. Warehouse schema deployment must
    # be handled separately to avoid schema reset risk during publish.
    "Environment", "Eventhouse", "Eventstream", "KQLDatabase",
    ...
]
Enter fullscreen mode Exit fullscreen mode

Publishing a warehouse through this path can reset its schema. An automation
that handles most cases and silently corrupts the rest is worse than one that
refuses the rest.

The part that actually breaks

The code above is not where deployments fail. They fail on prerequisites.

I ended up writing them down and there are 43, across six categories: Fabric
environment (6), identity and security (10), Azure DevOps configuration (10),
deployment configuration (7), validation (4), post-deployment verification (6).

Identity is the largest, and three of those ten are the same check repeated per
environment: is the service principal actually a member of this workspace. It
needs to be added to DEV, UAT and PROD separately. Nothing in the repository
indicates when it is missing.

That produces a misleading failure mode. A missing permission surfaces as an
error thrown by the deployment step, which is the last thing in the chain and
the first thing you look at. The error is real, the deployment did fail, and the
code is fine.

The checklist is in the repo at docs/checklists/DEPLOYMENT_READINESS_CHECKLIST.md.
Of everything this produced, it is the part I would copy first.

Environment template 

TENANT_ID=
CLIENT_ID=
CLIENT_SECRET=
TARGET_WORKSPACE_ID=
TARGET_ENVIRONMENT_NAME=DEV
REPOSITORY_DIRECTORY=./.
VALIDATE_ONLY=true
REMOVE_ORPHANS=true
ENABLE_SHORTCUT_PUBLISH=true
LOG_LEVEL=INFO
Enter fullscreen mode Exit fullscreen mode

Keep VALIDATE_ONLY=true until you are sure the target is right.

Known limitations

  • No approval gates. A merge to main deploys production unattended.
  • No parameter.yml, so items referencing a workspace-specific resource by GUID still point at the source environment after deployment.
  • One solution per repository. The layout and variable naming assume it.

Stack: fabric-cicd 1.2.0, Python 3.12, ubuntu-latest.

Full walkthrough: [https://www.linkedin.com/pulse/building-enterprise-microsoft-fabric-cicd-practical-guide-mintu-ghosh-f3dbf/]

Code: [https://github.com/vedaforge-team/fabric-cicd-reference/tree/v1.0.0].

Next post covers what happens when a second team needs the same repository, and
why "just add path filters" does not work.

Top comments (0)