DEV Community: Shai Karmani

Before You Put a Fabric AI Agent in Production, Steal This Checklist

Shai Karmani — Tue, 19 May 2026 12:09:34 +0000

Originally published at https://shai-kr.github.io/data-ninja-ai-lab/blog/2026-05-17-fabric-ai-agent-production-checklist.html.

A Fabric AI Agent demo can become useful faster than most teams expect.

Connect it to a semantic model. Ask a few business questions. Add context from Eventhouse, a Lakehouse, or a Warehouse. Suddenly the demo feels close to something people could use.

That is exactly where teams need to slow down for one hour.

Not to block the idea. To stop the first working demo from becoming a messy production workload.

This is the checklist I would use before moving a Fabric AI Agent past pilot stage.

1. Give the agent its own identity

A demo can run under a human account. Production should not.

If an agent depends on one person’s access, the operating model is fragile. Permissions change when that person changes role. Ownership becomes unclear. Offboarding becomes risky. Troubleshooting becomes personal instead of operational.

For a production agent, the better pattern is workload identity.

That means the agent has a dedicated service principal, with access that can be granted, reviewed, rotated, and removed without depending on someone’s user account.

This is the first line I would draw between a pilot and something ready for business users.

2. Start with one narrow use case

The easiest way to make an AI agent hard to govern is to connect it to everything.

Start smaller.

A useful production candidate sounds like this:

Explain sales variance from a governed semantic model
Summarize operational events from Eventhouse
Answer inventory questions for a specific operations team
Help finance users understand reconciliation status
Query a curated warehouse table for one business workflow

A weak production candidate sounds like this:

“Let it answer questions about our data.”

That is too broad. It gives the agent no clear boundary and gives the team no clean way to review access.

3. Map the data sources before adding them

For every data source the agent can reach, write down why it needs it.

Not in a 20-page governance document. A short access inventory is enough:

Workspace
Semantic model, Eventhouse, Lakehouse, or Warehouse
Read-only or operational access
Business owner
Approval date
Review date

The point is simple: someone should be able to look at the agent and understand its blast radius.

If nobody can explain what the agent can reach, the agent is not ready.

4. Separate dev, test, and production

Most demos start in one workspace, with one identity, and one person who understands the setup.

That is normal.

Leaving it that way is the problem.

Before production, I would want a clean path across environments:

Development for experimentation
Test for validation
Production for the restricted, supported version

The identities and permissions do not always need to be complicated. They do need to be deliberate.

If dev and production use the same broad access, every experiment becomes a production risk.

5. Confirm the audit path

If the agent gives a bad answer, uses the wrong source, or becomes part of a business process it was not designed for, you need evidence.

Before launch, answer these questions:

Which identity did the agent use?
Which data source was involved?
Who can review activity?
Who investigates issues?
How do we separate an agent issue from a model issue?

This is where a lot of AI work gets uncomfortable. The demo focuses on the answer. Production needs the trail behind the answer.

6. Treat new data sources as a change request

The first agent will not stay still.

Someone will ask to add finance data. Then operations data. Then a shortcut. Then an Eventhouse function. Then a warehouse table.

Some of those requests will be valid.

They should still trigger a review.

Every new data source changes the agent’s scope. That means the identity, permissions, audit path, and owner should be checked again.

The short version

Before a Fabric AI Agent goes live, I would want these six checks done:

Dedicated service principal
Narrow use case
Known data sources
Separated environments
Least-privilege permissions
Clear audit path and owner

If those answers are vague, the agent is still a pilot.

That is not a failure. It just means the platform work is not finished.

The goal is not to slow down AI agents. The goal is to make them safe enough to use with real business data.

Shai Karmani

Let’s connect on LinkedIn

Your Microsoft Fabric Bill Has a OneLake Problem

Shai Karmani — Tue, 19 May 2026 11:37:28 +0000

Originally published at https://shai-kr.github.io/data-ninja-ai-lab/blog/2026-05-13-microsoft-fabric-bill-onelake-problem.html.

Most Fabric cost conversations start too late.

The bill shows up. Someone opens the Capacity Metrics app. A few workspaces look suspicious. Then the team starts asking the questions they should have asked months earlier:

Why is this data still here?

Who owns it?

Is anyone using it?

Can we move it to a cheaper tier?

Can we delete it safely?

That is not a storage problem. That is an architecture problem.

OneLake storage tiers and lifecycle management are interesting because they push Fabric teams toward a more mature operating model. Not “how much data can we land in the lake?” but “what should happen to this data after it stops being hot?”

OneLake changes the storage conversation

OneLake is designed as the single logical data lake for a Fabric tenant. Microsoft describes it as the OneDrive for data: one place for analytics data across the organization, built on top of ADLS Gen2, used by Fabric experiences like lakehouses and warehouses.

That simplicity is useful. It also creates a trap.

When storage feels central and easy, teams can treat it like an infinite landing zone.

Raw files stay forever. Staging tables become permanent. Old extracts sit next to active analytical data. Development leftovers survive because nobody wants to break something. Warehouses and lakehouses grow quietly until cost becomes visible enough to hurt.

OneLake storage is billed by data stored, per GB. It does not consume Fabric CUs, but it is still a real cost line. And OneLake transactions consume existing Fabric capacity.

So the design question is not only “can Fabric store this?”

It is “what is the lifecycle of this data?”

Data has temperature

A lot of BI teams already understand this concept informally.

Some data is hot:

used in daily dashboards
queried by semantic models
refreshed often
tied to executive reporting
needed for operational decisions

Some data is warm:

used for monthly or quarterly analysis
relevant for trend comparison
still valuable, but not constantly queried

Some data is cold:

retained for audit
rarely queried
needed for compliance or reconstruction
expensive to keep in the wrong place forever

And some data should not be stored at all anymore.

The mistake is treating all of these as the same class of data because they happen to live in the same lake.

Storage tiers force a better conversation. Lifecycle management makes that conversation operational.

The value is not only cheaper storage. The value is that someone has to define rules.

Who owns this dataset?

How long does it need to stay hot?

What is the retrieval expectation after it cools down?

What regulation or business process requires retention?

What is the cleanup rule when nobody owns it anymore?

Those are governance questions, not just admin settings.

This is where Fabric FinOps becomes real

FinOps in analytics is often treated as capacity tuning.

Pause this. Scale that. Optimize a query. Move workloads away from peak hours.

All of that matters.

But storage lifecycle is a different layer of cost discipline. It is less about fixing an expensive day and more about preventing an expensive architecture.

A Fabric team should be able to answer a few basic questions for each important workspace:

What data is actively serving reports or models?
What data is kept only for history?
What data is duplicated because a pipeline was easier to build that way?
What data is retained because of policy?
What data has no owner and no clear use?

If the team cannot answer those questions, adding lifecycle rules later will be risky. You cannot safely automate movement or deletion when nobody understands what the data is doing.

That is why I would not start with “turn on tiering everywhere.”

I would start with classification.

A simple workspace review can expose most of the issue:

active analytical data
temporary processing data
historical data
compliance-retained data
orphaned or unknown data

Only after that does tiering become useful.

The practical operating model

For a Fabric-heavy organization, I would handle OneLake lifecycle as an operating model, not a one-time cleanup.

Start with these rules:

Every important dataset needs an owner.

If nobody owns it, nobody can approve tiering, retention, or deletion.

Every workspace needs a storage review rhythm.

Monthly for active production workspaces. Quarterly for slower domains. No review means no cost control.

Temporary data needs an expiry rule.

Staging and intermediate outputs are useful. They should not become permanent by accident.

Historical data needs a retrieval expectation.

If the business expects instant access, that is one decision. If slower access is acceptable, that is another. Cost follows that decision.

Retention needs to be explicit.

“Keep everything forever” is not a retention policy. It is usually a sign that nobody wants to make the decision.

Storage metrics need to be reviewed next to business context.

The Capacity Metrics app can show where storage is growing. It cannot tell you whether that growth is justified. That part still belongs to the team.

What I would not do

I would not treat storage tiers as a magic cost button.

Moving data colder can reduce cost, but it can also create support problems if the team does not understand access patterns. Deleting data can be correct, but only when ownership and retention are clear. Automating lifecycle rules too early can turn a messy lake into a risky lake.

The better sequence is:

Measure first.

Classify second.

Apply lifecycle policies third.

Review continuously.

That is slower than a cleanup script, but it is much safer.

And for most organizations, the expensive part is not the storage itself. It is the lack of decisions around storage.

The main takeaway

OneLake storage tiers are not just a cost feature.

They are a forcing function.

They force Fabric teams to define data temperature, ownership, retention, and cleanup rules. They turn storage from an invisible side effect into an architectural decision.

That is a good thing.

Because Fabric makes it very easy to centralize data.

The next maturity step is making sure the data does not stay hot, expensive, and ownerless forever.

Sources

Shai Karmani

Let’s connect on LinkedIn

Repository Intelligence: The Real Shift Behind Microsoft Fabric + Git

Shai Karmani — Tue, 19 May 2026 11:37:26 +0000

Originally published at https://shai-kr.github.io/data-ninja-ai-lab/blog/2026-05-10-repository-intelligence-fabric-git.html.

Repository Intelligence turns the Fabric repository into a reasoning surface for AI-assisted analytics engineering.

Most teams still treat Microsoft Fabric + Git integration as a source control feature.

That is understandable. The obvious benefits are versioning, pull requests, rollback, branch-based development, and CI/CD. Those are useful. I want them. Any serious analytics engineering team should want them.

But I do not think that is the real shift.

The bigger change is what happens when a Fabric workspace stops being a collection of things inside a UI and starts becoming a machine-readable representation of the analytics platform.

That is a different category of value.

Once the workspace is represented as a repository, the repo is no longer just a backup of the workspace. It becomes a context layer. And once that context layer exists, AI can work against it in a much more useful way.

Not as a chatbot floating above the platform.
Not as a natural language interface on top of one semantic model.
Not as a one-time parser for a PBIX file.

As a set of repository-aware engineering Skills that understand the structure, relationships, naming conventions, business logic, and drift inside the analytics estate.

That is what I mean by Repository Intelligence.

The mistake: stopping at source control

Fabric Git integration already gives teams a very practical foundation. Microsoft describes it as workspace-level integration with Git, where Fabric items are represented in a repository and the workspace structure, including folders, can be preserved. Supported items include semantic models, reports, notebooks, pipelines, lakehouses, warehouses, KQL assets, data agents, and more.

That matters because the repo is not just random exported files. Fabric items have a source format. For example, reports can include files like definition.pbir and report.json. Semantic models can include definition.pbism and a definition folder with TMDL files. Each item folder also includes system metadata such as .platform, with fields like type, display name, description, and logical ID.

That gives us something important: an analyzable surface.

Most teams use that surface for the normal ALM story:

commit changes
review diffs
merge to main
deploy through environments
roll back when needed

Again, all good.

But if that is where the architecture stops, the team has only moved the workspace into Git. It has not made the workspace intelligent.

The more interesting question is this:

What can reason over the repository now that the repository contains enough structure to describe the analytics platform?

The repository becomes the workspace's memory

The useful unit is not one file. It is the connected graph across the workspace.

A Fabric workspace has many kinds of knowledge hiding inside it:

semantic model tables, columns, measures, relationships, roles, and perspectives
report pages, visuals, filters, bookmarks, and dependencies
pipelines, activities, parameters, connections, and schedules
notebooks, Spark logic, lakehouse paths, and transformation patterns
warehouse objects, views, procedures, and SQL logic
data agents, instructions, examples, and configured sources
naming conventions and folder structures
business terms expressed across measures, reports, descriptions, and documentation

In the UI, those things feel separate.

In a repository, they can become connected.

A measure is not just a line of DAX. It belongs to a table. It uses columns. It may repeat business logic from another measure. It may feed visuals across several reports. Those reports may be used by a finance team. The same concept may also appear in a notebook, a pipeline name, a warehouse view, and a glossary entry.

That is the point.

The intelligence does not come from one file. It comes from the graph across files.

A single PBIX parser can tell you what is inside one report. Repository Intelligence should tell you what a change means across the workspace.

That is the difference between file inspection and platform understanding.

What I mean by AI Skills

A Skill is a controlled workflow over repository context, not a generic chatbot.

I am using the word Skill very deliberately here.

A Skill is not a generic chatbot prompt. It is a structured, reusable agent workflow designed to perform a specific engineering task against repository context.

In practical terms, I would expect a Skill to live close to the repo, often as a SKILL.md file or similar definition, with clear instructions, required inputs, safety rules, expected outputs, and examples.

A simple repository Skill might define:

Name: DAX Complexity Reviewer
Purpose: Identify measures that are too complex, duplicated, or risky to maintain.
Inputs: TMDL files, measure definitions, relationships, report dependencies.
Output: Markdown review with risk score, affected reports, and refactoring suggestions.
Allowed actions: Read files, write report, optionally open pull request.
Not allowed: Change business logic without human approval.

This is a different mental model than "ask my data a question."

Microsoft Fabric data agents are very useful for conversational Q&A over governed data sources like lakehouses, warehouses, semantic models, KQL databases, ontologies, and Microsoft Graph. They can generate SQL, DAX, or KQL under the user's permissions.

Repository-aware Skills solve a different problem.

They help engineers understand and improve the analytics platform itself.

The data agent answers business questions.
The repository Skill reviews the system that produces the answers.

Both matter. They are not the same thing.

What Repository Intelligence can actually do

Here are examples that become much more realistic once the Fabric workspace is represented as code.

1. Trace lineage across the workspace

A Skill can inspect semantic models, reports, pipelines, notebooks, lakehouse paths, and warehouse objects to build lineage that is closer to how the platform is actually maintained.

Not just "this table feeds this report."

2. Detect duplicated business logic

Every mature BI estate eventually collects duplicate definitions.

Revenue appears in five measures.
Active customer logic appears in a notebook, a SQL view, and a DAX measure.
A margin calculation changes in one report and not another.

A repository-aware Skill can search across TMDL, SQL, notebooks, pipeline expressions, and documentation to find similar logic and flag drift.

The best version is not just text similarity. It should understand semantic similarity.

Two measures can look different and still mean the same business thing.
Two measures can look similar and mean something different.

That is exactly the kind of review where an AI workflow can help, as long as it has the right context and a human keeps final judgment.

3. Score DAX and model complexity

Complexity is not automatically bad. Some measures are complex because the business is complex.

But teams still need a way to see where maintenance risk is building up.

A DAX complexity Skill could score measures based on signals such as:

length and nesting depth
repeated patterns
iterator usage
dependency chain depth
number of downstream visuals
use of ambiguous naming
duplicated logic across models
missing descriptions

The output should not be "bad measure, rewrite it."

A better output is:

Measure: Net Revenue YoY %
Risk: High
Why: deep dependency chain, repeated filter logic, used by 14 visuals across 3 reports.
Suggested next step: extract base revenue logic into a reusable measure and add a description.

That is actionable.

4. Identify governance drift

Governance problems usually show up slowly.

A workspace starts with clean naming. Then a few urgent reports get built. A pipeline is copied. A measure is created without a description. RLS roles drift. Sensitivity labels are not consistent. A notebook writes to a path nobody recognizes.

By the time someone notices, the workspace already feels messy.

A governance drift Skill can compare the repo against standards:

naming conventions
folder structure
required descriptions
semantic model role patterns
report certification rules
deployment rules
forbidden shortcuts or connection patterns
expected owner metadata

The Skill does not need to be dramatic. It just needs to be consistent.

Every pull request can get a small governance review before the mess becomes normal.

5. Regenerate useful documentation

Most documentation fails because it is detached from the system.

Someone writes it once. The model changes. The pipeline changes. The documentation becomes a museum.

The repository gives us a better option.

Documentation can be generated from the same files engineers are already changing.

A documentation Skill could maintain:

README files for each workspace or domain
semantic model dictionaries
measure catalogs
report inventories
pipeline summaries
data product ownership notes
change summaries for business stakeholders

The important part is not generating pretty prose.

The important part is keeping the documentation close to the source of truth and refreshing it when the source changes.

6. Translate business glossary into implementation

This is where things get interesting.

Most organizations have business terms that live somewhere outside the model. Sometimes in SharePoint. Sometimes in Excel. Sometimes only in someone's head.

A Skill can compare glossary terms against the repository and ask practical questions:

Is this term implemented as a measure?
Is the definition consistent across models?
Are the report labels aligned with the glossary?
Is there a measure description users can trust?
Does the DAX logic match the business definition closely enough to review?

It should not auto-create business logic and pretend it is correct.

But it can scaffold the work:

proposed measure names
draft descriptions
candidate DAX patterns
impacted reports
questions for the business owner

That makes the human review sharper.

7. Open pull requests with context

The natural end state is not just analysis. It is controlled action.

A repository-aware Skill should be able to open a pull request that includes:

the proposed file changes
the reason for the change
the affected Fabric items
the lineage impact
the test or validation notes
the governance checks it performed
questions that still require a human decision

This is where the repo becomes the operating surface for AI-assisted analytics engineering.

Not autonomous chaos.

Controlled, reviewable, auditable changes.

A practical starter architecture

The practical path is small: sync, index, review, assist, then controlled action.

If I were starting this in a real Fabric environment, I would not try to build a giant agent first.

I would start small.

Step 1: Connect the workspace to Git

Get the basics right first. Use Fabric Git integration at the workspace level. Keep folder structure intentional. Make sure the supported items that matter to your team are actually syncing.

Do not skip naming. AI depends heavily on names, descriptions, and structure. Bad naming is not just a human problem anymore. It becomes machine context debt.

Step 2: Build a repository index

Create a lightweight index of the repo:

item folders
item types
logical IDs
display names
semantic model files
report files
pipeline definitions
notebook files
dependency references
descriptions and labels

This does not need to be perfect on day one.

The first version can be a simple JSON index generated on every commit.

Step 3: Create three Skills, not thirty

I would start with three Skills that create obvious value:

Workspace Documentation Skill
Regenerates README files, inventories, and semantic model summaries.
Semantic Model Review Skill
Reviews measures, relationships, descriptions, naming, and obvious DAX risks.
Lineage Impact Skill
Answers "what breaks or changes if this asset changes?" for pull requests.

These are boring in the best way. They help every team.

Step 4: Run Skills in pull requests

Do not start by giving the agent write access to everything.

Start with read-only reviews in pull requests.

Let the Skill comment with findings. Let humans decide. Measure which findings are useful. Tune the Skill instructions. Add examples from real reviews.

Only after that should you allow limited write actions, like regenerating documentation or adding missing descriptions.

Step 5: Keep the human in the loop

Repository Intelligence should reduce blind spots, not remove accountability.

If a Skill suggests a DAX refactor, a person still owns the business meaning.
If a Skill flags governance drift, a person still decides priority.
If a Skill opens a PR, a person still reviews and merges.

That boundary is not a weakness. It is the safety model.

The real asset is context

Models will improve. APIs will change. Copilot features will keep moving. Fabric data agents will get better. The tooling around AI-assisted engineering will keep changing too.

But the durable asset is the context layer.

A clean, machine-readable repository that describes the analytics platform is valuable no matter which model or agent framework sits on top of it.

That is the part teams should pay attention to.

Not because Git is new.

Because Git turns the Fabric workspace into something AI can reason about.

Semantic models. Reports. Pipelines. Notebooks. KPIs. RLS roles. Naming conventions. Business logic. Documentation. All of it becomes part of the same graph.

That graph is where Repository Intelligence starts.

And I think that is the real shift behind Fabric + Git.

Useful Skills to build first

A starter catalog of repository-aware Skills for Fabric teams.

If you want a practical starting backlog, I would build these in this order:

Repo Inventory Skill
Creates a structured map of all Fabric items in the repository.
Lineage Impact Skill
Explains downstream impact for changed tables, columns, measures, notebooks, and pipelines.
DAX Complexity Skill
Scores risky measures and points reviewers to the right places.
Documentation Refresh Skill
Updates READMEs, model dictionaries, and report inventories from the repo.
Governance Drift Skill
Checks naming, descriptions, RLS patterns, owners, and folder standards.
Glossary Alignment Skill
Compares business terms against measures, labels, and descriptions.
Pull Request Context Skill
Summarizes what changed, why it matters, and what reviewers should check.

That is the path I would trust.

Small Skills. Clear boundaries. Real repository context. Human review.

That is how AI-native analytics engineering starts to become practical.

Sources and notes

Microsoft Fabric Git integration overview: https://learn.microsoft.com/en-us/fabric/cicd/git-integration/intro-to-git-integration
Microsoft Fabric Git source code format: https://learn.microsoft.com/en-us/fabric/cicd/git-integration/source-code-format
Microsoft Fabric data agents: https://learn.microsoft.com/en-us/fabric/data-science/concept-data-agent

Shai Karmani

Let’s connect on LinkedIn