DEV Community: CostFabric

Dataflow Gen2 Eating Your Microsoft Fabric Capacity? Why Default Settings Burn 60-80% Extra CUs

CostFabric — Tue, 27 Oct 2026 12:44:38 +0000

Originally published at costfabric.com.

If you open the Microsoft Fabric Capacity Metrics app on almost any tenant that recently migrated from Power BI Service or Azure Data Factory, one item kind usually dominates the background CU table: Dataflow Gen2.

It is common to see a single Dataflow Gen2 refresh that moves a few hundred thousand rows from SQL Server or an ERP into a Fabric Lakehouse consume more Capacity Units (CUs) than all of the company's interactive Power BI reports combined. Because background operations in Fabric are smoothed across 24 hours, a handful of frequent Dataflow Gen2 runs will quietly build a 40% to 70% baseline CU floor that never goes away, eventually pushing daytime users into CapacityLimitExceeded throttling.

Why does Dataflow Gen2 consume so many CUs for relatively small datasets, and how do you quantify how much of your capacity budget it is actually burning?

Why default Dataflow Gen2 runs are so expensive

When an engineer creates a new Dataflow Gen2 in the Fabric UI and selects a Lakehouse or Warehouse destination, Fabric enables a series of heavy compute steps by default that most teams do not realize are running:

Default Lakehouse Staging (Enable staging): By default, every query in a Dataflow Gen2 stages its output into an internal staging Lakehouse and SQL analytics endpoint before writing to your final destination. That means a simple transformation ends up reading from the source, spinning up Mashup compute, writing to OneLake staging storage, invoking the staging SQL engine, and then copying the data a second time into your target table.
Non-folding Power Query transformations: If a Power Query step breaks query folding (for example, merging across two connectors, applying custom M functions, or filtering after a type conversion that the source driver cannot translate), the Mashup container pulls the entire raw dataset into memory and processes it row-by-row at high CU rates.
High-frequency full refreshes: Because Dataflow Gen1 on Power BI Premium used to feel "free" until CPU maxed out, many teams carried over 30-minute or hourly full-table refreshes into Dataflow Gen2. In Fabric, every one of those runs adds CU-seconds to your 24-hour background smoothing ledger.

The compounding effect on your F-SKU bill

Because Dataflow Gen2 is classified as a background operation, Fabric spreads its CU consumption over the next 24 hours.

That sounds helpful when you only run one job a day. But when you run a set of Dataflow Gen2 refreshes every hour between 7:00 AM and 7:00 PM, their 24-hour smoothing tails stack on top of each other. By mid-afternoon, your capacity is still paying off the smoothed CU debt from the 7:00 AM, 8:00 AM, 9:00 AM, and 10:00 AM runs simultaneously.

When teams see their F16 or F32 sitting at 85% baseline utilization all afternoon, they often assume their business has outgrown the capacity and upgrade to an F64. In reality, they are paying an extra $2,500 to $4,000 per month just to run redundant staging writes in Power Query Online.

Step 1: Measure how much of your smoothing floor comes from background bloat

Before you start rewriting pipelines or upgrading your SKU, check your last 14 days of Capacity Metrics to see what your background smoothing floor is actually costing you in dollars per year:

Export your capacity utilization timeline from the Fabric Capacity Metrics app.
Compare your Average 24h background CU % against your Average interactive CU %.
Check how many hours per day your capacity would sit below 15% CU if that artificial background floor were removed.

You can run this check offline in a couple of minutes with our zero-macro Excel symptom checker or the open-source Python CSV summary script at https://costfabric.com/tools/.

Once you know how many dollars per year are going into background smoothing overhead, the next step is auditing which specific Dataflow Gen2 queries should have staging disabled, which loads belong in Data Factory Copy Activity or Spark notebooks, and how far you can safely step down your F-SKU afterward. If you want a concrete, artifact-level plan for your tenant, request a free 48-hour capacity readout at https://costfabric.com/check/.

Power BI Premium P1 to Microsoft Fabric F64 Migration: Why Most Teams Are Over-Provisioned

CostFabric — Tue, 27 Oct 2026 12:38:46 +0000

Originally published at costfabric.com.

With Power BI Premium per-capacity (P1, P2, P3) retired in favor of Microsoft Fabric F-SKUs, every licensing table and migration slide deck shows the same one-to-one mapping: P1 (8 v-cores) equals F64 (64 Capacity Units).

Because F64 is also the threshold where free Power BI viewer access kicks in (so report viewers do not each need a paid Power BI Pro license), procurement and BI teams almost always default to migrating their old P1 straight into a 1-year reserved F64.

When you inspect the Fabric Capacity Metrics app a month after the migration, however, a huge percentage of those F64 tenants look the same: average CU utilization across the week sits between 14% and 28%, overnight utilization drops below 10%, and the only time the graph crosses 60% is during a poorly staggered 7:00 AM refresh wave.

Here is why the P1 to F64 rule of thumb causes so much overspending, and how to check what your workload actually uses before you renew.

Why an F64 behaves differently than an old P1

On legacy Power BI Premium P1, teams routinely sized for peak morning concurrency because CPU spikes from unoptimized DAX or overlapping dataset refreshes could degrade interactive report rendering right away.

Microsoft Fabric capacities (F2 through F128+) enforce compute governance differently:

24-hour background smoothing: Scheduled semantic model refreshes, Dataflow Gen2 runs, and Spark jobs are smoothed across a full 24-hour window rather than creating a sharp instantaneous wall that blocks report viewers.
5-minute interactive smoothing: Interactive DAX queries are smoothed across at least 5 minutes, absorbing brief executive dashboard bursts.
Flexible workspace-to-capacity assignment: Unlike a monolithic P1 where every workspace had to live on the same capacity, Azure lets you provision smaller Pay-As-You-Go or reserved F-SKUs (F8, F16, F32) and separate production BI from bursty development or ETL workloads.

The Pro-license math trap

The most common defense for keeping an under-utilized F64 (~$5,008/month on a 1-year reservation, or ~$8,410/month Pay-As-You-Go) is that dropping below F64 requires Power BI Pro licenses for report viewers.

Do the arithmetic on your actual viewer headcount before accepting that assumption:

Dropping from a reserved F64 (~$5,008/month) to a reserved F32 (~$2,504/month) saves roughly $2,504 per month ($30,048 per year) in capacity spend.
Even at ~$10 to $14 per user per month for Power BI Pro, $2,504 per month covers roughly 180 to 250 paid viewer seats.
More importantly, in many Microsoft 365 Enterprise tenants, a large share of corporate users already hold Microsoft 365 E5 licenses, which include Power BI Pro at zero incremental cost.

If your organization either has fewer than 200 Pro-less viewers or already runs on E5, paying an extra $30,000 to $50,000 per year for an F64 that sits 75% empty is pure waste.

Where you actually have to be careful when right-sizing

Dropping from F64 to F32 or F16 is not just a matter of clicking "Scale" in the Azure portal on a Friday afternoon. While CU throughput scales linearly, per-artifact guardrails also step down with the SKU:

Max semantic model memory: An F64 supports up to 25 GB per model in memory, whereas an F32 caps individual models at 10 GB and F16 caps them at 5 GB.
Direct Lake framing and fallback guardrails: Max rows per table and max memory before a Direct Lake model falls back to DirectQuery decrease on smaller F-SKUs.
Smoothed background floor: If a single workspace has 30 unoptimized Dataflow Gen2 refreshes running with default staging enabled, that background floor will consume twice the percentage of an F32 as it did on an F64.

Check your 14-day utilization profile first

Before you sign a 12-month reservation on an F64 (or before you scale down blindly and hit an out-of-memory error on Monday morning), measure three symptoms from your last 14 days of Capacity Metrics:

What percentage of your 30-second windows sit below 15% CU?
What is your steady-state 24-hour background smoothing floor?
How many windows actually cross 50% or 100% CU, and which specific operations cause them?

You can run that symptom check offline in 5 minutes using the zero-macro Excel workbook at https://costfabric.com/tools/.

If the checker shows that more than half of your F64 spend is going to idle headroom, and you want to verify which target F-SKU and workspace layout your semantic models can safely step down to without hitting memory or throttling limits, request a free 48-hour capacity readout at https://costfabric.com/check/.

Why Pausing Your Microsoft Fabric Capacity Triggers Surprise Azure Bills (Smoothing & Carryforward)

CostFabric — Tue, 27 Oct 2026 12:37:42 +0000

Originally published at costfabric.com.

A lot of data teams move to Microsoft Fabric on a Pay-As-You-Go F-SKU (F16, F32, or F64) with a simple cost-saving plan: run the capacity during business hours, pause it at 7:00 PM via an Azure Automation runbook or Logic App, and cut the monthly bill in half.

Then the first month's Azure invoice arrives, and the savings are nowhere near 50%. Even worse, you see unexpected billing spikes right at the exact minute the capacity was paused.

If you have seen this happen, you are running into how Microsoft Fabric handles 24-hour background smoothing and carryforward settlement on pause.

How Fabric smoothing actually works under the hood

In Microsoft Fabric, your capacity does not bill you per query. It gives you a fixed pool of Capacity Units (CUs) every second:

An F16 gives you 16 CU-seconds per second (57,600 CU-seconds per hour).
An F32 gives you 32 CU-seconds per second (115,200 CU-seconds per hour).
An F64 gives you 64 CU-seconds per second (230,400 CU-seconds per hour).

To keep short spikes from immediately throttling your users, Fabric smooths compute usage over time. Interactive operations (like opening a Power BI report or running a DAX query) are smoothed across a minimum of 5 minutes. Background operations (like scheduled semantic model refreshes, Dataflow Gen2 runs, Spark notebooks, and Data Factory pipelines) are smoothed across a full 24-hour window.

When a heavy nightly refresh or Dataflow Gen2 batch consumes more CUs than your SKU provides in that window, Fabric borrows capacity from future windows over the next 24 hours. That borrowed usage is called carryforward.

What happens when you pause a capacity with active carryforward

As long as your capacity stays running 24/7, that 24-hour smoothing window works in your favor. A massive 6:00 AM refresh spreads its CU footprint gently across the rest of the day and night while the capacity sits mostly idle.

When you pause a Pay-As-You-Go capacity in Azure, however, the 24-hour clock stops immediately. Microsoft does not forgive whatever smoothed CU debt was still sitting in your future carryforward queue. Instead, when the capacity transitions to Paused, Azure settles all remaining accumulated carryforward and smoothed CU usage onto your bill at once.

Consider a concrete example on an F32 capacity (32 CUs):

At 4:30 PM, a set of heavy Dataflow Gen2 jobs and semantic model refreshes kick off.
Together they consume 460,800 CU-seconds of work. On an F32, that equals 4 full hours of 100% capacity compute, which Fabric normally spreads out at roughly 16.7% CU across the next 24 hours.
At 6:30 PM, your automation script pauses the F32 capacity to save money overnight.
Only 2 hours of that 24-hour smoothing window have elapsed. The remaining unabsorbed CU debt is immediately flushed and billed as an overage charge when the capacity pauses.

Even worse, if teams pack all of their refreshes right before the evening pause window or right after the morning resume window, they end up paying for the daytime active hours plus the settled carryforward from the background jobs. At that point, a paused Pay-As-You-Go capacity can easily cost more than a 1-year Azure Reservation running 24/7 (since a 1-year reservation already discounts the hourly rate by roughly 40.5%, putting the break-even point around 99.2 hours per week).

How to check if your capacity is carrying hidden smoothing debt

Before you build pause/resume automation or change your F-SKU size, look at a 14-day export from the Microsoft Fabric Capacity Metrics app and check three numbers:

Your baseline background CU % floor at 3:00 AM. If nobody is querying reports at 3:00 AM and no jobs are running, yet your capacity sits at 25% to 45% CU utilization, that entire floor is 24-hour smoothed background work from earlier in the day.
The timing of your heaviest background jobs relative to your planned pause hour. Any heavy background job that finishes within 6 to 12 hours before a pause will leave a large chunk of smoothed CUs unsettled when the shutdown hits.
Total weekly active hours plus carryforward drain time. Once you add the hours required to let background debt drain cleanly before pausing, many teams discover their true billable window is 110+ hours per week.

We put together a lightweight, zero-macro Excel workbook and a 60-line Python CSV script that calculate your idle CU hours, your 24-hour background smoothing floor, and your unutilized spend percentage from a standard Capacity Metrics export. You can grab the free offline checker at https://costfabric.com/tools/.

Diagnosing the bleed is step one. Actually fixing it without breaking morning report SLAs requires isolating the specific artifacts driving the smoothing floor, deciding whether to split bursty ETL onto a separate capacity or switch billing models, and scheduling your workload windows so carryforward hits zero before shutdown. If you want an engineer-reviewed breakdown of your own tenant's metrics, you can request a free 48-hour capacity readout at https://costfabric.com/check/.