DEV Community

CostFabric
CostFabric

Posted on Originally published at costfabric.com

Dataflow Gen2 Eating Your Microsoft Fabric Capacity? Why Default Settings Burn 60-80% Extra CUs

Originally published at costfabric.com.

If you open the Microsoft Fabric Capacity Metrics app on almost any tenant that recently migrated from Power BI Service or Azure Data Factory, one item kind usually dominates the background CU table: Dataflow Gen2.

It is common to see a single Dataflow Gen2 refresh that moves a few hundred thousand rows from SQL Server or an ERP into a Fabric Lakehouse consume more Capacity Units (CUs) than all of the company's interactive Power BI reports combined. Because background operations in Fabric are smoothed across 24 hours, a handful of frequent Dataflow Gen2 runs will quietly build a 40% to 70% baseline CU floor that never goes away, eventually pushing daytime users into CapacityLimitExceeded throttling.

Why does Dataflow Gen2 consume so many CUs for relatively small datasets, and how do you quantify how much of your capacity budget it is actually burning?

Why default Dataflow Gen2 runs are so expensive

When an engineer creates a new Dataflow Gen2 in the Fabric UI and selects a Lakehouse or Warehouse destination, Fabric enables a series of heavy compute steps by default that most teams do not realize are running:

  1. Default Lakehouse Staging (Enable staging): By default, every query in a Dataflow Gen2 stages its output into an internal staging Lakehouse and SQL analytics endpoint before writing to your final destination. That means a simple transformation ends up reading from the source, spinning up Mashup compute, writing to OneLake staging storage, invoking the staging SQL engine, and then copying the data a second time into your target table.
  2. Non-folding Power Query transformations: If a Power Query step breaks query folding (for example, merging across two connectors, applying custom M functions, or filtering after a type conversion that the source driver cannot translate), the Mashup container pulls the entire raw dataset into memory and processes it row-by-row at high CU rates.
  3. High-frequency full refreshes: Because Dataflow Gen1 on Power BI Premium used to feel "free" until CPU maxed out, many teams carried over 30-minute or hourly full-table refreshes into Dataflow Gen2. In Fabric, every one of those runs adds CU-seconds to your 24-hour background smoothing ledger.

The compounding effect on your F-SKU bill

Because Dataflow Gen2 is classified as a background operation, Fabric spreads its CU consumption over the next 24 hours.

That sounds helpful when you only run one job a day. But when you run a set of Dataflow Gen2 refreshes every hour between 7:00 AM and 7:00 PM, their 24-hour smoothing tails stack on top of each other. By mid-afternoon, your capacity is still paying off the smoothed CU debt from the 7:00 AM, 8:00 AM, 9:00 AM, and 10:00 AM runs simultaneously.

When teams see their F16 or F32 sitting at 85% baseline utilization all afternoon, they often assume their business has outgrown the capacity and upgrade to an F64. In reality, they are paying an extra $2,500 to $4,000 per month just to run redundant staging writes in Power Query Online.

Step 1: Measure how much of your smoothing floor comes from background bloat

Before you start rewriting pipelines or upgrading your SKU, check your last 14 days of Capacity Metrics to see what your background smoothing floor is actually costing you in dollars per year:

  1. Export your capacity utilization timeline from the Fabric Capacity Metrics app.
  2. Compare your Average 24h background CU % against your Average interactive CU %.
  3. Check how many hours per day your capacity would sit below 15% CU if that artificial background floor were removed.

You can run this check offline in a couple of minutes with our zero-macro Excel symptom checker or the open-source Python CSV summary script at https://costfabric.com/tools/.

Once you know how many dollars per year are going into background smoothing overhead, the next step is auditing which specific Dataflow Gen2 queries should have staging disabled, which loads belong in Data Factory Copy Activity or Spark notebooks, and how far you can safely step down your F-SKU afterward. If you want a concrete, artifact-level plan for your tenant, request a free 48-hour capacity readout at https://costfabric.com/check/.

Top comments (0)