DEV Community

kasmav
kasmav

Posted on

How a JavaScript Pivot Table Can Handle 10 Million Rows in the Browser

Most JavaScript pivot table examples start with a few hundred JSON records.

That is fine for a demo, but it does not answer the question teams eventually ask when they are building embedded analytics:

What happens when the user drops a multi-gigabyte CSV file into the browser?

I have been working on AeroPivot, a framework-agnostic pivot table web component, and one of the benchmark cases I wanted to validate was a 2.8 GB CSV file with 10 million rows and 33 fields.

This post is not a claim that every browser should process every 10M-row dataset. It is a breakdown of the architecture that made this benchmark practical, what the timing actually means, and what you should measure if you are evaluating large browser-local pivot workloads.

Benchmark page and video:
https://kasmav.com/aeropivot/benchmarks/10-million-row-pivot-table

Public examples and benchmark kit:
https://github.com/kasmav/aeropivot-javascript-pivot-table-examples

The Problem With Row Objects

Most frontend data examples use an array of objects:

const rows = [
  { region: "East", category: "Technology", sales: 1200 },
  { region: "West", category: "Furniture", sales: 850 }
];
Enter fullscreen mode Exit fullscreen mode

That shape is convenient for application code. It is not always ideal for large analytical workloads.

At a few thousand rows, the overhead is not a big deal. At millions of rows, the browser is paying for:

  • repeated object keys
  • object wrappers
  • property lookups
  • garbage collection pressure
  • conversion from row shape into aggregation-friendly structures

A pivot engine usually does not want to think in terms of full row objects. It wants dimensions, measures, dictionaries, grouping keys, aggregate state, and result windows.

That is why the benchmark path uses columnar ingestion.

Columnar Ingestion

Instead of passing 10 million row objects into the component, the application can pass separate columns:

const rowCount = 10_000_000;

const columns = {
  Region: new Array(rowCount),
  Segment: new Array(rowCount),
  Category: new Array(rowCount),
  Sales: new Float64Array(rowCount),
  Quantity: new Float64Array(rowCount)
};
Enter fullscreen mode Exit fullscreen mode

String dimensions can live in arrays. Numeric measures can use typed arrays:

columns.Sales = new Float64Array(rowCount);
columns.Quantity = new Float64Array(rowCount);
Enter fullscreen mode Exit fullscreen mode

Then the pivot table receives the columns directly:

const pivot = document.querySelector("pivot-table");

pivot.clearData();
pivot.fields = [
  { id: "Region", name: "Region", type: "string" },
  { id: "Segment", name: "Segment", type: "string" },
  { id: "Category", name: "Category", type: "string" },
  { id: "Sales", name: "Sales", type: "number" },
  { id: "Quantity", name: "Quantity", type: "number" }
];

pivot.appendColumnarData(columns);

pivot.config = {
  mode: "local",
  rows: [{ id: "Region", name: "Region", type: "string" }],
  columns: [{ id: "Segment", name: "Segment", type: "string" }],
  values: [{ id: "Sales", name: "Sales", type: "number", summary: "Sum" }],
  filters: [],
  sorting: [{ id: "Region", desc: false }]
};

pivot.completeStreaming();
Enter fullscreen mode Exit fullscreen mode

The important part is not the exact API. The important part is the memory shape.

For large pivot workloads, avoiding unnecessary row-object allocation can matter as much as the aggregation algorithm itself.

Separate Ingestion Time From Interaction Time

A common mistake in performance discussions is mixing several different timings together.

For a large browser-local pivot table, you should measure at least two phases separately:

  1. Initial ingestion
    The browser reads, parses, converts, and stores the dataset in the pivot engine.

  2. Pivot recalculation after ingestion
    The data is already resident in memory, and the user changes rows, columns, values, filters, or sorting.

In the AeroPivot 10M-row benchmark, the initial ingestion of the 2.8 GB CSV took about 2 minutes on a Windows machine with 24 GB RAM. After the data was resident in memory, pivot recalculation for the tested layout completed in about 2 to 4 seconds.

Those numbers mean different things.

The 2-minute ingestion time is about loading and structuring a multi-gigabyte file. The 2 to 4 second recalculation time is about interactive analysis after the engine has the data in the right shape.

For many internal analytics workflows, that distinction matters. A user may be willing to wait for an initial import if subsequent pivot interactions are fast.

Keep Heavy Work Off the Main Thread

If a pivot table blocks the main JavaScript thread while processing millions of rows, the rest of the app suffers.

The browser becomes unresponsive. React or Vue cannot update normally. The user cannot interact with surrounding UI. The page feels broken even if the calculation eventually finishes.

For large local workloads, Web Workers are not optional polish. They are part of the architecture.

In this benchmark path, heavy ingestion and aggregation work happens away from normal UI rendering. The component UI can remain responsive while the worker-owned engine builds the structures it needs.

This is also why the benchmark page is careful to describe the workload as browser-local local mode, not as a universal replacement for server-side analytics.

Rendering Is a Separate Problem

Even if aggregation is fast, rendering can still ruin the experience.

You do not want to render millions of source rows. You usually do not even want to render every possible pivot result cell at once.

A pivot UI needs virtualization:

  • render visible rows and columns
  • keep headers stable while scrolling
  • avoid diffing huge DOM trees through the app framework
  • separate pivot state from framework render cycles where possible

This is one reason a pivot-first component can behave differently from a general data grid with pivot features bolted on. A grid often begins with cells, rows, editing, and table rendering. A pivot-first component begins with dimensions, measures, aggregate state, drill paths, totals, and result windows.

Both approaches can be valid. They optimize for different jobs.

When Browser-Local Pivot Tables Make Sense

Browser-local analysis is a good fit when:

  • the dataset can fit within practical browser memory limits
  • the data is a bounded extract, not an endlessly growing warehouse table
  • users need fast repeated interactions after load
  • the data can safely live on the user's machine
  • the workflow benefits from avoiding server round trips for every filter or layout change

Examples:

  • local CSV or Parquet exploration
  • internal finance or operations reports
  • embedded SaaS analytics for bounded customer exports
  • ad hoc analysis tools where users bring their own file

When You Should Use Server-Side Analytics Instead

Browser-local processing is not the answer for every dataset.

Use server-side analytics when:

  • the dataset is too large for browser memory
  • access must be controlled per tenant or per role
  • the source data is sensitive and should not be downloaded
  • joins or transformations are better handled near the database
  • multiple users need governed access to the same data

In those cases, the pivot UI can still live in the browser, but aggregate queries should be handled by a backend service or an analytical database.

For AeroPivot, that path is DuckDB server mode, where the browser requests schema, members, aggregate windows, and drill-through pages instead of downloading the full source dataset.

Benchmark Caveats

A 10M-row benchmark is useful only if you understand the shape of the data.

The following can change results dramatically:

  • field count
  • string cardinality
  • number of measures
  • selected row and column dimensions
  • filter complexity
  • browser memory pressure
  • machine RAM and CPU
  • CSV parsing cost
  • whether numeric measures use typed arrays

A 10M-row sales dataset with repeated regions, categories, and segments is very different from a 10M-row event log where every user ID, session ID, and timestamp is nearly unique.

When evaluating any JavaScript pivot table, test your real data shape.

What I Would Measure

If you are comparing pivot table components, I would record:

  • file size
  • row count
  • field count
  • cardinality of each dimension
  • number of numeric measures
  • browser and operating system
  • available RAM
  • ingestion time
  • first pivot render time
  • recalculation time after changing layout
  • filter application time
  • export time if exports matter
  • whether the UI stays responsive during load

The last point is easy to underestimate. A benchmark that finishes quickly but freezes the whole app may still be a poor user experience.

Why This Is Packaged as a Web Component

AeroPivot is distributed as a native web component because the pivot UI should not be locked to a single frontend framework.

The same custom element can be used from:

  • React
  • Vue
  • Angular
  • Svelte
  • Next.js
  • plain JavaScript

For framework users, the main rule is to treat the pivot table as a native DOM element for large runtime data:

const pivotRef = useRef<HTMLElement>(null);

useEffect(() => {
  const pivot = pivotRef.current as any;
  if (!pivot) return;

  pivot.appendColumnarData(columns);
  pivot.config = config;
  pivot.completeStreaming();
}, []);
Enter fullscreen mode Exit fullscreen mode

Do not try to push millions of values through framework props or template bindings. Use the framework to own your application shell, and pass large analytical data directly to the component instance.

Closing Thought

The browser is not a data warehouse.

But for the right kind of workload, especially bounded analytical extracts, it can be a surprisingly capable local analytics runtime.

The key is to avoid treating large pivot data like normal UI state. Use columnar ingestion, typed numeric arrays, worker-owned processing, and virtualized rendering. Then measure ingestion and interaction separately.

That is the architectural idea behind this benchmark.

Benchmark page:
https://kasmav.com/aeropivot/benchmarks/10-million-row-pivot-table

GitHub examples and benchmark kit:
https://github.com/kasmav/aeropivot-javascript-pivot-table-examples

AeroPivot product page:
https://kasmav.com/aeropivot

Top comments (0)