1 million readings, 2 GB of RAM: Node.js typed arrays over JSON

A report query that touched a million sensor readings kept killing our Node.js backend on a 2 GB droplet. I replaced the million JSON objects with two typed arrays and one class. The first version never ran out of memory and was 8x slower than JSON.parse. The second was faster than JSON.parse, and memory stayed flat.

I'm Panth, and I lead the software team at Oizom, where we build air-quality monitoring. This is the written version of a four-part screen recording I made while building it. The videos are below if you would rather watch me get it wrong first.

What a data point looks like

A single reading from a device looks roughly like this:

type DataPoint = {
  device: string;
  t: number;                       // epoch seconds
  d: Record<string, number>;       // decimals: co2, o2, pm25, temperature …
  s: Record<string, string>;       // strings: location, firmware, tag …
};

d holds the gas concentrations and other numbers. s holds whatever labels the device or the user attached. Any key can be present or absent on any reading. Thousands of devices report every minute, so a report query can easily touch a million of these.

The backend ran on a DigitalOcean droplet with 2 GB of RAM, and it kept dying with out-of-memory errors on exactly those queries.

Why a million objects is the problem

Three things happen when V8 holds a million of those objects, and none of them show in the code.

Every property access is pointer chasing. points[i].d.co2 is not one memory read. The array holds a reference to the point, which holds a reference to d, which is a hash map that hashes "co2", finds the slot and hands back the number. In C the compiler knows the offset of co2 before the program runs. In JavaScript every value is a reference to somewhere else.

Every object is a hash map. An object with ten keys is buckets plus a hash on every access. A million of them is ten million hashed lookups for one pass over the data.

Every object is garbage. Each {} is an allocation the garbage collector tracks. Create a million per request and the collector works harder than your code. The numbers are boxed doubles spread across the heap, so the working set is far bigger than the data.

So the fix is not to optimize the loop. The fix is to stop creating objects.

Treat the data as an image

Put timestamps on the Y axis and gases on the X axis, and the readings are a grid. Some cells are empty because that device didn't report that gas at that time. That is what an image is: a rectangle of numbers in one contiguous array, addressed by row * width + column.

            co2    o2    pm25   temp
t1        [ 412   20.9   35.2   26.1 ]
t2        [ 415    –     36.0   26.3 ]
t3        [  –    20.8   34.1    –   ]

Instead of an array of objects, the table keeps:

• one Float64Array for all decimal values, times.length * gases.length long
• one Int32Array for string values, holding an index into a dictionary of unique strings, because labels repeat on almost every row and there's no reason to store "Bangalore" a million times
• three small arrays for the axes: timestamps, gas names, label names

A reserved value marks an empty cell. Time 0 is an unused row; an empty string is an unused column.

No object per reading, one allocation for the whole table. Reading co2 at row i is values[i * gases.length + gasIndex.co2]: the offset arithmetic JavaScript normally won't give you.

One class on top

The buffers are ugly to use directly, so everything goes through one class. This is the surface from the first video, and it barely changed later:

class DataPointTable {
  static create(rows: number, gases: number, labels: number): DataPointTable;
  static fromJSON(points: DataPoint[]): DataPointTable;
  static fromBuffer(buf: ArrayBuffer): DataPointTable;

  // axes
  times(order?: 'asc' | 'desc'): Iterable<number>;
  gases(): Iterable<string>;
  labels(): Iterable<string>;
  timeIndex(t: number): number;         // -1 if absent
  gasIndex(name: string): number;
  addTime(t: number): number;           // returns the row index
  addGas(name: string): number;
  removeTime(t: number): void;

  // cells
  get(ti: number, gi: number): number | undefined;
  set(ti: number, gi: number, v: number): void;
  row(ti: number, onlyExisting?: boolean): Iterable<[gas: string, v: number]>;

  // whole table
  toJSON(): DataPoint[];
  toBuffer(): ArrayBuffer;              // for sending to the browser
  compress(): DataPointTable;           // drop unused rows and columns
  stats(): { bytes: number; rows: number; used: number };
}

Nobody else in the codebase has to know there are buffers underneath. A report generator asks for a row and gets [gas, value] pairs. It never sees a Float64Array.

Version one: correct, and 8x slower than JSON

The first version kept rows sorted by timestamp, descending, because queries always want time order. So addTime had to find where the new row belonged, find the nearest free slot on either side, and shift every row in between by one. Adding a gas column was worse: every row moves to make room for the new cell, so the copy runs from the back of the buffer to the front, zeroing new cells as it goes.

I over-allocated to soften it: five spare rows and five spare columns at a time, so a burst of inserts doesn't reallocate each time. When the table is full, allocate a bigger one, copy, carry on.

Every function test passed, toJSON(fromJSON(x)) round-tripped, and compress() reclaimed the spare cells. Then I loaded a million points with thirty columns each. It never ran out of memory, which JSON always did at that size. It was also 8x slower than parsing the JSON.

What was slow

Not the typed arrays. The shifting.

The backend loads rows from the database, maybe runs one or two operations, then sends the data to the browser or feeds it into a report. It almost never inserts a row into the middle of a table. I was paying for ordered insertion on every load and getting nothing for it.

Two smaller things added to it. Finding a column by name was a linear scan of the gas array on every cell write, so a name → column index hash map came back, but only for the axes, which have thirty entries, not a million. And a binary search on the sorted timestamps didn't help, because the search was never the cost.

Version two: stop keeping order

The rewrite dropped the one assumption that hurt. Rows are no longer sorted. The table keeps two free lists instead:

private freeRows: number[];      // indexes of rows with t === 0
private freeCols: number[];      // indexes of columns with name === ''
private rowOf = new Map<number, number>();   // timestamp → row index
private colOf = new Map<string, number>();   // gas name → column index

addTime is now: if the timestamp is known, return its row; otherwise pop a free row and record it in the map. No search, no shift. If the free list is empty, allocate a bigger table, copy once and push the new rows onto the free list. removeTime zeroes the row and pushes its index back. No loop in the hot path.

A query knows how many rows it will return before it reads them, so the table is created at the right size up front, and the grow-and-copy branch almost never runs.

Order comes back only where it is read. times('desc') collects the live timestamps, sorts them once and yields them. One sort per query instead of one shift per insert.

	Memory with 1M points	Load vs JSON.parse
JSON objects	out of memory on 2 GB	baseline
Version one (sorted rows)	never ran out	8x slower
Version two (free lists)	flat	faster

The figure in the video description, roughly a 400% gain, is the load-and-serve path measured against the JSON version on the same box.

Skipping JSON entirely

That is why the class has toBuffer and fromBuffer. If the point is to stop building a million objects, serializing them to JSON for the browser undoes it. So the table serializes itself as its own buffers: a small header with the dimensions and the axis arrays, then the two value buffers as they already sit in memory.

On the server, database rows go straight into the table without a DataPoint[] ever existing. The buffer goes to the browser as is (base64 where it has to cross a text boundary), and the frontend rebuilds the same class from it. fromJSON and toJSON stay for tests and the odd tool; the production path never touches them.

What I'd tell past me

• The win came from removing allocations, not from typed arrays as such. Typed arrays are just the only way JavaScript lets you allocate a million numbers in one go.
• Don't maintain an invariant the read path doesn't need. Sorted rows cost me a full version.
• Keep the buffers behind one class. Every place that touched a Float64Array directly would have been a place the rewrite had to visit.
• Measure against what you're replacing, on the same box. "Never runs out of memory" isn't a win if it's 8x slower.

I had Claude write the function and performance tests for the second version. In the fourth video, the round-trip comparison catches a bug in compress() I wouldn't have found by hand.

Have you hit the point where plain objects stopped scaling in Node.js, and what did you move to?

Related reading

• IDW heatmaps, 5 seconds to 30 ms: the same instinct on a 1 km grid, where the "look at it as an image" idea came from.
• Postgres triggers and our Redis cache at Oizom: the other half of the same backend.
• await on a sync function in Node.js: another thing that isn't free even when it looks free.