DEV Community: Bryan English

Node.js Heap Dumps in 2021

Bryan English — Mon, 18 Jan 2021 18:37:03 +0000

In diagnosing memory leaks, one of the most useful tools in a developer's aresenal is the heap dump, or heap snapshot, which gives us insight into what objects are allocated on the JavaScript, and how many of them.

The Old Way

Traditionally, in Node.js, we've had two options for creating heap dumps.

Using the heapdump module.
Attaching a Chrome DevTools instance and using the Memory tab to create a heap snapshot.

In cases where it's feasible and simple, the second option is usually best, since it requires no additional software, and has a simple point-and-click interface to get the job done.

In production environments, it's often not an option, so users are left using the heapdump module. While this generally works without issue, there is an additional compilation step and a module to install in order to get this done. These are obivously not insurmountable hurdles, but they can get in the way of solving a problem quickly.

The New Way

The good news is that in newer versions of Node.js, you don't need the external module, since heap dump functionality is now part of the core API, as of Node.js v12.

To create a heap snapshot, you can just use v8.getHeapSnapshot(). This returns a readable stream, which you can then pipe to a file, which you can then use in Chrome DevTools.

For example, you can make a function like this that you can call whenever you want to create a heap dump file.

const fs = require('fs');
const v8 = require('v8');

function createHeapSnapshot() {
  const snapshotStream = v8.getHeapSnapshot();
  // It's important that the filename end with `.heapsnapshot`,
  // otherwise Chrome DevTools won't open it.
  const fileName = `${Date.now()}.heapsnapshot`;
  const fileStream = fs.createWriteStream(fileName);
  snapshotStream.pipe(fileStream);
}

You can call this function on a regular basis using setInterval, or you can set a signal handler or some other mechanism to trigger the heap dumps manually.

This new API function is available in all currently supported release lines of Node.js except for v10, where you'll still need the heapdump module for similar functionality.

Feel free to use the snippet above in your own applications whenever trying to diagnose memory leaks in the future. Happy debugging!

Yet another attempt at FFI for Node.js

Bryan English — Mon, 25 May 2020 14:26:07 +0000

(You can skip the long-winded origin story and head straight to the good stuff if you want.)

Earlier this year, I was working on optimizing a data path inside a Node.js library that creates a bunch of data, encodes it to MessagePack, then sends it off to an HTTP server. I thought that maybe we could do some interesting things in native code that would be harder to do in JavaScript, like an optimized MessagePack encoder, and less-costly multithreading. Naturally, calling into native code from Node.js incurs some overhead on its own, so I was exploring some alternatives.

At the same time, I had been reading about io_uring, a new feature in the Linux kernel that allows for certain system calls to be made by passing the arguments through a ring buffer in memory that's shared by the process and the kernel, for extra speed. This reminded me about how some features of Node.js are implemented by sharing a Buffer between the native and JavaScript code, through which data can be passed. This technique is much simpler than what io_uring does, mostly because it's done for a single purpose on a single thread. The clearest example I can think of in the Node.js API that uses this is fs.stat(), in which the results of the uv_fs_stat() call are stored in a Buffer which is then read from the JavaScript side.

The thought progression here was that this technique could be used to call native functions from JavaScipt in userland. For example, we could have a C function like:

uint32_t add(uint32_t a, uint32_t b) {
  return a + b;
}

And then to call it, we could have a shared buffer which would effectively have the following struct inside it:

struct shared_buffer {
  uint32_t returnValue;
  uint32_t a;
  uint32_t b;
};

To call the function form JS, we first assign the values to a and b in our shared buffer. Then, we call the function and then read the value form the struct:

function jsAdd(a, b) {
  const uint32buf = new Uint32Array(3);
  uint32buf[1] = a;
  uint32buf[2] = b;
  // This next bit is hand-wavey. I'll get to that in a bit!
  callNativeFunction(add, uint32buf.buffer);
  return uint32buf[0];
}

In this example, callNativeFunction would retreive the native function, then give it the arguments from the shared buffer, and put the return value back into the shared buffer.

At this point, great! We've got a way of calling native functions that bypasses a lot of the marshalling that happens between JS and native code by just putting data directly into memory from JS, and then reading the return value right out of it.

The detail here is that callNativeFunction is not a trivial thing to do. You need to have a function pointer for the function you're going to call, and know its signature. Fortunately, we can handle all this because we're only creating this native addon for one function. Case closed.

But what about FFI?

FFI (Foreign Function Interface) refers to the ability to call functions in native code (that is, from a low-level language like C or C++) from a higher level language, like JS, Ruby or Python. These languages all support some way of calling functions dynamically, without knowing function signatures at compile time, because there is no compile time. (Okay, that's not technically true with JIT compilers and all, but for these purposes we can consider them non-compiled.)

C/C++ does not have a built-in way of dynamically determining how to call a function, and with what arguments, like JavaScript does. Instead, the complexities of dealing with calling functions, passing them arguments, grabbing their return values, and handling the stack accordingly are all dealt with by the compiler, using techniques specific to the platform. We call these techniques "calling conventions" and it turns out there are tons of them.

In Node.js the typical thing to do is ignore all this and just write a custom wrapper in C or C++ that calls the exact functions we want. While dealing with these things at compile time is the norm, there are ways of handling them at run time. Libraries like libffi and dyncall exist to fill this void. Each of these libraries provides an interface to deliver arguments to functions and extract their return values. They handle the differences between calling conventions on many platforms. These calls can be built up dynamically, even from a higher-level language, as long as you create reasonable interfaces between libffi or dyncall and the higher-level language.

Enter `sbffi`

The shared buffer technique didn't actually pan out for the code I was working on, because it turned out that converting the data into something readable by native code and then into MessagePack was particularly costly. Moving operations to separate threads didn't really help.

That being said, I still think the approach has value, and I'd like more folks to try it and see if it makes sense for their workloads, so I put together an FFI library for Node.js using the shared buffer technique to get and dyncall to call the native functions dynamically. It's called sbffi and you can use it today as a simple way to call your already-compiled native libraries.

Take our add example from above:

// add.c
uint32_t add(uint32_t a, uint32_t b) {
  return a + b;
}

Now assume we've compiled to to a shared library called libadd.so. We can make the add function available to JavaScript with the following:

// add.js
const assert = require('assert');
const { getNativeFunction } = require('sbffi');
const add = getNativeFunction(
  '/path/to/libadd.so', // Full path to the shared library.
  'add', // The function provided by the library.
  'uint32_t', // The return value type.
  ['uint32_t', 'uint32_t'] // The argument types.
);

assert.strictEqual(add(23, 32), 55);

It turns out that while dynamically building up the function calls incurs some noticeable overhead, this approach is relatively quick. Of course, this test is for a very small function that does very little. Your mileage may vary, but it may be worth trying the shared buffer approach, either manually or with sbffi, the next time you need to call into native code from Node.js.

Butts, and the Internet

Bryan English — Wed, 23 Oct 2019 13:44:01 +0000

First... Butts

Let's talk about butts. Specifically horse butts.

There's a story that's been going around the Internet, probably since its inception, about how the dimensions of spaceship parts are indirectly derived from the width of a horse's butt. The short version is something like this:

Roman chariots were typically pulled by two horses. Therefore chariots were about two horse butts wide. When the Romans built stone roads, they put ruts in to keep the carriages aligned. They used the existing widths of ruts as a guideline for the stone ruts, and then carriages throughout Europe were built for those new stone ruts. Fast forward to the industrial age and you find that European railroad builders used the ruts in the stone carriageways as a guide for how wide to build the rails (i.e. the gauge). This meant a whole bunch of trains in Europe being built approximately two horse butts wide. When America started building railroads, European engineers were brought over and used the same measurements. A final fast forward to the space age, and you've got rocket boosters for the Space Shuttle being transported by rail through rail tunnels, and so they need to be skinny enough to fit through those tunnels.

A typical two-horse-butt train tunnel.

There are some obvious problems with this. The United States didn't have a common track gauge until after the Civil War, and it was only chosen because it happened to be the only one used in the North. Even more glaring here is the fact that train tunnels are quite a bit wider than train tracks, and in fact were not an issue they had to design around for the shuttle rocket boosters. Snopes does a great job of tearing this one down.

As it turns out, simple boring happenstance is the main reason these things seem to line up as they do.

CR+LF

Way back in the 1960s, the ISO and ANSI (then called ASA) were in the process of standardizing character sets. Part of any set of printable characters is a way to indicate that text needs to appear on a new line. The two contenders were the CR+LF two-byte combo, versus a single LF on its own. In C-like languages, these are represented as \r\n and \n respectively. The ISO drafts suggested either CR+LF or LF. The ASA draft only used CR+LF. Either way, a two character sequence was supported by both standards in order to produce the new line effect.

But why? Surely one character ought to do it. Indeed, in most use cases today, we only use LF, so what was the need?

This thing is why we have CR+LF

As it turns out, a lot of computing at the time was done using Teletype Model 33 ASR machines as terminals for input and output. These machines required both instructions in order to bring the printer head back to the start of the line, using CR ("carriage return"), and down one line, using LF ("line feed").

We no longer use Teletype machines, and haven't for some time. That hasn't stopped the various twists and turns of history from keeping the CR+LF alive, long after its original technical need had been obsoleted.

When Unix arrived on the scene in the 1970s, it used the LF character alone to denote a new line transitions in text files, taking the shorter option in the ISO specification. Despite this efficiency, later operating systems like MS-DOS and Windows preferred the CR+LF line delimiter, adhering to both standards.

In 1989, the earliest version of the World Wide Web was born, and with it, HTTP. Like any other text format, newlines needed to be represented. From HTTP/0.9 straight through to HTTP/1.1, the CR+LF was used to denote the end of an HTTP message, and in the case of later versions to delimit headers. Part of the reason the two-character form of newlines was used was the differences between operating system text formats. HTTP/2 and HTTP/3 now use a compressed binary header format that does not make use of of the CR+LF to delimit headers, but since only about 41% of websites use HTTP/2, and HTTP/3 isn't standardized yet, you're still likely using CR+LF under the hood in 2019, regardless of which operating system you use.

Much like with horse butts, there's a cool story of how some obsolete technology makes a surprise appearance in modern technology. Unlike the horse butts, we don't need to squint in order to see it right there in the technology we're using. In both cases, it's still a matter of boring happenstance that caused an old weird technical decision to last many years longer than it should.