Capturing stdout/ stderr in Node.js using Domain module

This weekend I am working on a project that enables developers to test multiple data aggregation scripts in parallel. Implementing this functionality requires that a single API endpoint evaluates multiple user submitted scripts. However, if either script fails, we need to retrieve the logs of the execution too, i.e. we need to capture what was written to stdout.

I have had this requirement before and I have already developed output-interceptor to solve it. It works by overriding process.stdout, e.g.

let output = '';

const originalStdoutWrite = process.stdout.write.bind(process.stdout);

process.stdout.write = (chunk, encoding, callback) => {
  if (typeof chunk === 'string') {
    output += chunk;
  }

  return originalStdoutWrite(chunk, encoding, callback);
};

console.log('foo');
console.log('bar');
console.log('baz');

process.stdout.write = originalStdoutWrite;
console.log('qux');
output;

In the above example, output evaluates to foo\nbar\nbaz\n.
If your application processes all tasks sequentially, then the above is all you need to capture program's output. However, it wouldn't work if there are concurrent operations – logs of multiple operations would be meshed into one blob.

Turns out that we can create an execution context using domain. I admit that I knew of domain module, but never had a practical use case for it: I thought it is primarily used to handle propagation of asynchronous errors. Therefore, the capability to achieve the above was a pleasant surprise.

The trick is to override process.stdout.write and check for process.domain. process.domain is a reference to the current execution domain. If process.domain can be recognised as a domain that we have created with intent to capture the stdout, then we attach the intercepted stdout chunks to that domain, e.g.

const createDomain = require('domain').create;
const originalStdoutWrite = process.stdout.write.bind(process.stdout);
process.stdout.write = (chunk, encoding, callback) => {
  if (
    process.domain &&
    process.domain.outputInterceptor !== undefined &&
    typeof chunk === 'string'
  ) {
    process.domain.outputInterceptor += chunk;
  }
  return originalStdoutWrite(chunk, encoding, callback);
};
const captureStdout = async (routine) => {
  const domain = createDomain();
  domain.outputInterceptor = '';
  await domain.run(() => {
    return routine();
  });
  const output = domain.outputInterceptor;
  domain.outputInterceptor = undefined;
  domain.exit();
  return output;
};

In the above example, captureStdout captures everything that was written to process.stdout while executing routine. If there are multiple routines running concurrently, then their execution domain is used to distinguish their output.

Here is a working demo that you can play with.

If you need this functionality in your program, then consider using output-interceptor: I have since updated output-interceptor to handle asynchronous functions using the same principle as described in this article.

I figured this is worth sharing as it provides an example of creating and maintaining a reference to the execution context beyond handling asynchronous errors.

A notice about “deprecation”

Earlier when I published a variation of this article, several people commented that domain module is deprecated and it should not be used.

Despite the big red banner stating that this module is deprecated – domain is used internally within Node.js a lot and it is not going anywhere anytime soon.

If you read the paragraph following the banner, it states that the module is pending deprecation once a replacement API is finalised. It is likely that async_hooks will eventually provide all functionality provided by domain module and will supersede it. In fact, domain is already implemented using async_hooks behind the scenes and this is unlikely to change – think of domain as a higher level abstraction of async_hooks.