Mock Service Worker adopts a brand-new request interception algorithm for Node.js.

Preface

One of the most prominent features of Mock Service Worker has always been the way it establishes the boundaries between what is mocked and what is not. Conventional API mocking solutions turn any fetch or axios call into a black hole through the substitution of the entire client with a mocked re-implementation. In contrast, MSW brings Service Workers to the table, which allows request clients to execute in their entirety, bringing more confidence to your tests and prototypes. The reason why it increases confidence is simple—your system underneath your test resembles your actual production system. Allow me to elaborate with a quick example.

Imagine you are writing an abstract test for a bank branch—one with a physical location where people come to open accounts, withdraw funds, and deposit their earnings and investments. Let's say you want to test that your bank can handle deposits. You model a situation where a customer walks in, goes to the cash stand, and hands their money to the teller. The teller then puts the deposit into the bank system, which updates the customer's account balance.

Since this is a test of an interaction, spending actual money to put into accounts isn't the best idea (what an expensive test that would be! 😉). Instead, you decide to mock certain parts of your bank during the test, so they don't actually happen.. You do remember, however, that the point of any test is still to test a system or its part, so you ought to introduce mocking carefully, understanding which parts of the "customer → bank" interactions can be replaced without sacrificing the actual functionality you wish to test—that your bank can handle deposits.

Ideally, it's only that last "cashier → bank system" procedure you should mock. The cashier would receive fake money (mocked), and interact with the bank systems that are already pre-configured (mocked) to respond with a "Okay, we've got the funds!" state. Walking into the bank, finding the cash stand, and speaking to a cashier are all crucial aspects for our customer which should remain authentic and real.

At this point, it should be clear I'm hinting at the HTTP communication with this story, where the customer is a request, and the bank is your request client that processes it. The final part—the "cashier → bank system"—is where you should employ API mocking.. So let's see what happens when you bring a common API mocking library to do the job.

Due to the implementation details of such libraries, what happens is that you end up with your entire bank being replaced. Yes, not just the cashier or a cash stand, the entire bank. Because a conventional interception of request can be roughly represented as follows:

// node_modules/api-mocking-library/index.js
const mockRequestClient = {}

global.requestClient = mockRequestClient

For many tools, the lowest level of operation becomes the request client. In other words, they replace window.fetch, axios, react-query and other clients during your tests, so your code no longer executes them. Basically, your customer no longer walks into your actual bank. Instead, they walk into a fake building, constructed to look and resemble the bank. Since the scope of the mock has grown from a single interaction to the entire building, the surface area where potential issues can happen increases drastically.

This is precisely where Mock Service Worker introduces interception of requests via the Service Worker API. This allows you to keep the bank, the cash stand, and the cashier as real as they are in production. Oh, but there's more! Even the "cashier → bank system" communication becomes real because the worker would intercept the deposit request after it's left the bank. This keeps the mocked surface to a ridiculous minimum.

This has been our algorithm to intercept requests that occur in a browser for years now. Unfortunately, this hasn't been the case for Node.js.

---

Request interception in Node.js

Node.js is an entirely different environment and, as one would expect, it has its own rules and limitations. It's like an alternative universe where you can still have your bank, but all its doors are now made of blue wood, and all the communication is conducted via woolen envelopes... that's just how banks work in this world.

There's simply no intermediate network layer like Service Workers in Node.js. Once requests happen, they happen, and nothing can intercept them past that point. This design choice is why request clients become the lowest point of interception available in a Node.js environment.

Due to this limitation, request interception in Node.js is implemented by stubbing request modules:

// node_modules/api-mocking-library/index.js
const https = require('https')

// My custom function replaces the "bank"
// (the "https" module) and becomes responsible
// for handling any issued requests.
https = function (...args) {
  // ...
}

Modules in Node.js share the same scope, so modifying a standard API like https in one module will affect its references in all other modules.

Module stubbing is a reasonable strategy within the circumstances, and that is also how MSW intercepts requests in Node.js.

Or rather, how it used to, until we tried something... unusual.

---

Brand-new interception algorithm

Starting from the latest version of Mock Service Worker (0.38.0), the library will employ a request interception algorithm that has been rewritten from the ground-up. There's no more module stubbing, no more replacing the entire bank. Instead, we are introducing a way to extend the request modules, allowing them to execute in their entirety and intercept the outgoing requests at the last possible moment (wait, we've heard this somewhere, haven't we?).

We achieve this through module extension.

Technically, both http and https modules are just wrappers around the ClientRequest class. That is actually the request that is being constructed and sent to a server. That is also a lower surface where we could move our logic in order to be even closer to the constructed requests.

Still, we do not wish to tread same water by hijacking the ClientRequest class and forcing it to do our bidding:

// Both "http" and "https" use the same "http.ClientRequest"
// configured accordingly for HTTP and HTTPS
// connections.
const { ClientRequest } = require('http')

// That is NOT what we want!
ClientRequest = class MyClientRequest {
  end(data) {
    // Now, suddenly, resolve with a mocked response!
  }
}

Unfortunately, this is no different than stubbing http/https modules directly.

What we've decided to do is to extend the ClientRequest class, effectively creating a child class that is much like its parent, albeit with a few deviations.

class NodeClientRequest extends ClientRequest {
  end(data) {
    // Respond with a mocked response.
  }
}

This may look similar at first, but, there is a fundamental difference between the choice of replacing or extending the ClientRequest class.

When you replace that class, you're removing it from existence, swapping it with a seemingly compatible class that you've written by yourself. This means you (the API mocking library in this context) become responsible for respecting and handling all the internals of that class. And those are many: establishing the socket connection, writing request/response body, handling headers, etc.

In our previous implementation, that's exactly what we were doing, handling all those moving parts by ourselves.

But what happens when you extend the class is an entirely different story.

Class extension preserves the behavior of the parent class, producing a child class that augments it. So, while we were previously forced to re-implement the response handling logic just to be able to intercept an original response, we can now hook into the ClientRequest.prototype.end method and simply use super() whenever we need to bypass a request.

class NodeRequestClient extends ClientRequest {
  end(data) {
    if (mockedResponse) {
      this.respondWith(mockedResponse)
      return
    }

    // Calling "super.end()" will perform the intercepted request
    // in the identical way it's perform without mocks.
    return super.end(data)
  }
}

The ability to execute the parent class' logic through super() is what allows us to keep the default behavior of request modules intact. We just call out to it whenever it's needed!

It's been a rather challenging implementation, as allowing ClientRequest to execute normally imposes a certain behavior difference when constructing requests.

Let's look at one of these challenges that we've faced during this rewrite.

Handling refused connections

Extending the ClientRequest class means that all requests begin to establish actual HTTP connection. This quickly becomes problematic. When you're prototyping against a non-existing endpoint in Node.js, such connections would be refused! In addition, when you're testing the code that hits actual production servers, connecting to those is not what you want your tests to do.

Presently, we've landed on the decision to always establish the connection, no matter if the request is supposed to be mocked or not.

While this sounds unconventional, the cost of establishing an actual socket connection is rather small. Note that we are still preventing any data from being sent or received through that connection. You can think of it as a HEAD request to your endpoint.

What about the scenarios when the connection fails? What about requesting non-existing endpoints?

It comes down to proper error handling in ClientRequest.

The connection itself happens right away, while first constructing the request instance. At that time, it's impossible for the library to know if there's a request handler created to intercept this particular request. However, it's not too early to handle connection errors.

So what ends up happening is:

1. Request instance attempts to connect to the server;
2. This connection is either established (the server exists) or refused (the server does not exist or couldn't otherwise establish the connection). In either case, no data is being transferred yet.
3. If the connection is refused, MSW catches the respective error and silences it until it knows there are no matching request handlers. Once this is known, the library replays the errors, propagating it to the process.
4. If the connection is established, MSW prevents any data from being written or received until it's certain there are no matching request handlers.

We are convinced that keeping the connection logic is crucial to maintain the integrity of the ClientRequest instances. This does not mean that you must request actual endpoints from now on, or even that you must be connected to the internet while testing. It only means that request instances execute more of its internal logic, including the logic that's responsible for establishing the connection.

---

What should I expect as MSW user?

Expect to update msw in your projects!

npm install msw@latest --save-dev
# or
yarn add msw@latest --save-dev

The new algorithm is an internal change, so there are no breaking changes to the public API or behaviors of MSW in Node.js.

That being said, this is quite a shift from our previous implementation, so we expect issues to be reported occasionally. We highly encourage you to do so whenever you encounter an unexpected behavior or a regression in your Node.js tests! Submit an issue here.

Our motivation behind this change is to reduce the mocked surface in Node.js, bringing you a similar level of confidence that you get when using MSW in a browser.

---

Afterword

I hope you're as excited about these changes as I am! There's a long road for us ahead, but it's a road we wish to spend on making your developer experience not just better, but unmatched. We've got a history of using unconventional approaches to API mocking in the browser, and we're thrilled to expand our vision to Node.js.

Follow MSW on Twitter to stay updated on the awesome features we're working on!

Make sure to check out the @mswjs/interceptors—the low-level interception library that powers Node.js support in MSW:

mswjs / interceptors

Low-level network interception library.

@mswjs/interceptors

Low-level network interception library.

This library supports intercepting the following protocols:

• HTTP (via the http module, XMLHttpRequest, or globalThis.fetch);
• WebSocket (the WebSocket class in Undici and in the browser).

Motivation

While there are a lot of network mocking libraries, they tend to use request interception as an implementation detail, giving you a high-level API that includes request matching, timeouts, recording, and so forth.

This library is a barebones implementation that provides as little abstraction as possible to execute arbitrary logic upon any request. It's primarily designed as an underlying component for high-level API mocking solutions such as Mock Service Worker.

How is this library different?

A traditional API mocking implementation in Node.js looks roughly like this:

import http from 'node:http'
// Store the original request function.
const originalHttpRequest = http.request

// Override the request function entirely.
http.request = function (...args

…

View on GitHub

Thank you!

Comments

[Ryota Murakami]

Great!
I have a vite-react-ts-extended that includes pre setup MSW Service Worker side and Node.js(via Jest) side.

This work definitely gives a benefit for my project!