Please, stop playing with proxyquire

Before we started: I am a father of 3 boys, and they often "play". I love to watch how they play, but they are not always doing it in a silent or even safe way. We all were kids, you know how it could be. Kids could not just sit, kids could not just stop, kids could not list listen. Relax, I am just kidding...

...Please, stop playing with proxyquire. There is just a simple and obvious reason for it - it’s time to stop playing games. And to explain the meaning of games here, I should ask you to stop using another library - rewire. Hey kids, it’s no longer fun.

It’s time to stop being a kid and accept responsibility. For what? For the code you own, and for the code you write.

Let’s first make clear why you may use these proxyquire and rewire, and why this “use” is nothing more than just a kidding.

Let’s play

There is a game. A Dependency Mocking game. Sometimes known as Dependency Injection game. Some boxes even labelled as Inversion of Control Game. A quite popular sandbox strategy game, where you are running your module's code in different environments and trying to find conditions to break it.

Dependency mocking is an ability to run your code in a sandboxed environment.

First, let's play in a rewire edition. It's named after Rewire - a magic wand and a source of the endless power. Once you need some control over your code, once you need to change the way it works - use it. It gives you the ability to rewire(yep!) a module, string it, and become a puppeteer.

Is it sounds like fun?

For me - yes. Let's draw an example -

• there is a file we want to test

var fs = require("fs"),
    path = "/somewhere/on/the/disk";

exports.readSomethingFromFileSystem = function(cb) {
    console.log("Reading from file system ...");
    fs.readFile(path, "utf8", cb);
}

• and a test for this file

// test/myModule.test.js
var rewire = require("rewire");

var myModule = rewire("../lib/myModule.js");

// and we could CONTROL IT!!!
myModule.__set__("path", "/dev/null");
myModule.__set__("fs", fsMock);

myModule.readSomethingFromFileSystem(function (err, data) {
    console.log(data); // YOOHOO!!
});

What was that? We just rewired a file! We changed the values of internal variables and make this file testable. We are testing gods, aren't we?

TL;DR - no, we are not. Leaky abstractions and no separation between internal variables and dependencies (which, ok, are variables), are not fun. This game would not end well...

Please, don't get me wrong, but rewire is just a violation of all established patterns, and could be used only by kids, which don’t care about game rules, but just want to play.

Since the very beginning, we are learning how to code, and how to do it "properly" - from language structures to data algorithms and architecture patterns. We are learning what is bad, what is good, and what is right. Like - globals and 1000 lines-long files are bad, SOLID is good, clean code is right. (working and shipped code is even better).

There are many bad things and many good things. And good usually means strict. Strict, boring, sad, compact, easy to understand and reason about, easy to start with, and transfer to another team. Cool and hacky solutions are not something somebody, anybody would say "Thank you" for. (It would be closer to "$%@# you")

"Strict" and "Standard". And, if you are able to use rewire, - your code is nor "strict", nor "standard".

Let me make this situation a bit worse:

• obviously, nothing would work if you used const to declare variables, so you are not able to change their values anymore.
• obviously, nothing would work after babel transformation as long as variable names would be changed. And that's documented limitation.
• there is a babel-rewire-plugin which would save the day, but does it changing anything?

So, according to the README - this library "is useful for writing tests, specifically to mock the dependencies of the module under test.".
But I would say - no. It has nothing with dependency mocking.

I urge you - stop using rewire. Yes - it's a very popular game, and a funny one. But it would not end well. Please stop. Right. Now.

Sinon way

Before jumping to the real fun, let's talk about another library, which is usually used to "mock"(or "stub") dependencies - sinon.

import * as Service from './serviceToMock'
import { someFunctionThatCallsMyOperation } from './controllerThatUsesTheService'
sinon.stub(Service, 'myOperation').return(5)
someFunctionThatCallsMyOperation() // Ends up receiving a 5 as answer

or like

var fs = require('fs');
sinon.stub(fs, 'readFileSync');
fs.readFileSync('/etc/pwd'); 

Is it clear what's happening here? sinon.stub(x,y) is just x[y]=Z – it's an override, a hack applicable only to the exported objects. A way to change something from inside.

This is a wrong way, a dead end. Sinon itself has a better way documented (listen, kid, to what adults are saying), but still many of you are using sinon to mock. Using sinon to mock dependencies is just not right. Just impossible, as long as it has no power upon module internals.

// lets extract to a local variable. There are many reasons to do it
const readFileSync = fs.readFileSync;

// for example this one
import {readFileSync} from 'fs';

// ...

sinon.stub(fs, 'readFileSync');
// ^ has no power this time ^

Every tools has the goal, and also has limitations. sinon.sandbox might mock - environment like timers or server, but has a limited power upon your own code.

There is a simple rule to define is game good, or game is bad - if at least one valid used case could not be solved, requiring you to change the game rules(your code) - then the game is bad.

Additionally, doing something like sinon.stub(fs, 'readFileSync'); is changing fs for all module consumers, not only for the current test or the current subjects under test. For example that's killing avajs test runner ☠️.

No. Changing (and using) globals (and module exports are globals due to the module cache) is not the right way. Hacking local variables is also not an option - they are also globals, just a bit more local.

It's even not a right way to mock something within classes, as long as it could be made only after their construction - techniques like DI, where you might inject all dependencies via constructor call:

• first - might require changing constructor signature just for a testing sake. Definitely does not work for "Some Frameworks"(like React) which has their own opinion on how your classes should look like.
• second - does not play well without classes (in terms of performance and garbage collection).

So, as long as I've mentioned classes...

A Secret game

Some games are shipped in a fancy box. Like ts-mock-imports - just listen to how it sounds - Intuitive mocking for Typescript class imports... By why "classes" are mentioned here? A limitation which should not exists.

// foo.js
export class Foo {
  constructor() {
    throw new Error();
  }
}

// bar.js
export class Bar {
  constructor() {
    const foo = new Foo();
  }
}

// test.js
import { ImportMock } from 'ts-mock-imports';
import { Bar } from './Bar';
import * as fooModule from '../src/foo';

// Throws error
const bar = new Bar();

const mockManager = ImportMock.mockClass(fooModule, 'Foo');

// No longer throws an error
const bar = new Bar();

// Call restore to reset to original imports
mockManager.restore();

Beautiful? But what's underneath? A single line behind a sugar.

// https://github.com/EmandM/ts-mock-imports/blob/master/src/managers/mock-manager.ts#L17
this.module[this.importName] = this.stubClass;

Direct module exports patching. Which does not work with ESM modules or webpack, as long as exports are immutable. Or, at least, expected to be immutable. The same "sinon" way.

And, but the way - this is not "beautiful". The example itself shows how mutable and fragile is the approach. There shall be no way to affect an already created class.

A good way to mock a class - inherit from it, and override endpoints you need.

1. Change Bar. We have to do it, as long as there is no way we can amend class constructor, but we could do whatever we want with class methods.

//bar.js
export class Bar {
-  constructor() {
-    const foo = new Foo();
-  }
+  constructor() {
+    this.createFoo();
+  }
+  // just moved code to a separate function
+  createFoo() {
+    const foo = new Foo();
+  }
}

Then test could be quite simple:

class TestBar extends Bar {
   createFoo() {
     spy.call();
   } 
}

// No longer throws an error
const bar = new TestBar();
expect(spy).to.be.called();

JFYI: we have changed the game rules(the code). We made it not more testable, but more "hookable"(or "patchable").

PS: React-Hot-Loader is working that way - it inherit hot-reloable Components from the Base components, changing all the places it might want, to be a bridge between React and your Code.

But it does not always work - we are able to seam Bar, but not Foo(which is "hard" wired), while we might need to mock Foo, if, for example, it will do something with fs.

In short

In short, all mentioned games above are not dependency mocking, as long as they are working and doing something after target module got required and initialized. It's too late. They work should be done a moment before.

Let me repeat - IT'S TOO LATE!.

Just RTFM. Really - the testing and mocking smells are well defined and known for the last 30 years. Just try to accept - methods listed above are not just anti-patterns(I am not sure what this word means) - they are just falsy ways.

The part about why you should use dependency mocking, not variable hacking has been ended.

Proxyquire

Proxyquire is an artifact of an antique commonjs (nodejs) era. It could be used to change the way how one module requires another. Literally - could change the behaviour of require function.
Then - you could do anything! Replace fs by fake-fs, mock fetch, or replace submodule with your own implementation.

Proxyquire is a million times better. It never touches the module itself, controlling only its external dependencies. It’s like a docker-compose — "Hey nodejs! Could you run this module in a different environment?!"

const myModule = proxyquire.load('./myModule', { // file to load
  'fs': myFakeFS  // dependency to replace
});

myModule === require('./myModule') // with 'fs' replaced by our stub

It's just beautiful - get myModule as-is, but within a different environment, replacing and external module dependency - fs - by that we said.

Let's try to fix the Foo-Bar example above:

const myModule = proxyquire.load('./Bar', { // file to load
  './Foo': myFakeFoo  // dependency to replace
});

// No longer throws an error, without any code changes this time.
const bar = new Bar();

JFYI: we haven't changed the game rules(the code). Dependency mocking might require moving code around, not changing it.

This simple ability solves most of the problems. There is only one constraint - you can mock only module's dependencies, keeping the module itself untouched. As a result - everything you might wanna “mock” or “control” - should be an external dependency. This leads to a more sound code separation between files - you have split function between files according to their “mockability”, which will come from testability, which will reflect usage. A perfect sandbox!

I mean - you file contains two functions, and you want to mock the first one to test the second one - then the second function depends on the first, and you might consider extracting it as an explicit dependency to another file. To move stuff around.

Even it might require some changes to your code - it does not breaks the game rules, and not making this game a bad game. It just changing the way you reason about it.

To be honest - proxyquire is the etalon for dependency mocking as a concept:

• able to mock dependencies
• but only direct dependencies
• and gives you control upon process, like callThought for partial mocking.

From this prospective - proxyquire is a quite predictable solution, which will enforce good standards, and never let down.

🤷‍♂️ Unfortunately - this is not true. By the fact it will blow up your tests, and would be moooreee predictable than you need.

Blow up?

Yes! Infect your runtime. To the very death.

The key lays in the proxyquire implementation details - once you require some file, which should be replaced, it returns another version of it, the one you asked to return instead of the original one, and this “rewire” initial file. Obviously, that “another version” got cached, and would be returned next time someone else would ask for the same file.

const myTestableFile = proxyquire.load('./myFile', {
   'fs': myMockedFs
});

const fs = require('fs'); // the same myMockedFs :) oh 💩!

Basically, this is called “poisoning”. Obviously, it would crush the rest of your tests. Obviously, there is a command to cure this behaviour - .noPreserveCache, which is (not obviously this time) is disabled by default, so you have to fix your tests manually.

Almost everybody went into this issue with proxyquire. Almost everyone had to add one more line(to fix cache) to the every test. Almost everyone spent hours before, trying to understand this strange behaviour, and why all tests after “that one” are broken, but only when executed in a bulk. It's a :tableflip:, not a fun.

And "obvious" and "catastrophic" problems like this are common.

Too predictable?

The second problem - is how straightforward proxyquire is. By fact - very straightforward. If you asked to replace something - only the exact match of your request would be executed.

(Quick advice here - it shall be exactly the same “filename” you have used in original require)

• If your tests are in another directory - use the name as it is written in the source file.
• If your imports are using absolute paths - use... use the relative path, which will be used to require a real file, after some (Babel?) plugin would translate it.
• If you did a mistake in a file name or a file path - so good luck mate, and happy debugging - no help would be given at all.

// './myFile'
import stuff from 'common/helpers';
....

// './myFile.test.js'
const myTestableFile = proxyquire.load('./myFile', {
  'common/helpers': mock // nope. You have to mock something else
});

It might be a real problem to understand what is your "file" name after babel transpile your imports or some another lib made name resolving a bit more fancy.

It's funny, but all common mocking libraries - proxyquire, mock-require, mockery does not get it right. They all require you to "predict" the file name.

Different modules are mocking in a different way, and in the different time. Majority override require(module.load), and works "before" the cache. Minority utilize require.extensions and live behind the cache wall. There is even one lib, which put your mocks into the cache, and thus has no real runtime.

The part about there is no perfect tool, and you should know limitations of your one has been ended.

Let's change the game rules. Make it more safe.

Game mode: easy

You will be surprised, how easy to fix the game, by adding new game rules:

const myTestableFile = rewiremock(() => require('./myFile'), {
  'common/helpers': mock // 😉 that's all
});

And if that's not enough:

const myTestableFile = rewiremock(() => require('./myFile'), () => {
  rewiremock(() => require('common/helpers')).with(mock) // 😉 that's 100% all
});

The trick is simple - by using require, instead of fileName it's possible to ask nodejs to resolve the right filename for us.

• plus autocomplete
• plus cmd+click (goto)
• plus types, if you have them. Or at least jsdoc.
• plus no issues with Windows, where file path your required is './a/b.js', but the file you required is actually 'a\b.js' - believe me - that's breaks a lot.

You know, comparing to the other libraries - it's like a magic.

rewiremock

Yes, rewiremock is a way to fix the game.

• working for nodejs, webpack and ESM environments.
• has two different APIs to help migrate from proxyquire or mockery.
• support webpack aliases, ts-aliases and any other aliases.
• support isolation(usage of unmocked dependency) and reverse isolation(when mock was not used)

You might notice, that 90% of this article is about how some things are not right. But, even if they are - there is a way to make it better. To make tests less smelly and painful.

You might hear, that dependency mocking is a bad thing. Still - by not using it, or not using it properly we usually going even worse ways.

Easy to mock code is easy to test code. Properly structured, with all things separated as they should, at their own places. Like a playground... before kids code...

And rewiremock would keep the game safe. And funny.

That's the article end. I've pointed on the issues with a common mocking/testing patterns, and gave you a direction to go. The rest is on you.

But if you want to know more?

theKashey / rewiremock

The right way to mock dependencies in Node.js or webpack environment.

                     /$$      /$$ /$$                     /$$      /$$                     /$$
                    | $$  /$ | $$|__/                    | $$$    /$$$                    | $$
  /$$$$$$   /$$$$$$ | $$ /$$$| $$ /$$  /$$$$$$   /$$$$$$ | $$$$  /$$$$  /$$$$$$   /$$$$$$$| $$   /$$
 /$$__  $$ /$$__  $$| $$/$$ $$ $$| $$ /$$__  $$ /$$__  $$| $$ $$/$$ $$ /$$__  $$ /$$_____/| $$  /$$/
| $$  \__/| $$$$$$$$| $$$$_  $$$$| $$| $$  \__/| $$$$$$$$| $$  $$$| $$| $$  \ $$| $$      | $$$$$$/
| $$      | $$_____/| $$$/ \  $$$| $$| $$      | $$_____/| $$\  $ | $$| $$  | $$| $$      | $$_  $$ 
| $$      |  $$$$$$$| $$/   \  $$| $$| $$      |  $$$$$$$| $$ \/  | $$|  $$$$$$/|  $$$$$$$| $$ \  $$
|__/       \_______/|__/     \__/|__/|__/       \_______/|__/     |__/ \______/  \_______/|__/  \__/

Quick start

1. Install

• yarn add --dev rewiremock or npm i --save-dev rewiremock

2. Setup

I would recommend not importing rewiremock directly from tests, but create a rewiremock.js file…

View on GitHub

PS: additional articles about dependency mocking and rewiremock:

• How to mock a dependency, and why
• Unit tests are production code
• Units tests for SkyNet(if written in JS)
• Mocking was a code smell

Comments

[MalikKillian]

I think there's too much focus here on which mocking framework is the least (or most) evil. The actual need for dependency mocking can be reduced by refactoring. You're trying to test this code but it's impure and inherently hard to test:

var fs = require("fs"),
    path = "/somewhere/on/the/disk";

exports.readSomethingFromFileSystem = function(cb) {
    console.log("Reading from file system ...");
    fs.readFile(path, "utf8", cb);
}

The external dependencies (if impure) should be parameterized:

const invokeReadFile = (fs, path) => (cb) => {
    console.log("Reading from file system ...");
    fs.readFile(path, "utf8", cb);
};

exports.readSomethingFromFileSystem = invokeReadFile(fs, path);

Now you have your exported function, working same as before, and you have a "private" function with the dependencies parameterized. Using a framework like rewire you can test invokeReadFile in a repeatable and isolated fashion.

[MalikKillian]

Whoops! console should've been parametrized too but you get the idea.

[Andrew Revinsky]

I honestly tried several times to encompass the simplest usage case (a.k.a. proxyquire) and couldn't. Is there a way for me to have a chat with you - for a more cohesive experience with your library?

[Anton Korzunov]

GitHub issues are always open for you, as well as t.me/thekashey