Exploring JS Modules

I had been looking into the internals of webpack lately being a not so expert in Frontend, I realized there is more to modules than what I had known, so I took a turn and started looking into the module system. I thought of writing some of them which I found interesting (there are so many amazing explanations and in-depth discussion in this area - this is my attempt to write down what I found interesting when exploring modules.! 🙂). Instead of directly jumping into a topic and making the content totally disoriented I'll give a try by connecting the dots with a little context.

Why Module

The obvious reason to provide isolation and clear scoping, JavaScript is weird in its own ways; everything is global in its scope, first class functions, closures and scopes are enough to cause trouble if not handled properly, maintainability would be hard. Just using separate JS files isn’t going to solve the problem either.

The modules timeline,

Immediately invoking function expressions (IIFE)

I was tempted to skip this section because it was an obvious one, but there is something at the end of this blog that ties to this section 😃

There are many design patterns which aim to solve the above mentioned issues on why we need modules. The module pattern or the revealing module design pattern is one of the popular ones out there, with the help of these patterns we could write IIFEs in all our files and bundle them together and not pollute the global scope.

There are some downsides it as well,

• IIFEs get executed as soon as they are created, for the same reason they have to be executed in order
• Which also means bundling them in the same order.
• A change to a single file would need the whole files bundled again
• Tree shaking is not possible

CommonJS (CJS)

With the birth of Node the commonJS module specification was introduced. Node was primarily for backend applications and so was CJS modules. It was not meant to be run on browsers. But the concept was cool and neat. So with the use of loaders and bundlers CJS modules could be run on the browser and thus node gave birth to the module systems.

The wrapper function

Node provides a bunch of variables that can be accessed in each of the file/module we create. A good example of that is the require function that is used to import other modules, the exports/module object, the __filename, __dirname. Before the code is executed node wraps the code in a function which helps in scoping, and also provides those magic variables that seem to appear as if they are global.

I'll not go in detail here you can read more about it on node documentation

The Require function & The order of execution

require -it is just a function the require statement can be used anywhere in the code, it is not necessarily to be on top. It can also be inside a conditional block just like any function call.

There is no way for the JS engine to evaluate the code beforehand until it reaches the require statement it would have no idea.

Require is synchronous - on backend apps require(‘./helper’) would fetch the file from the disk, this entire operation would be synchronous.

if getLogLevel is used before the require call it would fail - The order of execution start from index.js

The Module Object & monkeypatching 🐒

The export/module as well is just an object created for this file, the caveat to that is the exported objects don’t have live binding (something which was introduced as part of ES modules, more details below), meaning if the exporting module changes the value (especially for primitive types) the importer will not be able to see it and there can be cases where things may get a bit tangled with circular dependencies.

A common example,

Even though the the value in incremented in counter.js the exported value would never change (of course it would behave differently if its a reference type). The reason I explicitly had this is because of how ES modules is different here.

Since everything is being wrapped in an object (the exports object) it turns to be an reference type, and due to this changing the value of the reference type (not just changing, you can also add new props to the object) would be visible in other modules as well - Monkeypatching 🐒

index.js adds a new prop to helper (before util.js loads helper) - once util.js loads it is able to see the newProp

AMD/UMD

Since CJS was natively for the backend but it was being transpiled and used on the web, a new spec was introduced for the web.

Asynchronous Module Definition (AMD) - It was dynamic and loads the modules asynchronously (suitable for the web)
Universal Module definition (UMD) was developed with the aim of unifying all the modules (CJS, AMD) but the result was a bloated bundled code.
I kept this small just to touch base it, I felt the need for it little in the present era.

EcmaScript Modules (ESM/MJS)

This is the standard for modules in JavaScript going-forward, defined by ECMA specification. All these (CJS, AMD, UMD) specs were not native to JavaScript, they had their own implementation to abstract and provide a module. We don't need any of the above mentioned modules anymore, but some of the packages out may still be any of those modules. Since ESM is a standard specification we no longer need to transpile ESM modules to run on browsers, most of the major version support (even though it's not ideal without a bundler yet). Node as well supports ESM without any transpiling to CJS.

Apart from the obvious differences between the syntaxes there is more to how ESM is different than CJS,

• Unlike require being a function in CJS, ESM import is a statement (though there is a dynamic import() as well). Before the ESM module is evaluated the engine is able to identify and build a dependency graph of the dependencies. This changes the whole execution order of how CJS and MJS execute. Regardless of where the import statement is placed it will be loaded and parsed before execution - simply think of it being hoisted (not exactly).

• For the same reason, import statements can’t be used inside conditional blocks (even though import() can be used)
• Also we can’t have variables in the module specifier (require can have, because it's just a function), even before the code is executed the loader starts parsing the importing statements and will start creating the module reference.
• Since you can export value types (they still share the same memory, more info on the next section.!) monkey patching isn’t easy in ESM (or works on reference types).
• Bundlers like webpack , rollup leverages import/export statements to do tree shaking because of how ESM works.

Live binding and Export

The same counter example, if translated to ESM would work as expected when calling the increment() function. It is because they share the same memory/live binding. Think of it more as a reference (even for the value types.!).

The solution on the left would work as expected, but the solution on the right won't. The only difference was the default export

Two different versions in ESM,

1. The solution on the left with named exports - yields the expected output on increment()
2. The solution on the right with default export - yields the same output as 1 (similar to CJS) But, the only difference was just the default export does this mean default and named exports are different things ? Actually they behave the same way, the reason we don't see the value being incremented is because of what happens internally. (this where I found it interesting)

There would be an internal data structure that manages the binding (the export references). It would have a local name and an export name. a good blog I found online to understand this what do es-modules export

//export default in ES Modules
let count =1;
export default count;

//default export - translated internally
let count = 1;
let *default* = count; //not a valid syntax
export *default* as default // not a valid syntax

As you could see when you create a default export the localName on the module will no longer point to the actual variable, it instead points a variable default that has no access. Now, if count is incremented there is no way for incrementing the default, despite having the live binding. Hence such a behavior.

Wrapping up

Regardless of what the module system we develop it must be transpiled accordingly to reap the benefits. For instance Tree Shaking in webpack would be possible only with ESM. if the project is written in ESM but transpiled into CJS then there would be no way for webpack to perform the dead code elimination(yea, this is where I took a turn to look into the modules).

There are so many good and interesting articles out which explains more about modules,

• The counter example explained in Stackoverflow 
• Modules Cartoon Deep dive
• IFFEs to Modules
• Chapter on Modules - by Axel Rauschmayer
• TC-39 - Modules
• MDN Docs - Import statement

Spoiler Alert

A little sneak peak into the bundled code of webpack was something like this,

Webpack Compiled code

(function (modules) {
      // webpack bootstrap code
  })([
        //0
      function (module, exports, ...) {
        // your module A
      },
        //1
      function (module, exports, ...) {
        // your module B
      }
  ])

It is an IIFE that accepts an array of IIFE's which are our modules.!! 😲, it went back to how it all started to IIFEs.(not entirely true though). If you see the pointers on #IIFEs downsides they all have been handled better now.

But, where are my modules ?😂

Webpack : meh.! ¯_(ツ)_/¯