DEV Community: David Okeke

I wrote a module bundler. notes, etc

David Okeke — Wed, 24 Jul 2024 13:15:37 +0000

I built a simple JavaScript bundler and it turned out to be much easier than I expected. I'll share all I learned in this post.

When writing large applications, it is good practice to divide our JavaScript source code into separate js files, however adding these files to your html document using multiple script tags introduces new problems such as

pollution of the global namespace.
race conditions.

Module bundlers combine our source code from different files into one big file, helping us enjoy the benefits of abstractions while avoiding the downsides.

Module bundlers generally do this in two steps.

Finding all the JavaScript source files, beginning from the entry file. This is known as dependency resolution and the map generated is called a dependency graph.
Using the dependency graph to generate a bundle: a large string of JavaScript source code that can run in a browser. This could be written to a file and added to the html document using a script tag.

DEPENDENCY RESOLUTION

As previously mentioned, here we

take an entry file,
read and parse its content,
Add it to an array of modules
find all its dependencies (other files it imports),
Read and parse contents of dependencies
Add dependencies to array
Find dependencies of dependencies and so on and so forth till we get to the last module

Here’s how we would do that (JavaScript code ahead)

Create a bundler.js file in your text editor and add the following code:

const bundler = (entry)=>{
          const graph = createDependencyGraph(entry)

          const bundle = createBundle(graph)
          return bundle
}

The bundler function is the main entry of our bundler. It takes the path to a file (entry file) and returns a string (the bundle). Within it, it generates a dependency graph using the createDependencyGraph function.

const createDependencyGraph = (path)=>{
          const entryModule = createModule(path)

          /* other code */
}

The createDependencyGraph function takes the path to the entry file. It uses the createModule function generate a module representation o this file.

let ID = 0
const createModule = (filename)=>{
          const content = fs.readFileSync(filename)
          const ast = babylon.parse(content, {sourceType: “module”})

          const {code} = babel.transformFromAst(ast, null, {
              presets: ['env']
            })

           const dependencies = [ ]
           const id = ID++
           traverse(ast, {
                   ImportDeclaration: ({node})=>{
                       dependencies.push(node.source.value)
                   }
            }
            return {
                           id,
                           filename,
                           code,
                           dependencies
                       }
}

The createAsset function takes the path to a file and reads it’s content into a string. This string is then parsed into an abstract syntax tree. An abstract syntax tree is a tree representation of the content of a source code. It can be likened to the DOM tree of an html document. This makes it easier to run some functionality on the code such as searching through, etc.
We create an ast from the module using the babylon parser.

Next with the help of the babel core transpiler we convert convert the code content to a pre-es2015 syntax for cross browser compatibility.
Afterwards the ast is traversed using a special function from babel to find each import declaration of our source file(dependencies).

We then push these dependencies (which are strings text of relative file paths) into a dependency array.

Also we create an id to uniquely identify this module and
Finally we return an object representing this module. This module contains an id, the contents of our file in a string format, an array of dependencies and the absolute file path.

const createDependencyGraph = (path)=>{
          const entryModule = createModule(path)

          const graph = [ entryModule ]
          for ( const module of graph) {
                  module.mapping = { }
module.dependencies.forEach((dep)=>{
         let absolutePath = path.join(dirname, dep);
         let child = graph.find(mod=> mod.filename == dep)
         if(!child){
               child = createModule(dep)
               graph.push(child)
         }
         module.mapping[dep] = child.id
})
          }
          return graph
}

Back in our createDependencyGraph function, we can now begin the process of generating our graph. Our graph is an array of objects with each object representing each source file used in our application.
We initialize our graph with the entry module and then loop it. Although it contains only one item, we add items to the end of the array by accessing the dependencies array of the entry module (and other modules we will add).

The dependencies array contains relative file paths of all dependencies of a module. The array is looped over and for each relative file path, the absolute path is first resolved and used to create a new module. This child module is pushed to the end of the graph and the process starts all over again till all dependencies have been converted to modules.
Also each module is giving a mapping object which simply maps each dependency relative path to the id of the child module.
A check for if a module exists already is performed on each dependency to prevent duplication of modules and infinite circular dependencies.
Finally we return our graph which now contains all modules of our application.

BUNDLING

With the dependency graph done, generating a bundle will involve two steps

Wrapping each module in a function. This creates the idea of each module having its own scope
Wrapping the module in a runtime.

Wrapping each module

We have to convert our module objects to strings so we can be able to write them into the bundle.js file. We do this by initializing moduleString as an empty string. Next we loop through our graph appending each module into the module string as key value pairs, with the id of a module being the key and an array containing two items: first, the module content wrapped in function (to give it scope as stated earlier) and second an object containing the mapping of its dependencies.

const wrapModules = (graph)=>{
         let modules = ‘’
           graph.forEach(mod => {
    modules += `${http://mod.id}: [
      function (require, module, exports) {
        ${mod.code}
      },
      ${JSON.stringify(mod.mapping)},
    ],`;
  });
return modules
}

Also to note, the function wrapping each module takes a require, export and module objects as arguments. This is because these don’t exist in the browser but since they appear in our code we will create them and pass them into these modules.

Creating the runtime

This is code that will run immediately the bundle is loaded, it will provide our modules with the require, module and module.exports objects.

const bundle = (graph)=>{
        let modules = wrapModules(graph)
        const result = `
    (function(modules) {
      function require(id) {
        const [fn, mapping] = modules[id];

        function localRequire(name) {
          return require(mapping[name]);
        }

        const module = { exports : {} };

        fn(localRequire, module, module.exports);

        return module.exports;
      }

      require(0);
    })({${modules}})`;
  return result;
}

We use an immediately invoked function expression that takes our module object as an argument. Inside it we define our require function that gets a module from our module object using its id.
It constructs a localRequire function specific to a particular module to map file path string to id. And a module object with an empty exports property
It runs our module code, passing the localrequire, module and exports object as arguments and then returns module.exports just like a node js module would.
Finally we call require on our entry module (index 0).

To test our bundler, in the working directory of our bundler.js file create an index.js file and two directories: a src and a public directory.

In the public directory create an index.html file, and add the following code in the body tag:

<!DOCTYPE html>
<html>
    <head>
        <title>Module bundler</title>
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    </head>
    <body>
       <div id='root'></div>
       <script src= ‘./bundler.js> <script>
    </body>
</html

In the src directory create a name.js file and add the following code

const name = “David”
export default name

also create a hello.js file and add the following code

import name from ‘./name.js’
const hello = document.getElementById(“root”)
hello.innerHTML = “hello” + name

Lastly in the index.js file of the root directory import our bundler, bundle the files and write it to a bundle.js file in the public directory

const createBundle = require(“./bundler.js”)
const run = (output , input)=>{
let bundle = creatBundle(entry)
fs.writeFileSync(bundle, ‘utf-8’)
}
run(“./public/bundle.js”, “./src/hello.js”)



Open our index.html file in the browser to see the magic.

In this post we have illustrated how a simple module bundler works. This is a minimal bundler meant for understanding how these technologies work behind the hood.

please like if you found this insightful and comment any questions you may have.

HNG STAGE 0 TASK

David Okeke — Sun, 30 Jun 2024 21:52:50 +0000

The first website was created by on August 6 1991 by British Computer Scientist Thomas Bernie Lee and it contained information about the World Wide Web Project. It launched at the European Organization for Nuclear Research, CERN. On it, people could find out how to create web pages and learn about hypertext (coded words or phrases that link to content).

Tim Berners-Lee is also credited with developing the first web browser, Many others were soon developed, with Marc Andreessen's 1993 Mosaic (later Netscape), being particularly easy to use and install, and often credited with sparking the internet boom of the 1990s.

JavaScript, the language of the web was invented by Brendan Eich in 1995.
It was developed for Netscape 2, and became the ECMA-262 standard in 1997.
The invention of javascript provided great opportunities for developers to build impressive functionalities that run right in the browser but had created new problems, one of which entailed organizing javascript source files for a single web page.

The open nature of the web also provided for multiple solutions to be built for the same problems allowing developers a decent selection of solutions for whatever problems they faced while building for the web. The best way to organize and assemble multiple JavaScript code files into one file is to use module bundlers.

*JAVASCRIPT MODULE BUNDLERS *

A bundler is a development tool that combines multiple JavaScript code files into a single one that can be loaded in the browser and used in production. Generating a dependency graph when a bundler traverses your first code files is an outstanding feature. This implies that the module bundler keeps track of your source files’ and third-party’s dependencies starting at your provided entry point.
Dependency graph generation and eventual bundling are the two stages of a bundler’s operation.
The common JavaScript bundlers are Webpack and Rollup

*COMPARISON BETWEEN ROLLUP AND WEBPACK *

Rollup is a JavaScript module bundler that focuses on providing a simple and efficient way to bundle JavaScript code for modern web development. It is known for its tree-shaking capabilities, which eliminate unused code during the bundling process, resulting in smaller bundle sizes.
Webpack is a powerful module bundler for JavaScript applications. It allows developers to bundle and optimize their code, including JavaScript, CSS, and images, into a single output file.

Configuration
Webpack is highly configurable and allows for complex setups, making it suitable for large-scale projects with diverse requirements.
Rollup, on the other hand focuses on simplicity. It has a simpler configuration model, which makes it easier to set up and use for small to medium-sized projects.

Bundle Size
Rollup generally produces smaller bundle sizes compared to Webpack. It is known for its tree-shaking capabilities, allowing it to eliminate unused code and optimize the output significantly. Webpack, while it provides some optimization options, tends to have larger bundle sizes by default.

Code Splitting
Webpack has more advanced code splitting capabilities and provides various strategies for splitting code into separate bundles. Rollup also supports code splitting but has more limited options compared to Webpack.

Module Formats
Both Webpack and Rollup support multiple module formats such as ES modules, CommonJS, and AMD. However, Rollup is known for its excellent support for ES modules and is often preferred for projects targeting modern browser environments. Webpack, on the other hand supports a broader range of module formats, making it suitable for projects with legacy codebases.

Build Speed
Rollup is generally faster than Webpack when it comes to build times. It has a simpler and more streamlined build process, which leads to faster bundling. Webpack, due to its extensive feature set, can take longer to build, especially for larger projects.

Community and Ecosystem
Webpack has a larger and more established community compared to Rollup. It has a vast ecosystem of plugins, loaders, and tools developed by the community, which helps with integrating various technologies and simplifying complex setups. Rollup, while it has a smaller community, still has a decent number of plugins and tools available, but the ecosystem is not as extensive as Webpack's.

This article is a product of my hng intership journey. Learn more about hng here