In the beginning, there was the <script> tag.
We managed dependencies by carefully arranging our scripts in our HTML. You had to load jQuery before you loaded your plugins, your libraries before your app code. As we began adding more interactivity and evolved from websites to web apps, this began to get out of hand. Large projects had complex waterfalls of requests that were difficult to manage and optimize. We had defer and async attributes, but they only help in some circumstances. We needed a better way to manage our dependencies.
The first step forward was when we began concatenating our scripts together. This reduced the total number of HTTP requests and helped guarantee execution order, but it remained a manual process. Scripts had to be concatenated together in the correct order to work. We concatenated scripts into groups to balance each file's size against the total number of requests, but we still had to specify the order and grouping. This is about the time that the concept of having a build step for your Javascript gained popularity.
Grunt became the first widely popular "task runner," used to concatenate scripts and optimize assets. Its configuration grew unwieldy on larger projects though, and Gulp refined the ideas into a "streaming" API that was simpler to reason about and faster.
As we became more comfortable with the idea of having a build step, CoffeeScript entered as the first popular alternate syntax. With so many apps written with Ruby on Rails, web developers craved the simpler syntax of Ruby. Many of CoffeeScript's ideas were eventually folded into ES2015—you can thank it for => and ..., among others. Another concept it helped popularize was separating code into modules. Each compiled CoffeeScript file was inserted into its own IIFE (immediately instantiated function expression), scoping each script to prevent polluting the global namespace.
Require.js and Bower came onto the scene to help us wrangle our third-party code. Require.js introduced "asynchronous module definitions," or AMD modules, a packaging method still used by some apps. They were loaded into the browser on-demand, which was super cool! No more manually shuffling script tags. The syntax was a little clunky,
// from http://requirejs.org/docs/api.html
requirejs(['jquery', 'canvas', 'app/sub'],
function($, canvas, sub) {
//jQuery, canvas and the app/sub module are all
//loaded and can be used here now.
});
but it was much better than manually managing the order ourselves. Bower was initially a complement to npm, before npm had many modules that supported running in the browser. Eventually, Bower was deprecated in favor of npm, and Require.js added the option of passing a require function to emulate commonJS modules from node.
define(function(require, exports, module) {
var $ = require('jquery');
var canvas = require('canvas');
var sub = require('app/sub')
})
So now we had something that automatically managed which scripts to load and in which order to load them. Life was good. Slowly, a new problem began to develop: it was so easy to add dependencies that we began to use a lot. Because each dependency was loaded as a separate script, loading a web app would kick off dozens— or even hundreds—of HTTP requests for tiny .js files. The simultaneous requests would block each other from loading, delaying initial load.
There were several fixes developed for this. The problem was taken into consideration for the design of HTTP2, which added multiplexing to help alleviate the problem. Require.js added an optimizer tool that would bundle up these modules up into a single file or group of files, but it wasn't suitable for development and was tricky to configure. HTTP2 rolled out very slowly, and ultimately wasn't the silver bullet people hoped it would be.
Developers began experimenting with alternatives, and the number of tools for bundling dependencies exploded. Browserify, Broccoli.js, Rollup, webpack, and surely others that I never heard about. There are still more being created, with Parcel being the most recent addition I know of. They all have slightly different takes on API and features. webpack won mindshare for apps because of its excellent code splitting features and flexibility, and later iterations significantly improved usability (seriously webpack 4 is fantastic). Rollup has become a go-to tool for bundling libraries because it produces the smallest bundle in most cases.
This focus on tools for resolving dependencies revealed some shortcomings with CommonJS' require function. require was created as part of Node.js, and had some semantics that made it more difficult to use in the browser. TC39 standardized a module definition specification, ES modules, that better meets the different use cases in Node.js and the browser. It's still evolving—Node.js recently released version 10 with experimental support, and the dynamic import() function hasn't quite landed.
That brings us to today. Webpack is the de-facto bundler for several years now and has steadily improved over the years. Not only can we define bundles of Javascript, we can specify which files depend on stylesheets or images and load them only when needed. Loaders exist to inline images below a certain size, and some crazy folks have started writing their CSS in their JS (try it, it's great).
I didn't even touch on Yarn vs npm vs pnpm, services like unpkg, or any of the drama and arguments that got us where we are today. npm has taken off into the stratosphere after hitting a billion downloads a week in 2016, with the numbers at the beginning of 2018 dwarfing those. The challenges we have today are around when not to use dependencies, and keeping an eye on the total amount of code we're shipping.
This is just a representation of what I've experienced firsthand in the past 6 years of writing code that runs in the browser. It's a short period of time in the history of the web, but the amount of innovation and evolution has been incredible to watch.
Thanks for reading! I'm on Twitter as @cvitullo (but most other places I'm vcarl). I moderate Reactiflux, a chatroom for React developers and Nodeiflux, a chatroom for Node.JS developers. If you have any questions or suggestions, reach out!
Top comments (6)
Damn your loading gif, I thought that dev.to crashed, it kept loading.
As for the history, I think the first ones were 2006 Yahoo YUI Compressor and Google... I forgot its name, that minified multiple JS/CSS files. This cover many years of history before nodeJS.
Also, from my experience, JS/CSS files were never the problem, the Flash scripts, images and number of requests were the bottle necks.
We also had to use gaming techniques to group multiple images into 1 (maps and atlases), to make 1 request and save some kb (because they share the same colors).
Another limitation was the number of parallel requests, which we bypass sometime by using different subdomains for static files.
Good point! YUI Compressor and the Closure compiler were huge steps forward. I never got either to work in my projects, so they slipped my mind.
Thanks for the enlightening post, I never used much of the frontend frameworks of today cuz everything I did was with HTML+CSS, also I've spent most of my time coding with Java and SQL databases.
Nowadays getting into React (for new job purposes) it's been quite an exciting journey and also finding all sorts of things with js at the end of their name hahaha.
To add to this, I highly recommend this article recapping a similar progression, and explaining why bundlers are useful.
oh hohoho the loading gif got me. nice article! :D
I jumped from labjs and custom dependency loading to webpack. Never cared for require.js syntax or config or suggested project structuring.