Meteor offers you an out-of-the-box experience to use NPM packages in the Browser, that are targeted for the node plattform.
This is done by the meteor-node-stubs-package.
It does so by scanning your imports at build time and resolves the dependencies to provide a browser-friendly replacemant. Some of you might know this pattern from the famous browserify package and in fact it uses some of it's packages as replacements as you can see in the mapping file:
{
"assert": "assert/",
"buffer": "buffer/",
"child_process": null,
"cluster": null,
"console": "console-browserify",
"constants": "constants-browserify",
"crypto": "../wrappers/crypto.js",
"dgram": null,
"dns": null,
"domain": "domain-browser",
"events": "events/",
"fs": null,
"http": "stream-http",
"https": "https-browserify",
"module": "../wrappers/module.js",
"net": null,
"os": "os-browserify/browser.js",
"path": "path-browserify",
"process": "process/browser.js",
"punycode": "punycode/",
"querystring": "querystring-es3/",
"readline": null,
"repl": null,
"stream": "stream-browserify",
"_stream_duplex": "readable-stream/lib/_stream_duplex.js",
"_stream_passthrough": "readable-stream/lib/_stream_passthrough.js",
"_stream_readable": "readable-stream/lib/_stream_readable.js",
"_stream_transform": "readable-stream/lib/_stream_transform.js",
"_stream_writable": "readable-stream/lib/_stream_writable.js",
"string_decoder": "string_decoder/",
"sys": "util/util.js",
"timers": "timers-browserify",
"tls": null,
"tty": "tty-browserify",
"url": "url/",
"util": "util/util.js",
"vm": "vm-browserify",
"zlib": "browserify-zlib"
}
Try it yourself
You can test it yourself by creating a new Meteor project and import a node-specific package on the client:
client/main.js
import { Buffer } from 'buffer'
Meteor.startup(() => {
console.log(Buffer.from('Buffer on the client')) // Uint8Array(20) [ 66, 117, 102, 102, 101, 114, 32, 111, 110, 32, … ]
})
This is great, since you don't need to configure anything to make that work. Now here is the issue why this can easily bloat your client bundle.
Dynamically growing
When there is no need to stub a node package, the meteor-node-stubs package is only about 3.61KB in size. This is because Meteor's code-splitting will detect at build time, whether a node package is imported on the client or not.
Therefore, the meteor-node-stubs package only "grows" when you actually import a node module on the client.
For example our buffer increased the stubs package size by 23.89KB (detected by using Meteor's bundle-visualizer).
As you can see this can easily get out of hand! For example, if you use the crypto package on the client, your node-stubs will have to use crypto-browserify which adds about 630KB to the client if the whole crypto library is intended to be used.
Beware what you import
At this point you should have already realized, that simply importing anything on the client can lead to bloated bundles and thus very long load times and heavily delayed time-to-interact.
Think before import
It is your responsibility to analyze, which package you want to use and how make use of it.
Do you really need Buffer on the client? Do you really need crypto on the client or can you use the Web Crypto API instead?
Analyze co-dependencies
Beyond the node core packages there are also NPM packages. that specifically target the Node environment. Be aware of this fact and check it's dependencies. If the package depends on path for example, then meteor-node-stubs will in turn add path-browserify and if it depends on stream, then the stubs will include stream-browserify.
How to avoid bloated client bundles
1. Make use of code-splitting
Meteor allows to write isomorphic code and meteor-node-stubs plays an important role in it. You can therefore write code once and use it on the server and the client the same way.
This is totally fine, if it's what you intended. If you did not intend, but accidentally imported node-code to the client (for example due to tight coupling or bad design of imports) you will end up with an increased, but unused, client bundle size.
To resolve this, let's take a look at a short example where we want to create an SHA512 digest using crypto on the server and Web Crypto API in the Browser.
First, create a function createSHA512 under the path /imports/api/sha512/server/createSHA512.js. This is our server function:
import crypto from 'crypto'
export const createSHA512 = async input => await crypto.createHash('sha512').update(input).digest('base64')
Now let's add this to an export, say SHA512 but only on the server. Let's actually use the Web Crypto API on the client:
import { Meteor } from 'meteor/meteor'
export const SHA512 = {}
if (Meteor.isServer) {
SHA512.create = async input => {
import { createSHA512 } from './server/createSHA512'
return createSHA512(input)
}
}
if (Meteor.isClient) {
SHA512.create = async input => {
const encoder = new TextEncoder()
const data = encoder.encode(input)
const hash = await window.crypto.subtle.digest({ name: 'SHA-512' }, data)
const buffer = new Uint8Array(hash)
return window.btoa(String.fromCharCode.apply(String, buffer))
}
}
The function will behave the same on server and client and can be imported by both without the need for a stub:
/client/main.js and / or
/server/main.js:
import { SHA512 } from '../imports/api/sha512/SHA512'
SHA512.create('The quick brown fox jumps over the lazy dog')
.catch(e => console.error(e))
.then(hashed => console.debug(hashed))
The above code will print for both server and client the same result, B+VH2VhvanP3P7rAQ17XaVEhj7fQyNeIownXhUNru2Quk6JSqVTyORJUfR6KO17W4b/XCXghIz+gU489uFT+5g==. However, under the hood it uses two different implementations and the client bundle does not need to stub the crypto package. Saved 630KB 🎉
2. Use dynamic imports
If you can't omit a certain node-targeted package on the client and you don't need it immediately at application start, you should use dynamic-import to defer the import of modules at a later point in time.
This will still increase the amount of data sent to the client but will keep the initial bundle size small to ensure a fast page load and time-to-interact.
3. Use ServiceWorkers for caching
Meteor signs the bundles with hashes, so you can use ServiceWorkers to prevent reloading the same code every time. This gets even more performant, when combined with dynamic imports.
You can set this up following my "Three step Meteor PWA"" tutorial.
Summary
Meteor node stubs is a great feature and an important part of Meteor's build system. However, it's power can easily turn into an issue if you don't take a close look at which packages you import where and why.
I regularly publish articles here on dev.to about Meteor and JavaScript. If you like what you are reading and want to support me, you can send me a tip via PayPal.
You can also find (and contact) me on GitHub, Twitter and LinkedIn.
Keep up with the latest development on Meteor by visiting their blog and if you are the same into Meteor like I am and want to show it to the world, you should check out the Meteor merch store.
Top comments (2)
Great job on this article @jankapunkt !! This is a really helpful reference for the Meteor community! I'm using your ServiceWorker and really appreciated the ease of turning my app into a PWA with it 😀
I've not tried dynamic imports yet, but they look really useful for large dependencies on the client that could load at a later time. I didn't know about them and I will make use of them soon.
I've been thoughtful about my client side imports, but I think I could do a lot more to reduce their size soon.
Thanks for being such a clear thinker & tester, you're adding A LOT of value to the Meteor community! 🎉
Hey @mullojo thanks a lot for the response. I am very grateful to hear my contributions to be of value to others! Regarding dynamic import: our use case is to have a very rich client UI with lots of pages and features and at the same time clients may have an unstable or low connection. If you want to keep a great performance in terms of loading times then dynamic import is a must. Once you get into the design logic for dynamically loaded pages and content then it will become a no-brainer.
If you have any questions about dynamic import that could be worth to become an article please let me know ;-)