Using Multiple WebWorkers With Create React App

#react #webpack #typescript #webdev

There are a few great articles about using one WebWorker with Create React App. The gist of them is to use worker-loader and something like rescripts-worker-loader to modify the Webpack config. Easy (after trying ten times).

Problem

I tried and succeeded to run multiple WebWorkers with Create React App and that was harder. At first I got this lovely error message

Conflict: Multiple assets emit to the same filename bundle.worker.js

Yuck!

Solution

Here's the solution, assuming you're using rescript, and an explanation follows.

In your rescriptsrc.js add the following:

function makeMultipleWebworkersWork(config){
    // Change the output file format so that each worker gets a unique name
    config.output.filename = 'static/js/[name].bundle.js'
    // Now, we add a rule for processing workers
    const newRules = [{

        test: /\.worker\.(c|m)?[tj]s$/i,
        type: "javascript/auto",
        include:  config.module.rules[1].include,
        use: [
            {
                loader: "worker-loader",
            },
            {
                loader: "babel-loader",
                options: {
                    presets: ["@babel/preset-env"],
                },
            },
        ],
        // Here we append all the old rules
    },...config.module.rules]
    // now update Webpack's config with the new rules
    config.module.rules = newRules
    return config
}
module.exports =[
    makeMultipleWebworkersWork,
/*Your other stuff goes here */
]

Explanation

After a day of debugging, I realized that CRA's configuration of Webpack doesn't take filenames into account for output (at least in dev mode), so all the workers get emitted to to the same bundle.worker.js file, but they overwrite each other not append.

The code above configures worker-loader and makes sure each worker goes to a different file, as desired.

Why

I'm building an open source in browser NLP thing. As you might guess, it trains NLP models in the browser interactively.
Using multiple workers is a great why to crunch a lot of data without blocking the users main thread, e.g. keeping the app responsive and engaging.

Still a bit early to release it, but follow me here and on twitter if you'd like to keep updated.