DEV Community: zhangHongEn

hmr of remote module

zhangHongEn — Mon, 11 Dec 2023 03:23:37 +0000

Hot update of a single application is generally available out of the box, while hot update of remote modules requires some adaptation processing according to the development environment of multiple projects, and there is no mainstream mature solution.

Problem manifestation

https://github.com/wpmjs/examples/tree/main/module-federation-react-hmr You can clone this example and run it locally

There are three modules: app1 (localhost:9001), app2 (localhost:9002), and app3 (localhost:9003). As the root application, app1 introduces app2 and app3 through module-federation. At this time, start the development environment of the three apps. Try Modify local code.

When you access the port of each app individually and modify the corresponding code, the page will be updated.
When accessing localhost:9001 directly, only the code page of app1 will be updated, and the code pages of app2 and app3 will not be refreshed or changed (websocket successfully triggered and completed the code update, and the react component did not change)

Cause Analysis

As a result, only the first loaded root application can be hot updated, and other subsequently loaded modules cannot be hot updated. The reason is actually that the hot update conditions of react are relatively strict:

React and react-dom are singletonsReact and react-dom are singletons
react, react-dom is the development version
Register REACT_DEVTOOLS_GLOBAL_HOOK with global through react-refresh
react-dom.development gets REACT_DEVTOOLS_GLOBAL_HOOK (react-dom will get REACT_DEVTOOLS_GLOBAL_HOOK when the code is executed for the first time, so you must strictly follow the order of loading react-refresh first, completing the registration hook, and then loading react-dom. If the order is wrong, hot update cannot be started., there is no remedy, this is also the most error-prone and difficult to troubleshoot)

Well, since the previous example uses module-federation to share react and react-dom and both are locally started services, it actually meets the above conditions 1 and 2.

The reason why app2 and app3 cannot be hot updated is https://github.com/pmmmwh/react-refresh-webpack-plugin/blob/426490f7bf7bf166efc85a0588e5c3b8e64cd72f/client/ReactRefreshEntry.js#L17 This plug-in found that global hooks already existed, so it did not register app2 and app3. Only app1 successfully registered REACT_DEVTOOLS_GLOBAL_HOOK (by the way, react-dom.development is obtained directly from global REACT_DEVTOOLS_GLOBAL_HOOK, so it is not feasible to register each application individually.)

For specific questions, please see this issue: Component modifications are not applied in module federation · Issue #394 · pmmmwh/react-refresh-webpack-plugin

Solution

Multiple modules reuse the same REACT_DEVTOOLS_GLOBAL_HOOK:

https://github.com/zhangHongEn/universal-module-federation-plugin/tree/main/packages/single-react-refresh-plugin

Vue's hot update is not so troublesome to integrate. Logically, it only needs to satisfy the vue singleton and the development version plus vue-loader. If anyone encounters similar problems, you can leave a message.

Remote module optimization points

It is not difficult to find from the above example that although we have three modules that can be independently constructed and released, we need to start the root application + remote module project locally during development. If we only need to modify one module and disassemble the project, it will reduce efficiency. of. The better way is to start only the modules that need to be debugged or modified locally, so that they can be directly connected to the resources that have been deployed online or in the test environment (you can experience the micro front-end example built below), In this way, you can truly achieve minimal independent construction, independent release and independent debugging, to improve the development and deployment efficiency of the project and even reduce the scope of release impact.

This development method involves how to start a remote module so that the main application can be switched to the development version, which will be discussed separately later.

micro frontend

An application composed of multiple independently built and independently released modules can also be called a micro front end.

You can also use the following link to experience starting the remote module alone and directly connecting to the online environment for development and debugging:

Visit the micro front-end root application demo address: https://zhanghongen.github.io/universal-module-federation-plugin/
clone remote module mf-app-01: https://github.com/zhangHongEn/universal-module-federation-plugin/tree/main/packages/mf-example/app1
npm install && npm run start
Activate the local connection switch

You can see the effect in the picture below, a remote page, part of the page uses locally started components

The above micro front-end page is implemented using wpmjs and npm-federation:

https://github.com/zhangHongEn/universal-module-federation-plugin/tree/main/packages/npm-federation-webpack

Finally, I will share some micro-front-end related technologies.:

Loader:

systemjs、module-federation（Can be regarded as a mainstream remote module loader）
wpmjs（Versioned loading of remote modules, and integration of multiple module specifications such as: umd, system, module-federation）
npm-federation（Load the plug-in of the remote npm package in the module-federation method, and you can also load the umd package）

Open up the technology stack:

vuereact-combined（vue2 and react components refer to each other）

veaury（vue3 and react components refer to each other）

Application isolation: qiankun, single-spa, etc.

Remote package manager based on npm and module-federation

zhangHongEn — Fri, 22 Sep 2023 11:44:12 +0000

Remote package manager based on npm and module-federation

Recently I worked on a micro front-end open source project(web package manager), As the holiday approaches, I will briefly promote it, and I will continue to update after the holiday. Everyone is welcome to experience it. or star, Communication and discussion are also welcome, thank you

WPM introduce

The web package manager is a remote package manager based on module-federation (mf) and npm. Connected the existing umd specification and mf module. You can share remote modules more easily and without worries. You can easily access existing npm packages or use npm private sources and other sources to build a new remote module system.
It also has a built-in self-expandable debugging plug-in system, which can switch remote/local modules with one click, integrate hot updates, etc. to greatly improve development and debugging efficiency.

Features

Multi-module specifications (integrated module specifications such as system, umd, module-federation, etc.)
Debug mode/hot update (integrates debugging panel and hot update, can automatically connect to the locally started dev-server)
Version management (you can use private or public npm as the remote module storage source, or you can customize url splicing rules to store remote modules yourself)
Remote lock (supports dynamic configuration of remote module version)
Performance optimization (the plug-in automatically optimizes the loading links of multiple remote modules and their chunks to avoid waiting for multiple loads)

Improve efficiency value

I've been using wpm in my own work for two years and have only recently taken the time to decouple it. Simply put, its value lies in greatly improving development and construction efficiency as well as versioning, grayscale strategy and other management.

Technical advantages

The underlying technology uses npm + module-federation, which is more in line with the existing technology ecosystem and has the lowest access cost. To elaborate on this future, I will only briefly introduce it now. As the most complete basic capability (ts, ssr, various packaging tool plug-ins) and the most active remote module sharing and loading capabilities, module-federation can automatically be compiled by the packaging tool. Analyze how many js resource (chunk) requests should be initiated in parallel and in advance to load the next module to remove the loading waterfall. This will also bring performance improvements that cannot be achieved by introducing remote modules during manual coding runtime.

可以独立使用的底层能力

When I was working on wpm, I also made many other libraries to solve some problems.

webpack4 module-federation webpack4 project uses module-federation
module-federation-runtime Simulated and exposed module-federation runtime API supporting browser and node environments
universal-module-federation-plugin Allows customizing the loading behavior of remotes
wpmjs（umd + module-federation） Connect the runtime of umd + mf, load the sdk of the remote module from npm
npm-federation remotes supports remote npm modules
systemjs-intercept Intercept and customize the behavior of systemjs to obtain dependencies

Recent plans

At present, the open source version exposes about 60% of the capabilities. I will continue to update after the vacation.

Documentation, examples, development specifications, contribution guide supplement
wpm plugin supplement
Built-in hot update integration
Articles (wpm introduction, module-federation features and performance analysis)

Hundreds of micro-frontend applications in the company, How to smoothly upgrade mf from umd

zhangHongEn — Sun, 04 Dec 2022 15:38:24 +0000

MF introduce

webpack5 module-federationvery powerful, The front-end code (module) sharing has reached a new level, and remote code can be used like a local module

Before that, it is bound to use the most widely standard UMD module specification for sharing, Using loaders such as systemjs needs to process promise logic, which only has a small impact when using asynchronous components. If the remote module is a function or a configuration item (such as JSON data), then the usage restrictions will be more obvious, and There is also no development environment support such as mature supporting TS prompts。

MF can use remote modules almost without perception, and has solutions in various fields（webpack-4、5、vite/rollup、TS、SSR、medusa/dashboard等）, More efficient and stable than requiring manual coding to process remote modules asynchronously

MF enhance

The company has hundreds of micro front-end applications and modules, and the company's business architecture will also require a large number of product integration components (different businesses provide capabilities and components to each other), which is very compatible with the characteristics of MF。

So what's the problem with using MF directly??

Hundreds of micro-frontend projects need to be fully upgraded and adapted (if a team uses webpack5 to provide an MF module, the webpack4 team cannot consume this module)
How to reuse a large number of existing remote modules without refactoring (after upgrading mf, remote resources are divided into two types (1.UMD 2.MF), both remote modules will have dependencies, and the two dependencies need to be able to communicate and maintain singleton)

For the above two problems, we provide two open source projects as corresponding solutions, and improve the versatility of module-federation, no longer limited by webpack5 (module-federation/webpack-4), and support the integration of other module specifications (universal-module-federation).

module-federation/webpack-4
universal-module-federation

With the support of these two plug-ins, MF can even be used as a synchronously loaded remote universal module loader, and can be used directly in existing projects, It should also be able to reuse the original ecology such as ts, dashboard/medusa of MF。

How to support multi-version coexistence:

Many people may think that MF's remote cannot coexist in multiple versions. First of all, this is not a shortcoming of MF, nor is it unsolvable。

Since the webpack application needs to mount global variables on the window, use jsonp to load different chunks. And MF also has a globally unique module name that will pollute the global space. When multiple versions of the same application appear at the same time, the global module will conflict。
You only need to add the dimension of the version number to the configuration items of webpack, output.library.[name | uniqueName], ModuleFederationOptions.name, and the global conflict can be resolved (because the global jsonp variable name will appear In all chunks, the hash of all chunks packaged in each version of this setting will change, and the cache will be invalidated, but generally there is no need for multiple versions)。

Disadvantages of multiple versions:

Multiple versions of the same remote are loaded at the same time, and should not exist in the H5 environment. The appearance of multiple versions will increase the resources requested by the network and cause performance problems
In large-scale micro-frontend projects on the B-end, there is a high probability that multiple versions of remote will need to coexist. However, similar to H5, it generally only appears across micro-frontend applications. The same application still needs to ensure that the version is unified, and no redundant resources are required to ensure loading speed. However, there are qiankun and other micro-frontend solutions across applications to isolate window global variables, and this problem no longer exists.

future upgrade:

If webpack is eliminated, can the MF project continue to be used without refactoring

If turbopack succeeds webpack, MF official should continue to support
If other build tools succeed webpack, they can also make the MF project continue to run. You can refer to usemf (loading MF modules in other environments), and here In the process, there must be similar technologies.
Even in the era when esmodule can be used directly in the browser, and the original MF module file can be imported directly, it can also be used in the form of promise, but then the packaging process can be simplified and the ES module can be directly output for support.

Therefore, no matter how the front-end ecology changes in the future, there can always be a smooth upgrade plan, but we need to move closer to the open source ecology, so as not to build a car behind closed doors and lead to reconstruction in the future.

webpack4 module-federation

zhangHongEn — Sun, 23 Oct 2022 10:55:17 +0000

module-federation It is a new feature of webpack5 update, and it is easier to share front-end code, This article will introduce module-federation/webpack-4 ealization principle and its difference from webpack5

module-federation/webpack-4 ealization principle

Briefly explain the implementation principle, how do webpack4 and webpack5 achieve interoperability? There are three key points

usemf（Use the sdk output by webpack5 build to simulate a webpack5 environment to load module-federation in a non-webpack5 environment）

Follow the loading process of module-federation（1. init all remote container 2. merge shareScopes 3. Restore the sharing rules of webpack5-share）; output module-federation-container

// container
{ 
    async init(shareScope){}, 
    get(name){ return async factory() } 
}

// shareScopes example
{
  [default]: {
      [react]: {
          [18.0.2]: {
              get() {
                return async function factory() {
                    return module
                }
              },
              ...other
          },
          [17.0.2]: {
              get() {
                return async function factory() {
                    return module
                }
              },
              ...other
          }
      }
  }
}

Finally, a capability that webpack4 lacks, enabling jsonp-chunk to support waiting for dependencies (remote modules) to load

Implement the above process through plug-ins
Enlarge picture

Add a new entry to implement the loading process of module-federation and output the container
Intercept the module loading of remotes, no longer load local modules directly, but use remote modules
Intercept shared module loading, no longer load local modules directly, but use remote modules
Shared requests are intercepted, but the shared bundle still needs to be output, and the function merge shareScopes will be loaded

It introduces the two red parts in the figure, how to change the webpack4 loading process to support loading remote modules

Intercept imports, reserve dependency tags
1. Set alias, transfer remotes to a non-existent url (no existence can be intercepted in the second step)
2. Forward remotes to specific loader in "compiler.resolverFactory.plugin("resolver normal") --> resolver.hooks.resolve.tapAsync" hook
3. Leave a string in the loader to mark that the current module depends on the remote module, get and export the value of the remote module
4. The "compilation.mainTemplate.hooks.jsonpScriptchunk" hook makes the jsonp chunk wait for the remote module to load before executing

Source code analysis

https://github.com/module-federation/webpack-4

// module-federation/webpack-4/lib/plugin.js
apply(compiler) {
    // 1. Generate unique jsonpFunction global variables to prevent conflicts
    compiler.options.output.jsonpFunction = `mfrename_webpackJsonp__${this.options.name}`
    // 2. Generate 4 dummy modules for spare
    this.genVirtualModule(compiler)
    // 3. Initialize the remote module mapping relationship in entry chunks
    // 4. Load all container initialization dependency collections (shareScopes) in entry chunks
    this.watchEntryRecord(compiler)
    this.addLoader(compiler)
    // 5. Generate mf entry file (usually remoteEntry.js)
    this.addEntry(compiler)
    this.genRemoteEntry(compiler)
    // 6. Intercept webpack compilation of remotes and shared modules
    this.convertRemotes(compiler)
    this.interceptImport(compiler)
    // 7. Make webpack jsonp chunk wait for remote dependencies to load
    this.patchJsonpChunk(compiler)
}

1. Generate unique jsonpFunction global variables to prevent conflicts

compiler.options.output.jsonpFunction = `mfrename_webpackJsonp__${this.options.name}`

2. Generate 4 dummy modules for spare

Just register these 4 file code modules as webpack virtual modules, which can be introduced and used by subsequent processes

3. Initialize the remote module mapping relationship in entry chunks

4. Load all container initialization dependency collections

Initialize all containers (other mf modules), and export the loading process as a promise to mark the completion of the initialization phase (all jsonp chunks need to wait for the initialization phase to complete)

module-federation/webpack-4/lib/virtualModule/exposes.js

5. Generate mf entry file (usually remoteEntry.js)

// 1. Add mf entry using singleEntry
new SingleEntryPlugin(compiler.options.context, virtualExposesPath, "remoteEntry").apply(compiler)

// 2. Copy the last file generated by the remoteEntry entry and rename it
entryChunks.forEach(chunk => {
    this.eachJsFiles(chunk, (file) => {
      if (file.indexOf("$_mfplugin_remoteEntry.js") > -1) {
        compilation.assets[file.replace("$_mfplugin_remoteEntry.js", this.options.filename)] = compilation.assets[file]
        // delete compilation.assets[file]
      }
    })
})

output container api（module-federation/webpack-4/lib/virtualModule/exposes.js）

`
  /* eslint-disable */
  ...
  const {setInitShared} = require("${virtualSetSharedPath}")

  // All exposed modules are preset here with dynamic-import
  const exposes = {
    [moduleName]: async () {}
  }

  // 1. Register the container globally in a global-like manner
  module.exports = window["${options.name}"] = {
    async get(moduleName) {
      // 2. Use code splitting to expose exported modules
      const module = await exposes[moduleName]()
      return function() {
        return module
      }
    },
    async init(shared) {
      // 4. Merge shares and wait for the init phase to complete
      setInitShared(shared)
      await window["__mfplugin__${options.name}"].initSharedPromise
      return 1
    }
  }

  `

6. Intercept webpack compilation of remotes and shared modules

Set aliases for remotes and shared modules, identify special paths, and forward them to a non-existing file path（Only non-existing file paths can be intercepted by resolver hooks and forwarded）（module-federation/webpack-4/lib/virtualModule/plugin.js）

const { remotes, shared } = this.options
    Object.keys(remotes).forEach(key => {
      compiler.options.resolve.alias[key] = `wpmjs/$/${key}`
      compiler.options.resolve.alias[`${key}$`] = `wpmjs/$/${key}`
    })
    Object.keys(shared).forEach(key => {
      compiler.options.resolve.alias[key] = `wpmjs/$/mfshare:${key}`
      compiler.options.resolve.alias[`${key}$`] = `wpmjs/$/mfshare:${key}`
    })

Intercept remotes, shared aliases, and forward to import-wpm-loader.js to generate code for requesting remote resources（module-federation/webpack-4/lib/plugin.js）

compiler.resolverFactory.plugin('resolver normal', resolver => {
  resolver.hooks.resolve.tapAsync(pluginName, (request, resolveContext, cb) => {
      if (is an alias from remotes, shared) {
          // Forward to import-wpm-loader with pkgName parameter
          cb(null, {
            path: emptyJs,
            request: "",
            query: `?${query.replace('?', "&")}&wpm&type=wpmPkg&mfName=${this.options.name}&pkgName=${encodeURIComponent(pkgName + query)}`,
          })
      } else {
          // request native module
          cb()
      }
  });
});

Generate code to request remote resources（module-federation/webpack-4/lib/import-wpm-loader.js）

module.exports = function() {
    `
    /* eslint-disable */
    if (window.__wpm__importWpmLoader__garbage) {
      // 1. Leave a code mark to identify the dependent remote module, which is used to make the chunk wait for the remote dependency to load
      window.__wpm__importWpmLoader__garbage = "__wpm__importWpmLoader__wpmPackagesTag${pkgName}__wpm__importWpmLoader__wpmPackagesTag";
    }
    // 2. When entering the code of this module, the remote module has been loaded, you can use get to get the synchronization value of the module, and return
    module.exports = window["__mfplugin__${mfName}"].get("${decodeURIComponent(pkgName)}")
    `
}

7. Make webpack jsonp chunk wait for remote dependencies to load

Use regular matching to the remote module that the jsonp chunk depends on, so that the chunk waits for the dependency to load
Make webpack jsonp load function support jsonp waiting to load dependencies（module-federation/webpack-4/lib/plugin.js）

Differences with webpack5

module-federation/webpack-4The plugin has implemented the main capabilities of module-federation, and can be found in webpack4 and webpack5 refer to each other , The following describes which parameters are not supported by the plugin

unsupported parameter

options.library

The priority of this parameter is not very high, the implementation in webpack4 is more complicated, and there are still problems in using it in webpack5, see for details "https://github.com/webpack/webpack/issues/16236" , Therefore, the implementation in webpack4 is similar to setting library.type = "global"

options.remotes.xxx.shareScope

The same mf container can only be initialized with one shareScope. If inconsistent webpack is set by using shareScope multiple times, it will report an error, and the shareScope can be set too much, which is confusing. Even in pure webpack5, the performance is unpredictable. It is recommended to use options. shared.xxx.shareScope, options.shareScope alternative

module-federation ecological package

The webpack-4 plugin has not yet integrated the ability of webpack-5 related packages（ssr、typescript、hmr、dashboard...）, However, 4 and 5 interoperability has been achieved, which can help you to use webpack5 to implement new projects without refactoring existing projects.

Supported parameters

options.remotes
options.name
options.shareScope
options.shared
options.exposes