DEV Community: Matan Assa

Dynamic remotes with ModuleFederation

Matan Assa — Mon, 15 Aug 2022 06:48:15 +0000

We use ModuleFederation extensively on Autodesk Construction Cloud.
Around 30 single-page-applications that are transforming into federated remotes that will later on be consumed by a single host that is still forming.

In this post I will cover why you would want to dynamically load federated modules, and share some creative ways I found to implement it.

Let's start by understanding how remotes are set and loaded traditionally using ModuleFederation.

Traditions 🤷

Traditionally, ModuleFederation allows to set remotes during build-time:

new ModuleFederationPlugin({
  name: 'host',
  remotes: {
    remoteA: 'remoteA@https://example.com/remoteA/remoteEntry.js',
  },
  shared: {
    ...
  },
});

While this will work for most use cases, questions will start to rise once you add another environment to the equation.

If we want to support more than one environment, for an example Staging & Production, the traditional way won't be enough for you.

You can easily build a different output for each environment and well, problem solved.
But in our case, due to technical restrictions, we couldn't do it.

🚨 Personal opinion 🚨

In general, I believe code should be agnostic to it's environment, the more it's aware, the more complexities you add for different environments. But that's for a different blog post.

Before we dive into solutions, we need to understand how ModuleFederation is loading remotes behind the scenes.

How remotes are loaded? 🤔

The URL passed for each remote in the remotes object is a direct URL to the remoteEntry file.

new ModuleFederationPlugin({
  name: 'host',
  remotes: {
    remoteA: 'remoteA@https://exmaple.com/remoteA/remoteEntry.js', // <-- this
  },
  shared: {
    ...
  },
});

The remoteEntry.js file contains references to the exposed modules of an application. It doesn’t contain the code for those modules, but it knows where to go get the code. That’s important because it allows those modules to be lazily loaded via import() if that is how you choose to load your code.

Once your app loads, a script tag is created and appended to your <head> tag.

It would look like this:

<html>
  <head>
    <script src="https://example.com/remoteA/remoteEntry.js" type="text/javascript"></script>
  </head>
</html>

When the script finished loading, the remote would become available on your window via window.remoteA.

Then, when you lazily load a remote module, this executes behind the scenes:
window.remoteA.get('./remoteComponent')....

Now that we understand how remotes are loaded, we can start to think of ways to mimic and improve it.

Promise based dynamic remotes 💡

Back to business, we're trying to find a way to load remotes dynamically.

Well, turns out there's a magical feature that allows us to pass a promise (kind of) to the remote instead of passing a static url as shown in the example above.

That means we can add a layer of logic that'll "decide" where to load the remote from.

Implementation 👨‍💻

Let's start with creating a remoteLoader.js file in our host app.

That remote loader will be used later on to dynamically load our remotes.

Here's the code:

const remotes = {
  remoteA: {
    local: 'https://localhost:3000/remoteEntry.js',
    qa: 'https://qa.example.com/remoteA/remoteEntry.js',
    staging: 'https://staging.example.com/remoteA/remoteEntry.js',
    production: 'https://example.com/remoteA/remoteEntry.js',
  },
};

const remoteLoader = (scope, remoteURLs = remotes) => {
  return new Promise((resolve, reject) => {
    if (!window[scope]) {
      const existingRemote = document.querySelector(
        `[data-webpack="${scope}"]`
      );

      const getRemoteURL = () => {
        if (window.location.hostname.includes('local'))
          return remoteURLs[scope]['local'];

        if (window.location.includes('qa')) {
            return remoteURLs[scope]['qa'];
        } else if (window.location.includes('staging')) {
            return remoteURLs[scope]['staging'];
        } else if (window.location.includes('example.com')) {
            return remoteURLs[scope]['production'];
        }
      };

      const remoteEnvURL = getRemoteURL();

      const onload = async () => {
        // resolve promise so marking remote as loaded
        resolve(window[scope]);
      };

      // if existing remote but not loaded, hook into its onload and wait for it to be ready
      if (existingRemote) {
        existingRemote.onload = onload;
        existingRemote.onerror = reject;
      } else {
        const script = document.createElement('script');
        script.setAttribute('data-webpack', scope);
        script.async = true;
        script.type = 'text/javascript';
        script.src = remoteEnvURL;
        script.onload = onload;
        script.onerror = () => {
          document.head.removeChild(script);
          reject(new Error(`[${scope}] error loading remote: ${remoteEnvURL}`));
        };

        document.head.appendChild(script);
      }
    } else {
      // remote already loaded
      resolve(window[scope]);
    }
  });
};

export default remoteLoader;

Let's break it down.

Initially, the code is checking if the remote is already loaded, that's important since we don't want to load a remote that's already there. (Duh 💁‍♀️)

Then, we detect the current env according to the hostname.
Yes, we can access the window since the code is lazily ran during run-time!

Once we have the URL, we build a script element and append it to the document head tag, and resolve the promise once the script has finished loaded.

How do we use it?

We're gonna do something pretty cool. NGL.

First, we'll expose the remoteLoader file as a remote from our host.

new ModuleFederationPlugin({
  name: 'host',
  remotes: {
    remoteA: 'remoteA@https://exmaple.com/remoteA/remoteEntry.js',
  },
  exposes: {
    './remoteLoader': './utils/remoteLoader',
  },
  shared: {
    ...
  },
});

We'll use that remoteLoader as a runtime module to fetch the remotes.
We need to keep in mind that in order to use it we need to make sure the host's remoteEntry file has finished loading.

To make everything work, we have one last step and that is to create a stringified promise that we'll pass to the remote.
The promise will:

Wait for the host's remoteEntry file to finish loading
Get the remoteLoader from the hosts exposed modules
Run the remoteLoader and resolve.

Here's the code:


const getRemoteLoaderPromise = (scope) => {
  return `promise new Promise((resolve, reject) => {
    let intervalId;

    const waitForHostToLoadAndExecuteRemoteLoader = () => {
      if (!window.host) {
        return;
      }

      clearInterval(intervalId);

      window['host'].get('./remoteLoader').then((factory) => {
        const remoteLoader = factory();

        remoteLoader.default('${scope}').then(resolve).catch(reject);
      });
    }

    intervalId = setInterval(waitForHostToLoadAndExecuteRemoteLoader, 10);
  })`;
};

new ModuleFederationPlugin({
  name: 'host',
  remotes: {
    remoteA: getRemoteLoaderPromise('remoteA'),
  },
  exposes: {
    './remoteLoader': './utils/remoteLoader',
  },
  shared: {
    ...
  },
});

Our code will have an interval that waits for window.host to become available with its modules.
Then, we'll leverage the exposed remoteLoader and use it to load remotes dynamically.
That's it!

After you put together all the pieces, you'll be able to use top-level imports for your remotes.

For an example:

import React from 'react';

const MyRemote = React.lazy(() =>
  import('remoteA/App')
);

const MyComponent = () => {
  return (
    <React.Suspense>
      <MyRemote />
    </React.Suspense>
  );
}

export default MyComponent;

Conclusion 😴

There are many approaches to implement dynamic remotes with ModuleFederation.
Honestly, I think I tried all of them.
This one was the most advanced way as it allows to have an actual JavaScript file that we can lint, test, and easily maintain.

Besides my use case, you can use the remoteLoader technically for everything.
Running code before loading remotes, fetching a manifest, communicating with a DB, tracking analytics, monitoring, you name it.

If you have any ideas on how to improve or share a different approach you took, let me know in the comments!

Resources

Local development with non-local federated modules

Matan Assa — Thu, 08 Jul 2021 07:16:12 +0000

Does complex usages require complex solutions? (spolier alert...)
In this case it doesn't.

We have multiple frontend applications, that are being deployed to three different environments (qa, staging and production) and each application is being served under a specific subpath (i.e: example.com/app1, example.com/app2) using CloudFront, Lambda@Edge and S3.

We are sick of full page loads when navigating between applications and I'm gonna cut to the chase here - we are trying to use module federation to have one host application which will act as a skeleton, and several remotes which acts as the content. In simple terms - microfrontends.

Ideally we want to be able to work with non-local remotes while in local development of the host application because we don't really want to run more than 15 applications everytime we work locally.

Problem is, until now each application had a fixed, relative publicPath, and now when we load the remote application in our local host application all the assets are relative to the wrong domain (the local one, that belongs to the host) causing all the assets to be fetched from the wrong place.

The problem in simple terms

Let's say we have an application, called app1 and it's deployed to QA environment.
As I previously mentioned, it's common sense that we want to be able to use the remote from QA locally when working on our host application but the relative publicPath that is set on the remote is causing trouble when it's being loaded on a local host - being relative to the wrong domain and failing to fetch the correct files.

We need to be able to modify the publicPath of a remote during runtime so that it includes the domain, and the subpath that the application is being served on.

Wait! There's more.

We want this to happen only when a local host is using a non-local remote, publicPath works just fine in all the other scenarios. (this is our insurance that we don't modify the publicPath when we don't need to)

The solution in simple terms

One would say you could solve it by creating a separate build for each environment, but that's not as futuristic as module federation, so no thank you.

Based on the circumstances, a custom webpack plugin that will accept the subpath of the current application, and will detect during runtime whether it's being rendered inside a local host and will modify (during runtime) the publicPath with the full domain and subpath, sounds better.

tl;dr

// DynamicPublicPathForRemoteInLocalHost.js

module.exports = class DynamicPublicPathForRemoteInLocalHost {
  constructor(props) {
    if (props.pathPrefix) {
      this.pathPrefix = props.pathPrefix;
    }
  }

  apply(compiler) {
    compiler.hooks.make.tap('MutateRuntime', (compilation) => {
      compilation.hooks.runtimeModule.tap('MutateRuntime', (module) => {
        const isPublicPathRuntimeModule =
          module.constructor.name === 'PublicPathRuntimeModule';

        if (!isPublicPathRuntimeModule) {
          return;
        }

        const [key] = module.getGeneratedCode().split('=');

        // This function mutates publicPath in runtime when remote is used in local env.
        // Without this plugin, when running host in local and importing remotes from QA, remote files are mistakenly being fetched from local instead of QA.
        module._cachedGeneratedCode = `${key}=(() => {
          if (window.location.host.startsWith('local') && !document.currentScript.src.includes('local')) {
            return new URL(document.currentScript.src).origin + '${this.pathPrefix}';
          } else {
            return '${module.publicPath}';
          }
        })();`;
        return module;
      });
    });
  }
};

And here's how you'd invoke it in your webpack config:

module.exports = {
  ...
  output: {
    publicPath:
      process.env.NODE_ENV === 'local'
        ? 'https://localhost:3001/'
        : '/app1/',
  },
  plugins: [
    ...
    new DynamicPublicPathForRemoteInLocalHost({
      pathPrefix: '/app1/',
    }),
  ],
};

Next time you'll try to use a non-local remote locally, the custom webpack plugin will detect that and will modify the publicPath to be absolute, therefore fixing the issue with relative publicPath.

What happens if you won't? absolutely nothing!
The custom plugin will fallback to whatever publicPath you set in your webpack config if it doesn't detect the local scenario, which is a local host loading a non-local remote.

Have a better solution? I will be more than happy to hear in the comments and hopefully make our design even better.

Resources:

This article is based on several github comments and some articles, all combined helped achieve this solution that suits our specific design.