TuanNQ

Posted on Sep 7

How I help a huge enterprise React project run dev 5x times faster

#vite #rsbuild #cra #webdev

TLDR: Please don't use CRA (Create React App). For small project, use Vite, for a huge enterprise project, use Rsbuild.

After 3 months stay at home chilling after finally graduate college, I passed the interview and was admitted into a huge corporation that primarily do the outsourcing job. (I was kidding, that 3 months waiting was mentally miserable!)

Then after a week of "learning" "corporation culture" and "study coding guidelines", I received the first serious assignment: coding the screen according to the documentation I was given.

The first step is clone and run the project. It was a huge enterprise 'monorepo' React project. After a config a tons of proxy networks because of security reasons, I spent 3 hours just to clone that project from a remote repo.

It basically consists of at least 7 small React applications (that I know of):

main_app_1 and main_app_2: this is 2 big parts of the application
This 2 big parts use the lib and common
lib and common in turns use components and ui_elements
components consists of more "advanced" elements and features that based on ui_elements
All 6 of this is then feed into container which is a Create React App project

Here's how you run the project, a more senior developer explained to me. Imagine 6 projects main_app_1, main_app_2, lib, common, components, ui_elements is like a "library". You first have to "build" this "library" into a dist folder, then the container will import and use from that to run the application.

The whole project was written in Typescript, so in order for the container to use it has to be compiled down to Vanilla JS using tsc (Typescript compiler) that bundle along with Typescript.

After spending another 3-5 hours to wire it all together with PNPM Workspace, and then waiting for an eternity to compile all of those gigantic project to Vanilla JS, I was finally able to run the whole project and start to get productive. Type pnpm dev in the console and start coding!

But there was just a small problem. Each time I wrote something I have to wait 30 minutes for it to appear on screen. If I was lucky. If I'm not lucky my computer will tell me "Out of memory" and I have to start all over again.

Even waiting for the dev server to start took about 5 minutes. And the bundle.js file it produces was so large that the browser spend another 2 minutes to load and process over it.

I requested more RAM and CPU for my poor computer. I was denied because technically I was "On Job Training".

This was crazy, I thought. There's no way anybody can develop in such an environment like this! Those more senior developers must all have a much more powerful machine than me. So I asked the more senior developer who helped me run the project how long he had to wait.
"Oh about 5 minutes", he said.
"So each time you write console.log() you have to wait 5 minutes for it to run, right?"
"Yeah, but obviously we'll write a bunch of code before save and wait again", he giggled.

That's fine for him, but what about me? In a gigantic insane React codebase with no documentation, how can I complete my assignments that forces me to use some obscure components if each time I console.log I have to wait 30 minutes? I calculated that each day I'll have a maximum of 16 tries to run my code. Finishing my assignment was impossible.

After seriously considering all my options, I decide to put the assignment aside and try to make the build dev process faster. I probably gonna lose my job anyway, so why not take a risk?

The compiling and Create React App bottleneck

Looking at the build process, I saw 2 major bottlenecks: tsc (Typescript compiler) and CRA (Create React App).

Each time I run the watch or build command, which uses tsc, I have to wait about 10 minutes for it to complete running. In fact, due to insufficient RAM, my watch command does not run at all! Despite I only watch 1 project (out of 6).

Why was it so slow? Turns out tsc was slow not because it was slow. It was slow because it has to meticulously check all types in project in order to generate the .d.ts file.

Next is the Create React App development server. It re-bundling 10.000 - 20.000 files again each time I add a new console.log().

Change tsc with swc

First, I dealt with the tsc bottleneck. Those .d.ts files are only necessary for one project to understand what types of another project.

But since the majority of time we developers just need to work on one project, re-generate all the .d.ts files of that project is unnecessary. When working in just 1 project, the Typescript Language Server (built-in in VSCode) will do all the type checking in the opened file for us.

Is there a way to just generate Javascript straight out of Typescript fast without worrying about types? After a bit of searching I found swc came to the rescue.

SWC is really fast. It's fast because it only compiles Typescript to Javascript without doing any of the type checking.

Compare to tsc took about 5 - 10 minutes just to compile, swc took about... 10 seconds to compile the whole project. While tsc watch does not work at all swc watch took only few hundreds miliseconds to compile the changed file!

Type checking is good and necessary but in this case speed is everything. When I focus on coding and trying your idea out, the feedback cycle more important than the "type correctness". But when I finish my implementation and ready to submit / push your code, that's the time to focus the "type correctness". So each time I create a PR I just need remember to run tsc again to check type and everything should be good.

The tsc replacement was relatively easy compare to next major bottleneck: replace the CRA (Create React App).

Replace CRA (Create React App) with Vite

When thinking about CRA (Create React App) alternative, the first option come to your mind is probably Vite. So am I. So I tried to replace CRA with Vite.

First, I build a simple CRA app in my computer and tried to replace it with Vite. To my amazement, it was quite simple to replace CRA with Vite. I don't have to change any code. It has only basically 3 steps:

Install Vite as a dependency
Create a vite.config.ts with the React plugin
Create an index.html file with the #root div in the root project instead of in public folder

With my small CRA project changed to Vite ran smoothly, I begin to work on the behemoth enterprise React app. First, I have to find and read and understand all of its config file.

After have a vague understanding of what config is needed to run that behemoth monster, I replicated it and found and install the equivalent Vite plugins. Next, I typed npm run dev in the console, hit Enter, and prayed.

The dev server did not even run. I was hit with this error:

Error: COMMITHASH is undefined

Why is that? The behemoth CRA app use git-revision-webpack-plugin so I install the equivalent of Vite which is vite-plugin-git-revision. But this 2 plugins does not work the same. In git-revision-webpack-plugin you only need to use the global COMMITHASH, but in vite-plugin-git-revision that global is GITCOMMITHASH.

I don't want to change any source code but only the configs, so I searched for configs of vite-plugin-git-revision in their documentation. But after quite a while I finally decided to write my own little custom config based on the source code.

Here's the diagram of my little custom config:

Here's the demo config:

import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react-swc'
import { exec } from "child_process";

const getCommitHash = new Promise<string>((resolve, reject) => {
  exec("git rev-parse HEAD", (err, stdout) => {
    if (err) {
      reject(err);
    } else {
      resolve(stdout);
    }
  });
});

const getVersion = new Promise<string>((resolve, reject) => {
  exec("git describe --always", (err, stdout) => {
    if (err) {
      reject(err);
    } else {
      resolve(stdout);
    }
  });
});

const getBranch = new Promise<string>((resolve, reject) => {
  exec("git rev-parse --abbrev-ref HEAD", (err, stdout) => {
    if (err) {
      reject(err);
    } else {
      resolve(stdout);
    }
  });
});

const getLastCommitDateTime = new Promise<string>((resolve, reject) => {
  exec("git log -1 --format=%cI", (err, stdout) => {
    if (err) {
      reject(err);
    } else {
      resolve(stdout);
    }
  });
});

export default defineConfig(async () => {
  const [commitHash, version, branch, lastCommitDateTime] = await Promise.all([
    getCommitHash,
    getVersion,
    getBranch,
    getLastCommitDateTime,
  ]);

  return {
    plugins: [react()],
    source: {
      define: {
        VERSION: JSON.stringify(version),
        LASTCOMMITDATETIME: JSON.stringify(lastCommitDateTime),
        BRANCH: JSON.stringify(branch),
        COMMITHASH: JSON.stringify(commitHash),
      },
    },
  };
});

I ran the development server again and prayed.

But I was hit with another error:

Error: process.env is undefined

Why is that? Turns out according to Vite, all environment variables must start with VITE_, otherwise Vite will not recognize it.

So, for example, in CRA environment variable is DEFAULT_LANGUAGE, in Vite it should be VITE_DEFAULT_LANGUAGE. Worse still, while CRA use process.env, Vite use import.meta. So I have to replace all process.env in the codebase with import.meta.

There's no way I'm gonna find and replace all process.env with import.meta in the whole codebase! If I do so each time I commit and push anything I'll have to be super careful and change it back to process.env all over again! That's unacceptable.

So after a quick search, I found a solution:

import { defineConfig, loadEnv } from 'vite';

export default defineConfig(({ command, mode }) => {
    const env = loadEnv(mode, process.cwd(), '');
    return {
        define: {
            'process.env.YOUR_STRING_VARIABLE': JSON.stringify(env.YOUR_STRING_VARIABLE),
            'process.env.YOUR_BOOLEAN_VARIABLE': env.YOUR_BOOLEAN_VARIABLE,
            // If you want to exposes all env variables, which is not recommended
            // 'process.env': env
        },
    };
});

Instead of using process.env which is unavailable, Vite find and replace all the string process.env.YOUR_STRING_VARIABLE in the codebase with the provided value. For example, if my code has process.env.DEFAULT_LANGUAGE, Vite will replace all of that string instance with the provided value (like "en").

So after meticulously find all the environment variables in that behemoth monster and put in to the config, my dev server finally run!

Contrary to CRA development server took about 10 minutes to start, the Vite development server start almost instantly. Though each time I enter a new URL into the browser, I had to wait 5 minutes for the page to load. That was quite long, I thought, but it is expected from running that behemoth monster.

But when I change a file, Vite HMR did not work! I had to stop and restart the development server again for the change to appear in the browser!

I tried to replicate the same config, environment, dependencies into the small test app in my computer but Vite works just fine. It seems that Vite HMR only died when dealing with such a behemoth monster of this scale.

Trimming 30 minutes waiting down the 5 minutes was a major improvement, but that's not good enough for me. Is there another alternative which is even faster that Vite?

After a quick search "Vite alternative", Reddit pointed me to Rsbuild.

Replace Vite with Rsbuild

Right from the get go Rsbuild promise you can consider it like a easy-to-replace, similar with Webpack based development tools like CRA,... Indeed, a lot of Webpack plugin works seamlessly with Rsbuild. The whole git-revision-webpack-plugin that I took hours to read, understand, debug and re-create can now just be used directly in rsbuild.config.ts.

Unlike CRA, Rsbuild use import.meta instead of process.env. But having to deal with that problem with Vite once, this is a piece of cake.

After copy all the environment variable config from Vite into Rsbuild, I started the dev server and prayed.

It works! After about 30 seconds, the development server started up.

I added a console.log('hi there') to a file. I held my breath. Is the HMR going to work?

4 seconds later, hi there appeared in the console. I added a <div>Hi there</div>, 5 seconds later, it appeared on the screen.

It felt like a miracle! Compare to CRA's start up time 30 minutes and constantly died because of insufficient RAM, Vite 5 minutes but HMR still not working, Rsbuild start up with only 20 - 30 seconds and HMR only 5 seconds feels like voodoo black magic forbidden quantum science come from the future.

According to the benchmark on the homepage, Rsbuild promise 3x faster than Vite + SWC. At first I was quite skeptical of that (how many "blazingly fast" claim did you hear?).

But in this real-world project from a real developer standpoint, it's not just 3x faster than Vite, it's 10 - 20x faster than Vite! I've never seen any project so amazing but so modesty in its claim.

I wish I could benchmark Vite and Rsbuild on this project and send to Rsbuild team as a testament of how good Rsbuild actually is... but I can't. As my company is a majorly outsourcing corporation, this project is confidential and belong to the customer.

I tried to search Google to find out why Rsbuild is so fast but can't find any result. If you know how Rsbuild works, why it's that so fast, please leave a comment, I'd love to know!

The irony aftermath

After about 3 - 4 days researched how to convert from CRA to Rsbuild, I finally can just complete the inane assignment in just 3 - 4 hours. Then I had to be "audited" by a Solution Architect who actually design the whole architecture of this behemoth monster. The purpose of the "audit" is for me to be judged whether I'm "ready to work" or not.

After answering all the inane questions that a Fresher / Junior developer like me suppose to answer, I surprised him by showing the 4 seconds wait time and all the diagrams above explaining how I do that. It took a while but finally he understand all the .d.ts, tsc, swc, CRA, Rsbuild, build processes, development processes ideas stuffs. He said he would "welcome any contributions".

1 - 2 weeks later, based on my solution, the somewhat more advanced by the Solution Architect with more colorful and flowery command line was put in for the whole huge FE team to use.

So what about my pay raise? Oh I'll continue to be a Dev I and have to wait until the end of the year to get a "performance review". Saving 5 minutes for each development cycle for each developers multiply by 20 ~ 30 devs must be a great financial saving for the corporation, I guess. Just wait until the end of the year and they'll consider give me a pay raise. I'm waiting each day everyday in the whole year for that.

But put all of the compensation and corporate's politics aside, here came the obvious question. Why nobody came up with a solution like mine? After all, if a random guy like me with a bit of tinkering can do this, why didn't all those "more senior developer" come up with any solution?

The answer has 2 parts.

Part 1 is that not a lot of people understand the build process and keep up-to-date with the current state of React tools. In fact, out of all "more senior" developers I asked, only 2 - 3 understand what they're doing. The others, despite already working for 1 - 2 years in this project, don't have a slightest idea of how all the setup work. This is expected.

But part 2 of the answer is totally unexpected. Turns out the waiting 5 minutes for build dev is totally not a bottleneck. It's fast. It's blazingly fast compare to all the time to just reading and understanding the Requirement document and following the corporation's procedures. For 7 - 8 hours of actual coding, you have to spend 1 - 2 weeks to understand what you supposed to code. The Requirement document is not written for human to understand, it's like law book written in Latin supposed to blame anybody-but-me when things go wrong. And it's often wrong and contradict with itself so you have to ask those write those Requirement document to "revise" it. And all the questions you ask him to "revise" it? Oh it has to be documented too.

From a standpoint of a guy who try to spend every minutes of waking moment to do something productive, this kind of inefficiency is wild. All for blame game.

But at least my client has a lot of money to spend though.

If you have any good job, please contact me. I'd love to know.