DEV Community: Hieu Nguyen

How to Setup Monorepos with Git for JavaScript and TypeScript

Hieu Nguyen — Sun, 15 Aug 2021 09:28:49 +0000

When your app gets bigger, managing files within a project gets more complex. You may start to have modules shared between front-end and back-end projects. Often, you also need to manage different versions of those modules.

A monorepo is a way to structure your projects to manage that kind of complexity all in one place.

I failed to set up a monorepo with Lerna a few times. Yalc and Yarn Workspace can be troublesome when I need to move a project out of the monorepo.

Finally, I found a way to make it work using git submodules. Git is great for resolving code conflicts. Git branches can be used for versioning. You can have unlimited private repositories for free when using Github or Gitlab. Besides, with TypeScript or JavaScript (using webpack), you can configure module aliases to create beautiful import paths.

In this post, I’ll show you how to set up a dependency, structure your project, and configure module aliases for a monorepo. And discuss the disadvantage I’ve encountered of using this setup.

See git-monorepo-project on Github for the final result*

1. Setup a dependency

A dependency is a git repository. It can either contain a complete module (i.e with package.json and bundled/transpired JavaScripts files), or it may only have plain JavaScript or Typescript files.

Besides, we often need different versions of the dependency, known as versioning. Which allows us to make changes in a specific version, without affecting projects that use other versions.

Creating a dependency repository

You can create a public or private repository (make sure contributors have access), and push the code there.

Dependency versioning

For versioning, you can use branches. For example, using the main branch for the latest version, stable@v0.0.1 branch for the stable 0.0.1 version, and so on

2. Structure a project

The main idea of setting up a monorepo with git is to add dependencies (in step 1) as submodules.

In the project structure, a submodule is a local directory. Hence, we can easily import and treat them as a local directory. And because it’s a git repository, any committed changes will also apply to the copies in other projects (after pulling the changes)

Project structures

One way to structure your project is to have all dependencies under the src/packages directory. Here’s a project directory tree example:

project-root/
    ├── .gitsubmodules
    ├── package.json
    ├── tsconfig.json
    ├── webpack.config.js
    └── src/
        ├── index.ts
        ├── packages/
        │   ├── module1 (submodule)/ 
        │   │   ├── package.json
        │   │   └── src/
        │   │       └── index.ts
        │   ├── module2 (submodule)/
        │   │   └── index.ts
        │   └── ...
        └── ...

See the git-monorepo-project for example

Add a dependency

After creating a dependency repository, you can add it as a submodule using the git submodule add command, and store it under the src/packages directory. Here is an example:

$ git submodule add https://github.com/username/module-name.git src/packages/module-name

To add a specific version of the dependency, use the --b flag when adding the submodule. For example:

$ git submodule add -b stable@v0.0.1 https://github.com/username/module-name.git src/packages/module-name

Now, you can import the new dependency as a local directory. For example, import Module1 from “../packages/module1”;

Working from another computer

After setting up the monorepo, it’s easy to install a project or a dependency in another computer. It's useful when you have many workstations (ie. PC, laptop), or if you have someone working with you.

To set up the monorepo in another computer:

Clone the main project with the --recursive flag on the new computer. It will download the repository and all the submodules. For example: git clone --recursive https://github.com/username/main-project.git
Install node modules (if needed) using “npm install”

Now the project should be ready to work on!

3. Configure module aliases

A common problem when setting up a monorepo as above is that it results in ugly import paths. For example, the import path in the src/pages/dashboard/profile/ProfileMenu.tsx file will be "../../../packages/module1".

Luckily, you can set up module aliases for shorter import paths. Note: if you're using webpack to transpile Typescript, you'll need to set up module aliases for both JavaScript and Typescript.

Configure module aliases for JavaScript

You can configure the module alias for webpack in the webpack.config.js file, using the resolve.alias configuration. For React apps created with CRA, you can use react-app-rewired to override the webpack configurations.

For example:

module.exports = {
    …,
    resolve: {
        alias: {
            // import Module1 from “module1”
            "module1": "path/to/src/packages/module1",

            // this config allow importing any modules 
            // under src/packages directory
            // i.e import Module1 from “packages/module1”
            "packages": "path/to/src/packages",
            ...
        }
    }
}

See the webpack.config.js file for example

Configure module aliases for Typescript

You can configure module aliases for Typescript in the tsconfig.json file, using the compilerOptions.paths configuration.

For example:

{
    "compilerOptions": {
        …,
        "baseUrl": "./src",
        "paths": {

            // import Module1 from “module1”
            "module1": "packages/module1",
            "module1/*": "packages/module1/*",

            // this config allow importing any modules 
            // under src/packages directory
            // i.e import Module1 from “packages/module1”
            "packages": "packages",
            "packages/*": "packages/*",
            ...
        }
    }
}

Make sure the "baseUrl" (as above) is also present. It helps the compiler resolve dependency paths. See the tsconfig.extends.json file for example

Once you have set up repositories for dependencies, structured your project as above, and configured your module aliases - your monorepo is ready!

4. Disadvantages

I’ve been using this approach for over a year. Here are a few issues you could encounter, and how to deal with them.

Making dependencies

In case you’re trying to convert an existing project to a monorepo structure, it might take some time to set up. For example, separate some parts of the code and push them into their own repository.

But afterward, they are should be more independent, make it much easier to work with, or moving around.

Dealing with dependencies of a dependency

It’s quite common when you’re using a dependency, which depends on other modules. In this case, I’ll install them in the main project.

Let’s say Project-1uses Module-A, Module-A uses Module-B, and they all belong to the monorepo. And Module-B was added to Module-A as above. In this case, I’ll need to do the same for Project-1. This means adding Module-B s a submodule, and config the module alias.

Also, make sure the module aliases should be the same in both Project-1 and Module-A.

Takeaways

It’s often difficult to manage multiple projects and dependencies in a big app. A monorepo is a way to structure them all in a single repository, making it easier to work with.

Git provides submodules, branches, and the ability to manage code conflicts, which is useful for setting up a monorepo.

You can set up monorepo with git by separating each dependency into its own repository, then adding them as submodules. Besides, we get to configure module aliases to attain nice and readable import paths.

Thanks to Carl Poppa for proofreading and feedback.

Create a Glowing Loader with CSS and HTML

Hieu Nguyen — Fri, 16 Apr 2021 20:52:57 +0000

We want our products to leave a good impression on first-time users. So whenever we build something, we make it possible first, then ask if it can be a little unique. And the glowing loader is one of those attempts.

Here is the original version on jsfiddle (33 changes).

Create your own loader in HTML and CSS

The loader design has a container and 3 elements: the background button, glowing spinner, and the logo. While the button and the spinner are all overlay layers (use absolute positioning).

Let's start with a straigt forward HTML markup as following:

<div class="logo-container">
  <span class="spinner"></span>
  <span class="background"></span>
  <img class="logo" src="yourlogo.svg" height="28" />
</div>

1. The container

The main point of the container is to groups the elements together, and positioning its children logo in the center. Let's use flex in this example.

.logo-container {
  /* align children in the center */
  display: flex;
  align-items: center;
  justify-content: center;
  position: relative;

  /* a circle with 60x60 pixels */
  width: 60px;
  height: 60px;
  border-radius: 50%;
}

Now we have a plain logo.

Before going into the button background and the spinner. We need to make sure they are overlay layers, by applying absolute positioning as following.

.background,
.spinner {
  position: absolute;
  display: inline-block;
  height: 100%;
  width: 100%;
}

2. The button background

The button background is a circle, with a matte-ish gradient and shadows. Here's one way to do it

/* The button background layer */
.background {
  border-radius: 50%;
  background-image: linear-gradient(0deg, #0f1013, #252730);
  box-shadow: 0 4px 4px -1px rgba(0, 0, 0, 0.6),
    0 4px 6px 1px rgba(0, 0, 0, 0.3),
    0 1px 2px 1px rgba(0, 0, 0, 0) inset,
    0 18px 32px -2px rgba(255, 255, 255, 0.1) inset;
}

It'll look like this now:

3. Bring logo to front

In case you can't see your logo, you will need to use z-index to bring the logo on top. Simple as the following CSS:

.logo {
  z-index: 2;
}

4. Glowing spinner

Glowing spinner layer use a top-border attribute, with shadow to add the glowing effect. It can be done as following:

.spinner {
  border-radius: 50%;
  border-top: 2px solid #ae34db;

  /* glowing with shadow (30% of #ae34db) */
  box-shadow: 0 -5px 5px #ae34db4d;

  /* add spin animation */
  animation: spin 1s linear infinite;
}

Then add the animation keyframes. It's basically a keyframe that rotate the light spinner in a 360 degree circle

@keyframes spin {
  0% {
    transform: rotate(0deg);
  }
  100% {
    transform: rotate(360deg);
  }
}

You should see a glowing spinner with your logo inside. Here is our final result.

You can play with the spin keyframe to change the glowing colors, or spread the shadow more to make it like a siren light.

Now you know how to create a glowing loader. It contains a matte-ish gradient background, with a glowing spinner. You can check out the final result on jsfiddle;

Let us know what you think and thanks for the support.

A simple mistake that leaks 20M users' data

Hieu Nguyen — Mon, 22 Mar 2021 09:07:27 +0000

It’s one of the silliest mistakes you can make. But who knows. When you get busy, things can happen.

A few days ago, I was on this page — a social platform, with 20M users. Wandering on a user profile, then I opened his followers page. Must have been switching between debugging my website, out of curiosity, I had a look at a XHR data request on their website.

And baam, the request contains a list of users, with personal information. Also the access tokens from their social platform accounts.

With those access tokens, hackers use them to gain control over social accounts. I guess a large portion of those tokens is still valid. Besides, personal data with contact are valuable. Many companies want them for cold calls/emails, risking users’ privacy.

For 20 millions users, this issue is serious.

There is not a good explanation for this mistake. The tokens doesn’t look fake. Maybe, their team built the API for convenience, without being careful enough.

Prevent this mistake as a developer

You can prevent this mistake by following Principle of least privilege. Which limiting permission for users to perform any action, only allow what they have to, no more. And besides:

Store sensitive info in a different table
Build API with single purpose (i.e don't use 1 endpoint to get user info, and their sensitive data)
Always validate if request has the right permission

Especially, double check if you have a page that list users (i.e list of followers). And prevent hackers craw your APIs (i.e using rate-limit, IP blacklist)

Some afterthought.

Using social platform for authentication (signup, login) is still dangerous. Even though the platform itself is secured, a few apps you connected to are secured. But if one app leaks users’ access token, that’s it.
Maybe passwordless authorization (login with a magic-link) is a near and brighter future. Unless you use the same password everywhere, and one of your online shopping being hacked. Or worst, that online shopping doesn’t securely store passwords.
Sometimes rushing to deliver a project can cause mistake like this :P. Also make it easier for anyone to report incident like this.

P/s: I've contacted the team to fix the issue mentioned above. A friendly reminder to check double your API with sensitive data. We’re all busy, mistakes can happen.

Not Another Productivity Hack

Hieu Nguyen — Tue, 13 Oct 2020 12:48:55 +0000

When I was younger, most of my time spent writing code, with 5 hours of sleep a day. I often read about productivity hacks, hope to squeeze a little more works. But nothing really works.

Having an unhealthy lifestyle drained my energy. In the past year, even if I wanted to, I can't do it anymore. I need more breaks, more sleep, and more time away from the projects.

Turn out, it was for the better.

Most personal productivity hacks are to kick-start some unused parts of our energy. But when you have already drained them out, there is no more energy to kick-start. You're just working too much.

My change of habit helps me feel much better.

Almost every day, I sleep 7:30+ hours, often have a noon nap for a productive afternoon. I started to eat on time, 3 meals a day. For the last year, I exercise after work, at least 5 times a week, 30 mins each.

In case you're drained out, and looking for productivity hacks. Maybe you don't need one. Maybe all you need is to take care of yourself better.

Sleep more, eat more, be healthier

Posted on Not Another Productivity Hack — Hieu's Blog

Fixing lazyload images with React Server-Side Render

Hieu Nguyen — Sat, 10 Oct 2020 03:04:21 +0000

I've recently realized my server render pages has a problem. The offscreen lazyload image don't show up.

In case you don't know, offscreen images won't load until it shows up onscreen. It helps to reduce unecessery loading to improve performance.

As usual, I impletemented image lazy loading by (1) get all img elements, (2) move photo URL to data-src, and remove src. Then (3) observe whenever each of them scrolled onscreen, move the photo URL back to src to load.

It works when React running on the client-side. But when render on the server-side, the images don't show up.

Turn out, it's a little different. As usual, the server will return a hydrated React page. Then the client will re-render the page again as a normal React page.

The problem was, the lazyload function executed twice as well. The first time, src is a link, and set to empty after assigning the link to data-src. Then the second time, src was empty, and that empty value is again, assigned to data-src. And when users viewing the page, it will start to load an empty value.

I fixed this problem by checking if src is not empty, before moving it to data-src.

In case you're looking for the code, here is a basic lazyImages.ts with some favors (add loading class, check for offscreen images)

You can use the lazyImages as following:

lazyImages();

// offset before the image is scrolled into view
lazyImages({ rootMargin: "100px" });

Posted on Fixing lazyload images with React Server-Side Render