Migrating a repo to an Nx monorepo while retaining history

Guilherme Tod — Thu, 12 Jan 2023 20:29:29 +0000

Why you should migrate and retain history

If you reached this article, it is likely that you are planning to either start using Nx monorepos or to migrate a an existing project into an existing monorepo.

If you're unfamiliar with Nx monorepos, make sure to check the many advantages it can bring you, including better code reusability, dependency maintenance and developer experience.

Now, if you decided that you're going to migrate your application into an existing monorepo, you might be wondering 1) how to do it the best possible way and 2) if it is possible and worth the effort to maintain your history at all.

The short answer is simply git. Git will allow you to pull your changes automatically instead of requiring you to manually copy and paste all your files within a folder. Also, your history will be almost entirely preserved with a minimal effort, which is an amazing benefit.

Another benefit of using git is continuity. We’ll get into more detail later, but after you create the first pull request for the migration, your teammates will review it, you may go into a deep QA process and many other things can happen between the PR creation and the actual merge and release.

Now, what if during this process a teammate needs to push a fix or an important feature on the original repo? If you don’t use git, you’ll probably be left with the single alternative of finding and replicating every change manually and potentially creating another entire process of review.

However, if you use git for the first step, you’ll see that you can later continuously pull the changes from one repo to another with the traditional git merge command. That not only speeds you up, but is also much more reliable in terms of getting the new code correctly.

Getting ready for the migration

Migrating an entire app to another repo can be a pretty impactful change. That means that your first deployment from the monorepo will naturally be a very hesitant.

Before you start the actual migration, there are a few steps recommended in order to have a smoother transition. The first deployment from the monorepo will likely be a very cautious one. After all, big changes like this always come with a bit of hesitance.

To minimize the changes that will happen during the actual migration, we can anticipate some steps to make sure that you break big changes into several and smaller steps that will allow you to focus into making sure every one of them gets completely ready and to solve potential issues gradually instead of dropping a massive change that can leave you with a pack of problems to solve.

Match dependencies within the repos

One of the most time-consuming tasks from the migration will potentially be matching dependencies. That is especially true if you have major version differences in significant packages, like your framework.

To do that, you can use a JSON comparison to see the differences between the two package.json files, specifically at the dependencies and devDependencies fields. Simply paste one file at each side and check the mismatched versions by taking semver’s resolution into account. For this step, you will not add any dependencies to either repo, as the goal here is to only make sure that both applications run correctly with the same dependencies.

After you make sure that all the dependencies are matching, do a deployment for the affected repos and keep an eye out on the next few days for new bugs that might be related to the upgrades while you work on the next steps.

Warning: While it is possible to manage dependencies with multiple versions within a repo, Nx strongly discourages it, so consider making an effort to match dependencies before attempting to keep different versions.

If possible, match all the code style

If you have lint checks running on CI/CD, you’ll eventually need to adapt the to-be-migrated repo to the new code style in order to be able to push the new code on the monorepo.

While it’s true that you can easily have a custom lint configuration for different projects within a monorepo, you likely want to follow a similar code style to guarantee consistency between your team. So while this is not mandatory, it is a good thing to consider. This step should take into consideration linting, formatting and other project specific settings like TypeScript strict mode.

Once more, it’s recommended to do this before you start with the repo migration to ensure that the migration is restricted to its purpose and does not include any other side effects.

This is also helpful for not running into conflicts when you continuously pull changes to the monorepo as you won’t have to deal with code style and formatting differences.

Raise all known issues

When you finally approach the moment of deployment, it is natural that your team will keep an eye out for any issues. This however may lead to teammates reporting existing issues as if they were related to the migration, which can lead to some tension by making others think that some bugs were introduced.

Also, it’s common that every other team member has a few issues and problems tracked only in their mind that didn’t make it to a real ticket that then gets known to the entire team.

To mitigate this, it is encouraged that you talk to the teammates to document all the small issues they can think of so that they are easily consulted by anyone before they raise their hands to report a bug considered to be caused by the migration.

Performing the migration

Creating the project

Once all the previous steps are done, it is time to work on the actual migration. This step is surprisingly easy since we’re using git to do the work for us.

We’ll start by adding a Nx project that will later hold the migrated code. Assuming you’ll be migrating an Angular app named single-app-repo, simply run:

nx generate @nrwl/angular:application single-app-repo

Note: always check in the docs for the generator to see the options available

You’ll get a new apps/single-apps-repo folder that contains a few configuration files at its root and a src directory, which will hold the implementation and contains an example app.

Bringing the code

With the project setup, we can start bringing in the original repo code. Assuming the repo that will be migrated is named inside the org your-company, run the following command:

git remote add single-app-repo github.com/your-company/single-app-repo

This will add a new origin that points to the existing repo, which you’ll use to perform the initial merge and to further update if that repo happens to have other activity prior to the first merge.

After this, we’ll merge the code from the original repo. Assuming the latest changes you want to pull are on the main branch, run:

git merge single-app-repo/main --allow-unrelated-histories

This will bring all your files to the root of the application and naturally result in some conflicts.

Solving the conflicts

You probably are facing conflicts in your package.json and package-lock.json/yarn.lock files. For these files specifically, you won’t be able to retain history as they will live at the same path on both repositories, so you can simply accept the current changes and discard the original repo’s changes.

For any other conflicting files, it’s worth looking at each and handling them separately as you consider whether they should stay at the root of the monorepo or should be sent to the project’s directory. In either case, it’s recommended that you accept the current changes, but if the second one is the answer, you should create the file manually inside the dir after you solve the conflict.

Now, lastly, you can take all the added files and move them to apps/single-app-repo. It’s important to adapt your structure to the Nx one, which means keeping the previously on root files inside the apps/single-app-repo and the actual source files inside apps/single-app-repo/src. If you already used Angular’s default structure, your work will probably be done by copying over the files to the indicated folder. Also, if you are creating an Angular app, it’s also required to adapt your angular.json file into the project.json file previously created by the NX generator.

After everything is done and moved, do a last check to see if the history is correctly maintained. If you’re on VSCode, you can use GitLens and Git Graph to have a better visualization of it.

First, stage all your files. Then, go to a file from the new project that you are confident that should have kept history and see if the blame is being correctly displayed. Lastly, open Git Graph and try to find commits from the original repo to ensure they are being carried over too.

If everything is well, you’re ready to finally merge with git merge --continue and start testing your brand new project.

Wrapping up

Clean up some of the project default files

You probably overridden most of the files generated by Nx initially with your own implementation. Still, it’s possible that some files were left over. They won’t affect your application, but it’s good for housekeeping that you take a last look and try to remove files. Look for nx-welcome file, a favicon that you won’t use and even loose and unnecessary .gitkeep files.

Say goodbye to the old repo

Now that you finished migrating the old repo, you can safely remove it from the remote with git remote remove single-app-repo.

Lastly, it might be a good idea to also tag your repository as deprecated and archive it to ensure no one keeps working on it.

Things to watch out for

After you perform the merge, you’ll need to start testing to see if everything is still working as expected. If your application is building and serving plus you performed the steps to avoid side effects, it’s safe to assume that your code will behave as expected since at the end of the day it’s the same code.

However, there are a few spots where it’s worth taking a careful look since they can get more easily affected by the repository change, so make sure you double check:

Environment settings
All sort of assets, including images, icons and etc
Localization files
Path references for other directories, including node_modules

If all of those are fine as well as your other tests, congratulations, you just migrated your entire repo to NX and can enjoy all of its benefits.

Bonus tip

If git isn’t correctly detecting a file’s history when you are continuously merging, abort your merge with git merge --abort and increase the renameLimit to make sure that git is more flexible for considering a file as renamed rather than moved. This is specially used if you weren’t able to keep the same formatting and linting and are now left with modified files due to these differences.

DEV Community: Guilherme Tod