Migration to a monolithic repository without the code freeze

We’ve recently migrated 10 projects to a single Voucherify git repository. Here’s a short memo on why we introduced the new approach and how we carried out the transition without blocking the feature delivery pipeline.

Why monolith

2 years of Voucherify development gave our code base enough time to outgrow a single project and become a handful of interconnected services. Naturally, in the course of building our platform, we’ve noticed a growing number of code used in several places — popularly called “commons”. So far, we’ve simply duplicated the code where it’s needed. This is clearly the fastest way and we chose it deliberately because our dev team iterates fast.

But there comes a time when duplicating and maintaining the commons becomes troublesome and actually slows down the iteration speed. The burden of remembering all the places where the util is used is just too high. Therefore we’ve brainstormed how to solve the issue and actually ended up with 2 solutions.

The first one is to create a module repository. E.g. wrap projects into NPM modules, introduce a versioning scheme, and set up a private module repository. Although this approach has some indisputable benefits, it’s not a 100% fit for our case. Here are the bullet points from our meeting:

With so many repos, somebody has to learn how to maintain (and do so) the NPM repo. With monolithic repo, no dependency management issue, no manual upgrading module versions
Code reviews become easier with monolith
Navigation in the source tree and projects becomes easier too
Testing and continuous delivery becomes easier — you know you have a consistent state at commit X

Basically, deciding to take the mono path is about having the “commons” issue solved and keeping the iteration pace at the same level. Additionally, we’re still a small, lean team — working on a single project as a matter of fact. That’s why there’s little chance that our developers will waste time on understanding irrelevant pieces of code. It’s quite the opposite, everybody should be watching every change. For the very same reason, we don’t have to introduce any access control measures.

So, we agreed it’s a good move and then we started wondering how to execute the transition in a way which doesn’t block ongoing development.

Migration process

The transition turned out to be super smooth. We’ve kept the commit history, even with the commit hashes unchanged. This was achievable with a single git command — git subtree. The procedure looked as follows:

Putting the repos into a single one.

git subtree add --prefix=project-a git@github.com:voucherify/project-a.git master

git subtree add --prefix=project-b git@github.com:voucherify/project-b.git master

...

Changing the deployment process so that it handles the new “commons” properly (changes in Dockerfile, Procfile, and shell scripts which I won’t be discussing in this article; if you’re interested, let me know in comments).
Updating the changes pushed in the meanwhile.

 git subtree pull --prefix=project-a git@github.com:voucherify/project-a.git master

 git subtree pull --prefix=project-b git@github.com:voucherify/project-b.git master

The ongoing work is being committed to developer branches on a particular repo (also, for the sake of simplicity we have merged all awaiting pull requests to master).
Migration of branches from multiple repos, an example for project-a repository.

#!/bin/bash
export BRANCHES=$(cat <<HERE
development-branch-1
development-branch-2
development-branch-3
development-branch-4 
HERE
)

for branch in $BRANCHES
do
    echo “============== copying ${branch} ============= “
    git checkout master
    git checkout -b $branch
    git subtree pull --prefix=project-a git@github.com:voucherify/project-a.git $branch
    git push -u origin $branch
    git checkout master
done

In the meantime, developers clone the new monolithic repository and reconfigure the IDEs correspondingly.
When a “merge master” gives a signal that he migrated all developer branches, developers can check out remote branches with the following command:

 git checkout -b branch_name origin/branch_name

That’s it, Voucherify runs on the monolithic repository from now on.

Summary & future enhancements

It’s been a month since we performed the migration. By getting easy-to-maintain utils at their fingertips, the team has increased productivity. They are also happy to have only a single instance of IDE running instead of 5 (browsing search results becomes a bit harder though). Now, the next improvement is to extract more “commons” that will make building features even faster: modules like commons-auth or other business-related stuff.

Originally published at www.voucherify.io.