Monorepos, or the sedimentation of Escape across ages

This post is taken from an original article on Escape's blog

This article is the first of a two-part series. It will discuss theoretical guidelines and goals regarding the monorepo and its usages at Escape. The second article will focus on the concrete realization of these goals.

Introduction

Every team at Escape works in the same repository. We decided to follow the so-called monorepo workflow. This repository is our ground; its layers are our (concise) history.

This article describes how we structured this repository and gives a look back at how it evolved alongside the size of Escape as a company. Here is a list of things you will find in this text:

• Historical decisions we made that ended up being mistakes, why we made them, and how they impacted us over time.
• Guidelines employed to structure our monorepo now that the team is growing up.

The good old days

Initially, the team was tiny and made of less than five developers, all responsible for their own part of the stack. The monorepo was there primarily to make the management more effortless. Our entire development workflow happens in one place, whether we try to start the local environment, code, commit, have a green CI, or deploy.

Single install, a single command to start the local environment, is uniquely defined. Thus, it becomes possible for us to rely on the diverse expertise within the technical team.

In the end, with a small technical team, thanks to this monorepo, we managed to maintain a fully-fledged enterprise-grade technical workflow involving:

• multiple services
• continuous-integration and deployments
• complete local environments installed and started with a single command line

All that being said, at this point, our monorepo was there for the main reason of convenience. It did not impact how we write code, and this monorepo was holding several monolithic projects.

Monolithic here does not reference the microservice/monolithic architecture debate but rather the fact that there were no shared dependencies within the monorepo. Each project was a poetry or npm project. In a nutshell, our root-repository management was for everything not involving software-related code.

️Days went by, months, and Escape kept growing. A few months later, we were not three developers anymore, but 5, then almost 10. The code within every project has gone bigger and bigger. A few months ago, the codebase reached a point where it is painful to develop in this monorepo.

The projects have become too heavy, in terms of the amount of code, of complexity, making everything harder, with the following consequences:

• Longer installations Too many dependencies to install before being able to code on its part.
• Too much complexity It is challenging for a new developer to navigate the existing codebase and thus to be onboarded. It is harder for the maintainers to ensure non-breaking changes. It takes longer to ship new features in production (more time to get feedback during development and more time to ship it)
• Longer builds CIs too long
• We (the technical team as a whole) are tempted to think that we need more developers than needed when we would need more efficient workflows and architectures and to be trained around these. This is opinionated and much less factual, of course.

Solving the complexity of Escape

So, now, the complexity of the projects held by our monorepo is officially a concern. How do we solve it? We need to go a (or many) steps further in the philosophy of the monorepo.

Monorepos are supposed to hold many small projects, not a small number of significant projects.

Smaller projects mean many but small & parallelized builds, more but shorter CIs (fewer dependencies to install, less code to lint/test per project). It also means more reusability/testability of small portions of the code, inevitably leading to lighter unit tests.

This next part describes our now-targeted workflow for developing in a monorepo, where we try to leverage its advantages in terms of standardization and use it to write smaller projects.

The thesis of Escape's monorepo

The first interest in monorepos comes from the need for standardization. A good monorepo helps break down complexity if it meets these two conditions:

In the following section, easy means "accomplished with a single documented command."

1. It is easy to scaffold a new sub-project meeting all the company's requirements.
2. It is easy to link projects together as part of a shared-dependency system.

It is essential to understand that a monorepo is a company-wide decision. It involves consequences for the technical acts and how the team is organized.

Standardized project structure

Living up to requirement 1) requires that it is possible at Escape to start a new project in the monorepo quickly. Also, every project, as soon as it is created, should conform to these company-wide standards:

• Up-to-the-standard build configuration (using a unique-per-language across the monorepo, eventually-extended configuration)
• Same for linting and formatting (using unique-per-language, eventually extended configurations)
• Standardized IDE configurations
• Git hooks configurations if you intend to use them (we do ! See our article announcing Mookme)
• Testing out of the box with proper configurations, standardized coverage assessments

Everything above should be scaffolded in the project created with a single command. Starting a fully-fledged new project needs to be a 30-seconds business.

Dependencies management

Besides, ensuring 2) would mean the following:

• A single command is enough to declare that project A is dependent on project B
• This declaration should be enough to include project B in the build of project A (alongside other dependencies and only these dependencies)
• This declaration should be enough to trigger A's quality controls when B is updated
• When developing project A with something needing changes on project B, I only need to start a watcher on project B to trigger the watcher on project A for new changes.

Conclusion

Looking back at how our repository was formerly structured, it is easier for us to define how we want the technical of Escape to be structured. This led us to a set of rules that fit our situation as a company. Thanks for bearing with me during this small hike across the short history of Escape's engineering!

The next part of this series will focus on the practical challenges leaning around the concrete enforcement of this standard.

Follow Escape on Twitter, or subscribe to our newsletter for more human-readable developer experience & application security content!