I love monorepos! Or at least I love Meta's (Facebook's) monorepo, which is the only real monorepo I have ever worked with. Here is why I think it’...
For further actions, you may consider blocking this person and/or reporting abuse
Meta doesn't use Mercurial any more. The source control team at Meta made their own system called Sapling. It's based on Mercurial but as far as I know, a bunch of the core code was rewritten in Rust.
It uses a virtual filesystem called EdenFS, which uses FUSE on Linux/MacOS and ProjFS on Windows. The only repo files that actually exist on disk are the files for the part of the repo you're working on - other files are lazily downloaded by EdenFS from the source control server as needed.
An interesting thing about the Sapling client is that it supports both Git and Mercurial repos.
The system has evolved, yes. Sapling is yet another example of how our dev tooling teams are pushing the boundaries.
Interestingly, not much has changed from the developers' perspective - I am still using the same
hgcommands today as I did 5 years ago when I joined, even though so much has changed behind the scenes.I think of EdenFS as a counterpart of Virtual File System for Git Microsoft implemented when they tried to migrate their Windows codebase to git. Yes, it's a virtual file system (you learn immediated the first time you need to run
eden doctor:D)The simplest solution to that is to not have sensitive code only selected developers should be able to access :P
By the same token all code should be freely available on GitHub :D
Well, on a platform without ulterior motives, but ideally yes :)
Wow, on the one hand, I see the points of your argument. It's really challenging for me to wrap my brain around this point of view. This sounds like a reasonable approximation of hell to me. After a certain size, monorepos tend to make me feel like any action I take is more liability than asset. I'll have to consider it further.
I believe tooling is the crucial factor. I was part of the .NET Framework team when Microsoft, in the spirit of "eating your own dogfood", decided to migrate .NET Framework source control to the TFS (Team Foundation Server). When I showed up in the office at 8 am on the very first day after the migration, checking out a single file took about 15 minutes. Then, more people came, and checking out files was simply impossible. The .NET Framework codebase wasn't small but also wasn't extraordinarily large. When I work in Meta's monorepo, I don't feel like I am working in a huge codebase. I work on a subset of projects but can easily access all code via different means, be it VS Code, web tools, or command line.
I don't totally get how the versioning or history would work then. The problem is that if you develop a new feature for some time as a team, the whole codebase could have changed, because there are many other people working on it, too. So you can either just checkin your work all the time (unfished, so thousands of feature flags?) or you will have merge conflicts all the time.
So how is this solved then?
We check-in our work all the time. We do use feature flags but mostly for bigger features. Checking in features in progress is fine. It requires some planning and structuring your changes in a way that don't impact the existing code. Many features can be built bottom-up where there is only one integration point with the existing code that is often guarded by a feature flag.
Note that even though the codebase is huge and gets thousands commits per day these commits rarely touch your projects directly. There might be some that touch code you depend on but if they are introducing a breaking change the author must fix it or the tooling won't allow to merge.
if most commits (in other modules) do not touch your code, or your code mostly do not touch others, then mostly monorepo is not needed?
I guess it is similar to systems development/architecture, it is ok to start with a monolith, but when it has stabilized and grown it is better to break it down into modules/microservices or whatever.
It depends. Maybe this is the case if you can guarantee that you will never need to take a dependency on some code it is fine. Most Meta's services are built using the same underlying infra/packages. This helps avoid writing and maintaining the same boilerplate code for each service. As a result even if you don't touch this code (or even know about it), it is your dependency. If teams owning this code make a change all services get it automatically.
Grea writeup! I'm curious though, when at Facebook, what was the work/life balance like?
Thanks! I am glad you like the post.
I am still there and can't complain about the work/life balance. There are intense weeks sometimes, like everywhere, but this is rare - at least on our team.