Want to develop and share smaller modules between repositories? Here are some useful options.
Sometimes you need to have code from one repository inside another repository. This usually creates a problem, since Git manages source code and not dependencies. As a result, a whole set of tools were created to support a code-sharing development workflow on a more modular level, aspiring to bridge the gap between the Git Repo SCM and the sub-repos within it.
Git Submodules allow you to keep a Git repository as a subdirectory of another Git repository. In theory, this lets you clone another repository into your project and keep your commits separate. Why is that useful? because it lets you use another project from within the project you’re working on.
However, if you run a quick Google search for Git submodules, the results will not be positive. This is because of some major drawbacks around git submodules, such as being locked to a specific version of the outer repo, the lacking of effective merge management, and the general notion that the Git repository itself doesn’t really know it’s now a multi-module repository.
Git, at its base, also isn’t built to handle dependencies and relationships between components. The workflow around code-sharing, therefore, becomes complicated, and Submodules are struggling to deliver our desired workflow. In mercurial, subrepositories are named “feature of last resort” to be avoided.
In this post, we’ll suggest and review useful alternatives to Git submodules to help develop shared multiple components across projects and in a repository. Feel free to give them a try, and add your own suggestions in the comments.
1. Bit (for JS)
Bit is an open-source extension to Git that handles both source-code and dependencies across different projects and the granular component level.
As a result, it creates a workflow for developing many smaller components/modules within a repository, while being able to share them in other repositories with full control over code changes and dependencies.
You can point Bit to files and directories in your repository you would like to use in other repositories, and it will start tracking their source-code while automatically running through the code and defining all dependencies.
Then, the components can be tagged with a version and exported to a remote hosting collection (preferably via bit.dev), from where they can be imported into other projects- where you can also develop them and make changes.
Since Bit tracks the code and manages the dependencies across the projects, you can easily merge code changes between the different repos and update the dependency graph across your entire codebase.
The result is twofold:
You can easily manage multiple modules within a repository while Bit handles the dependency relations between them. When you change a module and update its version, you can run bit status Bit will notify you that the module is changed and ask if you’d like to update dependents. Bit even lets you develop modules with different environments within the same repository, for example, TS inside a JS repository.
You can share these modules in multiple repositories while Bit will extend Git’s workflow to control and sync changes between them, and easily update your entire dependency graph across repositories.
If working with bit.dev, you’d also get a UI for discovering shared components, collaborating and even playing with examples online. Take a look.
2. Git subtrees
](https://res.cloudinary.com/practicaldev/image/fetch/s--vS7qND5m--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://cdn-images-1.medium.com/max/2000/1%2AqBgXG2ckpppn9fvJuMytGQ.png)
Source: Atlassian blog
An “experimental alternative to the git-submodule command” which “Merges and splits subtrees from your project into subprojects and back”. Subtrees allow subprojects to be included within a subdirectory of the main project, optionally including the subproject’s entire history. A subtree is a subdirectory that can be committed to, branched, and merged along with your project in any way you want. The idea of the subtree merge is that you have two projects, and one of the projects maps to a subdirectory of the other one and vice versa. When you specify a subtree merge, Git can figure out that one is a subtree of the other and merge appropriately.
As opposed to submodules, subtrees’ sources files are stored in the repo. It’s not just a link, the code is really there. There’s also fewer steps required and fewer changes to the workflow, so many find this option preferable.
3. Git slave
Gitslave is a “script for coordinated version control of large projects combining code from multiple independent repositories using Git”. It provides a wrapper around Git that manages a tree of directories with multiple “slave” repositories in the directories. Meaning, your Git command line will be able to control all “slave” repositories from the “master” repository itself.
This can be useful when developing different sub-projects within a reposiotry and in need of consolidating the workflow and updates between them.
The wrapper runs your Git commands on each of the repositories and combines the output for all of them. Gitslave is an added supplement and aside from one new file in the superproject, adjustments to .gitignore, and perhaps a few private config variables, does not affect your repositories.
4. Lerna (for multi-package JS repos)
“Ahhh! Lerna doesn’t manage source code! It’s not a tool for managing repos in another repo!” — Yes, you are right. It does not manage source code. It will not consolidate the Git workflow for sub-projects in your repository. You won’t be able to control or merge changes to sub-projects using Lerna, or extend any other source-code management workflow for sub-directories in the project or between different repositories. Still, JS developers can use Lerna to achieve some of the goals aimed to be achieved with Git Submodules.
If your use case is right, Lerna can prove to be a useful way to develop and publish modules from a single repository. You can use Lerna to refactor your repository into a multi-package repository that hosts different modules in different directories in the repository, leverage Lerna’s features to automate the versioning and publishing of the modules, and get the job done.
Each module will be set up with all the configurations of a standalone package, including the package.json file and everything else. So, when your ultimate goal is to create a JS repository in which you can develop multiple packages and publish them, Lerna comes in handy as a Git-Submodules alternative in the sense that it makes this workflow practical.
Honorable mentions
Add more in the comments… Cheers 🍺
Medium
Oldest comments (7)
git submodules are simply horrible - great post!
Subtree is my preferred way of solving this issue!
Very useful, thanks for writing!
Never saw git subtree before. Could be a great solution. Thanks!
We use submodules very effectively but you need a strict Dev team and a pipeline to perform the merge and very good separation of concerns.
Hey! Can you elaborate on this? Is there a blog post maybe already?
If working on c/c++ project and using cmake as a build script then the new FetchContent method work as a charm. Main feature is to handle 2 dependencies to the same project. This will only fetch once the required project and not twice like submodule would do.