Make bulk changes over multiple repositories

With the increasing popularity of microservice architecture and with many organizations having plenty of internal libraries, it's easy to come to a state where keeping all repositories in sync might be hard. Some have opted to move to monorepos, which have some advantages, but also disadvantages which makes many organizations not willing to, or unable to adopt this approach.

One of the biggest problems with keeping separate repositories might be that smaller changes, such as a dependency update, or a change in a file like the pull request template might require a lot of tedious manual labor. But this should not have to be the case, we should be able to automate these tasks, even if they are spread out over multiple repositories.

Introducing multi-gitter

The solution to the problem of making similar changes over multiple repositories is multi-gitter. It allows you to make changes by running a script or program, in the context of all repositories in your GitHub/GitLab organization, or in a list of repositories. For all repositories where a change was made, a pull request will be created. All pull requests can then be verified to be correct by a Continuous integration pipeline and then potentially merged with another part of the tool.

Script it once, run it everywhere

The script can do any type of change. As long as you can script something for one repository, you can do the change for any number of repositories.

Example run, replacing apples with oranges using bash:

Your changes, your tooling

multi-gitter is agnostic to the tool you use to make these changes. A simple bash script might be enough for some of the times. But if you prefer to modify your file with a Node.js or Python script, that works excellently too.

Example: Replace the the pull request template file (if it exist)

#!/bin/bash

# Absolut path to the file that should replace the one in the repo
REPLACE_FILE=~/test/pull_request_template.md
# Relative from any repos root
FILE=.github/pull_request_template.md

# Don't replace this file if it does not already exist in the repo
if [ ! -f "$FILE" ]; then
    exit 1
fi

cp $REPLACE_FILE $FILE

Run the script with multi-gitter:

multi-gitter run ./replace.sh -O my-org -m "update pr-template" -B update-pr-template

Example: Replace text in a file with Node.js

const { readFile, writeFile } = require("fs").promises;

async function replace() {
  let data = await readFile("./README.md", "utf8");
  data = data.replace("apple", "orange");
  await writeFile("./README.md", data, "utf8");
}

replace();

Run the script with multi-gitter:

multi-gitter run "node $PWD/script.js" -O my-org -m "replace apples with oranges" -B fruit-replace

Many ways to run

Since any type of script can be run to modify the repositories, multi-gitter does also need to accommodate for many ways of running that code, which it does. It does allow you to run the script in any collection of repositories, like everything in an organization, by specifying a list of repositories, all repositories owned by a user, or a combination of these. If you want to speed up the process, you can make the run parallel, or you might want to specify reviewers of the created pull requests if needed.

One of the most common extra flags used is the --dry-run flag. With it, you can try out the script, without actually making any changes. And by setting the log-level to be a bit more verbose with --log-level=debug you can see every single line that would have been changed as a git diff.

Try it for yourself

The best way to understand any tool is to play with it out yourself. You can find multi-gitter here to try it out. Maybe even help others by adding an example script that was useful to you.