Introducing tsrc

Introduction

Handling source code versioning in a software company is challenging. You have to decide how to organize your sources.

The first method is to put everything in a giant repository.

The second method is to split the sources across multiple repositories.

Both methods have their pros and cons.

With the "giant repository" approach you sometimes cannot use existing source control software like git, because they do not scale enough for very large projects, or you have to make your own patches, like facebook does with Mercurial.

With multiple repositories it gets easier to just use git as usual, but you'll likely need a tool on top of it so that working with multiple repositories is easier.

A popular solution for the second case is to use git submobules, but:

• You need a 'master' repository on top of the workspace

• When you update a submobule, you have to make a commit in the parent repository too, and this step is easy to skip.

Also, we found out that to make sure all the repositories are in a consistent state, we could simply push the same tag on several repositories.

tsrc

Enter tsrc. We use it everyday at Tanker to manage our sources.

It has a nice and intuitive user interface that takes care of running git commands for you in multiple repositories.

It also features (optional) commands to interact with GitLab.

Let's see how it works.

Note: we will only be showing the basic usage of some tsrc commands.
You can use --help to discover all the available options.

Installation

tsrc is written in Python3 and can be installed with pip:

# Linux
$ pip3 install tsrc --user
$ Add ~/.local/bin to PATH

# macOS
$ pip3 install tsrc --user
$ Add ~/Library/Python/3.x/bin to PATH

# Windows
$ pip3 install tsrc
# PATH is already correct, set by Windows at installation

You can find the sources on github.

Usage

Cloning the repositories

tsrc is driven by a manifest file that contains the names and paths of repositories to clone.

It uses the YAML syntax and looks like:

repos:
  - src: foo
    url: [e-mail]:acme/foo

  - src: bar
    url: [e-mail]:acme/bar

The manifest must be put in a git repository too. You can then use the following commands to create a new workspace:

$ mkdir ~/work
$ cd work
$ tsrc init [e-mail]:acme/manifest.git

In this example:

• foo will be cloned in <work>/foo using [e-mail]/acme/foo.git origin url.
• Similarly, bar will be cloned in <work>/bar using [e-mail]:acme/bar.git

Making sure all the repositories are up to date

You can update all the repositories by using tsrc sync.

• The manifest itself will be updated first.
• If a new repository has been added to the manifest, it will be cloned.
• Lastly, the other repositories will be updated.

Note that tsrc sync only updates the repositories if the changes are trivial:

• If the branch has diverged, tsrc will do nothing. It's up to you to use rebase or merge
• Ditto if there is no remote tracking branch

This way, there is no risk of data loss or sudden conflicts to appear.

(By the way, this is a good example on how to implement this directive from the Zen of Python: "In the face of ambiguity, refuse the temptation to guess".)

So that you know where manual intervention is required, tsrc sync will also display a summary of errors at the end:

Managing merge requests

Since we do most of our operations from the command line, it's convenient to be able to do GitLab operations from the shell too.

We leverage the GitLab REST API to create and accept merge requests.

For instance, here is how you can create and assign a merge request:

# start working on your branch
$ tsrc push --assignee <an active user>

When the review is done, you can accept it and let GitLab merge the branch once the CI passes with the following command:

$ tsrc push --accept

Note how --accept will not merge the pull request immediately. This is by design. We believe that continuous integration is only worth it if it prevents bad code from landing into master, thus we make sure you cannot by-pass the CI.

Other goodies

tsrc status

You can use tsrc status to quickly get an overview of the status of your workspace:

tsrc foreach

Sometimes you just want to run the same command on every repositories.

tsrc has you covered:

$ tsrc foreach -- some-command --some-opts

(Note the -- token that separates the options of some-command from the options of tsrc)

tsrc log

If you have multiple repos, chances are you are going to use several of them when doing a release, so you'll probably end up putting the same tag on several repos.

Thus, you may want to quickly get an overview of everything that changed between two tags.

That's where tsrc log comes in handy. It will run git log with nice color options and present you a summary:

Conclusion

We hope you'll find this tool handy for your own projects.

Feel free to try it, contribute, and give us feedback.