Limiting tags when using git fetch

At work, our CI creates tags for each build when our pipelines are completed. The reason is that our deployment tooling needs some kind of identifier, and they settled on a tag. The problem for me with that is I have a repository that contains over 1000 tags. Now, before they did this, I was able to do git tag --contains <commit-ish> and see in which release a commit was found. I still have the ability to do so, but I needed to filter out all the automated nothingness of the tags created by the CI.

In addition, they also delete tags after X days to do some cleaning up, but I still get to have all the tags because I don't have a maintenance script running around wiping old tags. The problem is then multiplied by other forks having all the (outdated) tags. In short, it is a bit of a mess. So I wanted to limit the tags I fetch.

According to the git release notes and github blog by one of its maintainers negative refspecs are supported by git push and git fetch.

In an unedited git repository config, you can see what git wants to fetch by default for a remote:

[remote "upstream"]
  url = [e-mail]:owner/project.git
  fetch = +refs/heads/*:refs/remotes/upstream/*

A negative refspec would be ^refs/heads/aBranchIDontWant*.
I tried using a refspec for to exclude some tags like so: fetch = ^refs/tags/*-development. But it didn't work. The trick is to set tagOpt = --no-tags, which is similar to git fetch --no-tags, which will then work:

[remote "upstream"]
  url = [e-mail]:owner/project.git
  fetch = +refs/heads/*:refs/remotes/upstream/*
  tagOpt = --no-tags
  fetch = ^refs/tags/*-development

Now git fetch will, by default, not fetch tags ending with -development.

Refspecs syntax matters

Figuring this out opened the pathway to solving my problem. One caveat appeared: The refspec +refs/tags/release*:refs/tags/release/* creates a funny error, literally: error: * Ignoring funny ref 'refs/tags/release//v2023.1.0' locally.

Ok, so what is happening here? I'll show you by using a different example: +refs/tags/v*-release:refs/tags/myrelease-*. When you fetch, you see this output: * [new tag] v2023.4.2-release -> myrelease-2023.4.2. It renames the tag from v2023.4.2-release to myrelease-2023.4.2. Aha. So the problem lies in the fact that we have release/v2023.4.5, and now the rename acts and makes the ref /v2023.4.5, and that is a "funny" thing. We can work around it, by doing: +refs/tags/release/*:refs/tags/release/*. If you use +refs/tags/release/*:refs/tags/* you are creating tags locally as v2023.4.5. It is a bit of a tricky situation. If you have tags names release/v10 and release-v10, you need to be really specific with your refspecs :)

Knowing this makes our recipe for fetching only specific tags easy:

[remote "upstream"]
  url = [e-mail]:owner/project.git
  fetch = +refs/heads/*:refs/remotes/upstream/*
  tagOpt = --no-tags
  fetch = +refs/tags/v*:refs/tags/v*
  fetch = +refs/tags/release/*:refs/tags/release/*

Then there is git fetch --tags, which deals with things a wee bit differently. This one fetches all the tags, so you need to specify which tags you do not want. This list became rather long because of (perhaps my limited knowledge of) refspec globs:

[remote "upstream"]
  url = [e-mail]:owner/project.git
  fetch = +refs/heads/*:refs/remotes/upstream/*
  tagOpt = --no-tags
  fetch = +refs/tags/v*:refs/tags/v*
  fetch = +refs/tags/release/*:refs/tags/release/*
  # things we want to ignore with git fetch --tags
  fetch = ^refs/tags/0.*-release
  fetch = ^refs/tags/*-master
  fetch = ^refs/tags/*-preprod
  fetch = ^refs/tags/*-production
  fetch = ^refs/tags/*-development
  fetch = ^refs/tags/202*
  fetch = ^refs/tags/randomexperiment/*

I cannot exclude *-release, because our version tags also contain -release. I wanted to use [0-9]+.*-* to exclude every non-release tag, but that wasn't possible. I had to be verbose and ignore each one: master, preprod, production, and development. Some tags in our repo are mistakes by a release manager, so I also added them to my exclusion list.

This piece of configuration makes sure I don't have to fetch all the tags from the upstream remote, and I can do git tag --contains <commit-ish> again without having to look at tags that don't have any real meaning to any human.

The biggest remaining issue now is that I have to copy this configuration to all the remotes I have for this project. That is because you have to set it on the remote and not as a more general configuration item. But that is a problem for another day. Happy tagging!

Comments

[Christophe Colombier]

Interesting article, I can see you had to iterate a lot.

[Wesley Schwengle]

The biggest hurdle was the tagOpt. The rest was a bit of playing around to see how refspecs work when using git fetch. Luckely you can do git fetch --no-tags refs/tags/xyz*:refs/tags/yyy* and fiddle with that on the command line.