DEV Community

Cover image for Limiting tags when using git fetch
Wesley Schwengle
Wesley Schwengle

Posted on • Edited on

Limiting tags when using git fetch

#git #fetch #tags #howto

TL;DR

If you are using git tag --contains=<commit> but it becomes unworkable due to a flood of CI-generated tags, you can fix it by setting tagOpt = --no-tags in your configuration and by using negative refspecs, which limit the number of tags you are fetching. Both are placed on the remote.

[remote "upstream"]
  url = git@gitlab.com:owner/project.git
  fetch = +refs/heads/*:refs/remotes/upstream/*
  tagOpt = --no-tags
  fetch = ^refs/tags/*-development
Enter fullscreen mode Exit fullscreen mode

Tag explosions while needing to find commits in tags

At work, our CI creates tags for each build when our pipelines are completed. The reason is that our deployment tooling requires an identifier, and they have settled on using a tag. The problem for me with that is I have a repository that contains over 1000 tags. Previously, I could use git tag --contains <commit-ish> to see which release a commit was found in. I still have the ability to do so, but I needed to filter out all the automated nothingness of CI-generated tags.

In addition, they also delete tags after X days via a scheduled job, but I still get to have all the tags because I don't have a maintenance script running around wiping old tags. The problem is then multiplied by other forks having all the (outdated) tags. In short, it is a bit of a mess. So I wanted to limit the tags I fetch.

According to the git release notes and github blog by one of its maintainers, negative refspecs are supported by git push and git fetch.

In an unedited git repository config, you can see what git wants to fetch by default for a remote:

[remote "upstream"]
  url = git@gitlab.com:owner/project.git
  fetch = +refs/heads/*:refs/remotes/upstream/*
Enter fullscreen mode Exit fullscreen mode

A negative refspec would be ^refs/heads/aBranchIDontWant*.
I tried using a refspec to exclude some tags like so: fetch = ^refs/tags/*-development. But it didn't work. The trick is to set tagOpt = --no-tags, which is similar to git fetch --no-tags, which will then work:

[remote "upstream"]
  url = git@gitlab.com:owner/project.git
  fetch = +refs/heads/*:refs/remotes/upstream/*
  tagOpt = --no-tags
  fetch = ^refs/tags/*-development
Enter fullscreen mode Exit fullscreen mode

Now git fetch will, by default, not fetch tags ending with -development.

Refspecs syntax matters

Figuring this out opened the pathway to solving my problem. One caveat appeared: The refspec +refs/tags/release*:refs/tags/release/* creates a funny error, literally:

error: * Ignoring funny ref 'refs/tags/release//v2023.1.0' locally
Enter fullscreen mode Exit fullscreen mode

Ok, so what is happening here? I'll show you by using a different example: +refs/tags/v*-release:refs/tags/myrelease-*. When you fetch, you see this output:

* [new tag] v2023.4.2-release -> myrelease-2023.4.2
Enter fullscreen mode Exit fullscreen mode

It renames the tag from v2023.4.2-release to myrelease-2023.4.2. Aha. So the problem lies in the fact that we have release/v2023.4.5, and now the rename acts and makes the ref /v2023.4.5, and that is a "funny" thing. We can work around it, by setting the refspec to:

+refs/tags/release/*:refs/tags/release/*
Enter fullscreen mode Exit fullscreen mode

If you use +refs/tags/release/*:refs/tags/*, you are creating tags locally as v2023.4.5. This could potentially become a bit of a tricky situation. If you have tag names release/v10 and release-v10, you need to be really specific with your refspecs :)

Knowing this makes our recipe for fetching only specific tags easy:

[remote "upstream"]
  url = git@gitlab.com:owner/project.git
  fetch = +refs/heads/*:refs/remotes/upstream/*
  tagOpt = --no-tags
  fetch = +refs/tags/v*:refs/tags/v*
  fetch = +refs/tags/release/*:refs/tags/release/*
Enter fullscreen mode Exit fullscreen mode

Then there is git fetch --tags, which deals with things a wee bit differently. This one fetches all the tags, so you need to specify which tags you do not want. This list became rather long because of (perhaps my limited knowledge of) refspec globs:

[remote "upstream"]
  url = git@gitlab.com:owner/project.git
  fetch = +refs/heads/*:refs/remotes/upstream/*
  tagOpt = --no-tags
  fetch = +refs/tags/v*:refs/tags/v*
  fetch = +refs/tags/release/*:refs/tags/release/*
  # things we want to ignore with git fetch --tags
  fetch = ^refs/tags/0.*-release
  fetch = ^refs/tags/*-master
  fetch = ^refs/tags/*-preprod
  fetch = ^refs/tags/*-production
  fetch = ^refs/tags/*-development
  fetch = ^refs/tags/202*
  fetch = ^refs/tags/randomexperiment/*
Enter fullscreen mode Exit fullscreen mode

I cannot exclude *-release, because our version tags also contain -release. I wanted to use [0-9]+.*-* to exclude every non-release tag, but that wasn't possible. I had to be verbose and ignore each one: master, preprod, production, and development. Some tags in our repo were added by a release manager in error, so I've also added them to my exclusion list.

This configuration ensures I don't have to fetch all tags from the upstream remote, allowing me to run git tag --contains <commit-ish> again without seeing tags that have no real meaning to humans.

The biggest remaining issue now is that I have to copy this configuration to all the remotes I have for this project. That is because you have to set it on the remote and not as a more general configuration item. But that is a problem for another day. Happy tagging!

Top comments (2)

Collapse
 
ccoveille profile image
Christophe Colombier •

Interesting article, I can see you had to iterate a lot.

Collapse
 
waterkip profile image
Wesley Schwengle •

The biggest hurdle was the tagOpt. The rest was a bit of playing around to see how refspecs work when using git fetch. Luckely you can do git fetch --no-tags refs/tags/xyz*:refs/tags/yyy* and fiddle with that on the command line.