The Saga of `latest` in Container Registries

When you are delivering continuously, it makes sense that you always want to use the latest dependent binaries. Similarly, the testing team would like to use the latest binaries produced for tests. With that being said, it’s important to make it easy for users to pull the latest images from a container registry. I had mentioned this in my earlier blog:

10 Things You Should Expect From a Container Registry

Prasanna Raghavendra ・ Nov 24 '19 ・ 3 min read

#containerregistry #kubernetes #devops #docker

Docker Hub provides a simple tag model where you tag an image as latest and when you pull latest you get that image. This model has inherent issues as discussed in various forums. One example below:

The misunderstood Docker tag: latest | by Marc Campbell | Medium

Marc Campbell ・ Jan 13, 2016 ・ 3 min read

Medium

.
In addition to the issues discussed in the blog, sometimes you may have a version that’s tagged latest in your local repo but that may be old because the remote server built a newer one.

Example:

Prasannas-MacBook-Pro:keys prasannaraghavendra$ docker images
REPOSITORY                                 TAG                                   IMAGE ID            CREATED             SIZE
bintray_kramdown                           latest                                9b919eb86d92        2 months ago        512MB
registry.access.redhat.com/ubi8/ubi-init   latest                                1de7d7b3f531        2 months ago        223MB
registry.access.redhat.com/ubi8/ubi        latest                                11f9dba4d1bc        2 months ago        208MB

As you can see, these are two-month old images but tagged latest. If I run a ‘compose’ or a ‘run’, expecting a ‘latest’, what I end up with is a two-month-old image! In addition, there will be no error anywhere in the process, unless you start looking at the CREATED field above.

There is a certain way to handle this situation - by always deleting latest, but when you do this, you lose bandwidth when you actually have the latest locally.

In addition to this, there are a number of other requirements that can make this more complicated than just getting the latest version of an image from a repo. Let us quickly consider these three examples:

1. I want the latest of all tested dependencies- I am not just looking at the latest of an image, I also want to pull the latest of the local dependent images that are tested against this image.

2. I want the latest of released branches- I am looking at the latest released images across all released branches to ensure I can verify a newly discovered vulnerability.

3. I want the latest of a given branch- I am checking out the latest image in a branch, but then how would I then ensure all my dependencies are pulled from a similar branch or not?

All three examples relate to the necessity for container registries to provide the ability to add metadata and a flexible query construct.

I must say, Both Artifactory and JFrog Container Registry provide this efficiently.

Let’s look at the example of the query below using JFrog CLI:

jfrog rt s "libs-staging/*/*application*.tar.gz"  --limit=1 --sort-by=created --sort-order=desc | jq '.[0].props."dependencies.*"'

The above example pulls from JFrog Container Registry the latest image in staging for application, and parses the list of dependencies that are already marked in the build info [aka metadata]

This is of course just the beginning, you can extend the way you want using the developer in you!