DEV Community

Sergey Shinder
Sergey Shinder

Posted on

Our build cache was saved once in January and never again

In September I went looking for the reason our order service took seventeen minutes to build. In January it had taken six, and nothing large had been added since. The log of a single run had four thousand lines beginning with Downloading from central.

The workflow caches the local Maven repository with actions/cache, under the key maven-${{ runner.os }}. Somebody wrote it that way when we moved to GitHub Actions, and it worked at once: the first run on main saved the cache, and every run after it restored it.

Caches in GitHub Actions cannot be changed once saved. When a job finds an exact match for its key, it restores that entry, and at the end the post step logs Cache hit occurred on the primary key and saves nothing. A key that never changes matches forever. So the cache we restored in September held the dependencies of our January build. Every version change since then, a Spring Boot upgrade, two AWS SDK releases and about eighty Dependabot merges, was downloaded fresh on every run and thrown away with the runner. GitHub evicts caches that nobody has used for seven days, but ours was read dozens of times a day. It was never unused, only out of date.

Nothing failed. Each week was a little slower than the one before, which is the kind of change nobody notices, and every run said cache hit.

The key now ends with a hash of every pom.xml in the repository, hashFiles('**/pom.xml'). A changed dependency means a new key, so the run falls back through restore-keys to the closest older entry, downloads only the difference and saves a new entry. We split the step into restore and save, and only runs on main save, so pull requests use main's cache without filling the repository's allowance with their own. The next day the build took six minutes. A final step counts the Downloading lines in the log and writes the number to the job summary, and a weekly report charts build time and downloads per workflow, so a cache that has stopped helping shows up as a slope on a chart.

A cache key decides when a cache is allowed to change. Ours said never, and the cache did exactly what it was told.

– Sergey Shinder

Top comments (0)