DEV Community

Cover image for Troubleshooting Auth Issues Pulling Multiple Images With BuildKit in Github Actions
Joseph Yi
Joseph Yi

Posted on

Troubleshooting Auth Issues Pulling Multiple Images With BuildKit in Github Actions

#devops #docker #githubactions

Just wanted to cut to the chase about an unexpected issue I encountered while migrating to Docker Hardened Images in my GitHub Actions CI pipeline. I was building a multi-stage Dockerfile pulling multiple images from the dhi.io registry when I hit this error:

Error: buildx failed with: ERROR: failed to build: failed to solve: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

This was despite having docker/login-action step configured for dhi.io, which always reported Login Succeeded!. What ended up working was directly configuring the docker/setup-buildx-action (BuildKit) step with dhi.io. This ensures BuildKit's daemon configuration includes the registry before any image pulls occur:

      - name: Setup buildx
        uses: docker/setup-buildx-action@v3
        with:
          buildkitd-config-inline: |
            [registry."dhi.io"]
              mirrors = ["dhi.io"]
Enter fullscreen mode Exit fullscreen mode

I was able to reproduce this issue and created a repository where I demonstrate a failing pipeline without this configuration and a successful pipeline with this configuration. In the failing pipeline, I also demonstrated how the first stage will build just fine when targeted:

        run: |
          docker buildx build \
            --tag ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:test \
            --load \
            --target=builder \
            .
Enter fullscreen mode Exit fullscreen mode

Whereas the second stage fails when targeted:

        run: |
          docker buildx build \
            --tag ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:test \
            --load \
            --target=aot-cache-training-runner \
            .
Enter fullscreen mode Exit fullscreen mode

To be clear, this issue is probably not unique to the dhi.io registry; presumably it would happen with any private registry in multi-stage builds where BuildKit pulls multiple images in parallel. The root cause appears to be a race condition: docker builds can start pulling images before BuildKit's daemon has fully applied the registry configuration from docker/login-action.

Here are some takeaways for anyone encountering similar authentication issues:

  1. If docker/login-action succeeds but builds fail with "pull access denied," the issue is likely BuildKit daemon configuration, not credentials
  2. Use buildkitd-config-inline to explicitly configure registries before builds
  3. This is especially important for multi-stage builds that pull from multiple private registries in parallel

Effectively it should look like this in GHA:

      - name: Authenticate to Docker Hardened Image Registry
        uses: docker/login-action@v3
        with:
          registry: someregistry.io
          username: ${{ secrets.REGISTRY_USER }}
          password: ${{ secrets.REGISTRY_PASSWORD }}
      - name: Setup buildx
        uses: docker/setup-buildx-action@v3
        with:
          buildkitd-config-inline: |
            [registry."someregistry.io"]
              mirrors = ["someregistry.io"]
Enter fullscreen mode Exit fullscreen mode

As more teams adopt Docker Hardened Images and other enterprise level hardening solutions, understanding this BuildKit configuration quirk will save hours of debugging and get you closer to being well-architected. Feel free to add comments either here or on the sample repository, especially if it helped!

Top comments (0)