DEV Community

Cover image for How to Build a Three-Layer Artifact Integrity Pipeline for Container Releases
Ilyas Rufai
Ilyas Rufai

Posted on

How to Build a Three-Layer Artifact Integrity Pipeline for Container Releases

#docker #devsecops #security #containers

Your team tags myapp:latest in the registry. Someone on the team pushes a patched image with the same tag from their laptop. Kubernetes pulls it overnight. Nobody notices until behavior changes in production.

Container releases need integrity at three points: when the image is built, when it is stored, and when it is deployed. Scanning alone does not prove the image your cluster runs is the one CI built.

In this tutorial, you will learn how to build a three-layer artifact integrity pipeline for container releases: sign at build with Cosign, enforce registry immutability, and verify signatures before deploy.

Who this is for: Platform engineers and DevSecOps engineers shipping containers to Kubernetes or ECS.

Prerequisites:

  • Docker or buildkit-based builds in CI
  • Container registry (GHCR, ECR, or similar)
  • GitHub Actions or equivalent CI

TL;DR

  • Layer 1 (Build): generate SBOM, scan, and sign the image digest with a Cosign keyless or KMS-backed key.
  • Layer 2 (Registry): immutable tags, retention policies, restrict push to CI roles only.
  • Layer 3 (Deploy): admission policy or deploy script verifies signature before apply.
  • Pin deploy to digest, not mutable tags.
  • Verification fails closed: unsigned or changed digest blocks release.

Why "Scan and Push" Is Not Enough

Control Proves Does not prove
Vulnerability scan Known CVEs at scan time Image was not swapped after scan
:latest tag Nothing stable Same tag, different digest
Private registry Network access control Insider or leaked push credentials

Key idea: Sign the digest. Verify at deploy. Immutable storage in between.

Three layers: build sign, registry immutability, deploy verify

Layer 1: Build, Scan, and Sign

Example GitHub Actions job (after image push):

name: release-image

on:
  push:
    branches: [main]

permissions:
  contents: read
  packages: write
  id-token: write

jobs:
  build-sign:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@11bd71901bbe5b1630ea94775b8ad9d445e94a5f

      - name: Log in to GHCR
        uses: docker/login-action@9780b0c442871bb9410bab68882983f40faa2790
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push
        id: build
        uses: docker/build-push-action@0565240e94d8a432caa0bc856117427372fc2402
        with:
          push: true
          tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
          provenance: true
          sbom: true

      - name: Install Cosign
        uses: sigstore/cosign-installer@d6867f3a4f1248d4a17893a47b12155122a1c126

      - name: Sign image with keyless OIDC
        env:
          DIGEST: ${{ steps.build.outputs.digest }}
        run: |
          cosign sign --yes "ghcr.io/${{ github.repository }}@${DIGEST}"
Enter fullscreen mode Exit fullscreen mode

Keyless signing binds signatures to your CI identity via OIDC. For stricter policy, use Cosign with AWS KMS or a HashiCorp Vault key.

Run Trivy before signing (fail on critical CVEs if that is your policy):

      - name: Scan image
        uses: aquasecurity/trivy-action@0.28.0
        with:
          image-ref: ghcr.io/${{ github.repository }}:${{ github.sha }}
          severity: CRITICAL,HIGH
          exit-code: 1
Enter fullscreen mode Exit fullscreen mode

Layer 2: Registry Immutability and Push Control

GitHub Container Registry

Use digest-pinned references in deploy manifests. Avoid overwriting release tags.

For ECR:

aws ecr put-image-tag-mutability \
  --repository-name myapp \
  --image-tag-mutability IMMUTABLE
Enter fullscreen mode Exit fullscreen mode

Restrict ecr:PutImage to the CI role OIDC identity, not developer IAM users.

Retention and provenance

  • Enable artifact attestations where supported (GHCR + GitHub attestations).
  • Delete untagged manifests on a schedule to reduce orphan tamper targets.

Layer 3: Verify Before Deploy

Before kubectl apply or ECS deploy, verify signature:

IMAGE="ghcr.io/org/myapp@sha256:abc123..."

cosign verify \
  --certificate-identity-regexp "https://github.com/ORG/REPO/.github/workflows/release-image.yaml@refs/heads/main" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  "${IMAGE}"
Enter fullscreen mode Exit fullscreen mode

Deploy script pattern:

#!/usr/bin/env bash
set -euo pipefail

IMAGE_DIGEST="$1"
cosign verify "${IMAGE_DIGEST}" # flags as above
kubectl set image deployment/myapp app="${IMAGE_DIGEST}"
Enter fullscreen mode Exit fullscreen mode

For Kubernetes clusters, use Kyverno or OPA Gatekeeper to enforce signature verification at admission:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: verify-image-signatures
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-signature
      match:
        any:
          - resources:
              kinds: [Pod]
      verifyImages:
        - imageReferences: ["ghcr.io/my-org/*"]
          attestors:
            - entries:
                - keyless:
                    subject: "https://github.com/my-org/my-repo/.github/workflows/release-image.yaml@refs/heads/main"
                    issuer: "https://token.actions.githubusercontent.com"
Enter fullscreen mode Exit fullscreen mode

How to Verify This Works

  1. Happy path: signed image deploys; cosign verify exits 0 in CI deploy job.
  2. Tamper test: push unsigned image digest to cluster manually; admission or deploy script rejects it.
  3. Tag mutation test: with immutable tags enabled, retag push fails at registry.
  4. SBOM: download attestation from registry and confirm it matches the build job.

When This Breaks Down

  1. Local dev images: developers bypass verification on minikube; keep verification mandatory only on shared environments first.
  2. Keyless offline verification: requires network access to Rekor/Sigstore; plan for air-gapped with KMS keys.
  3. Third-party base images: you sign your layer stack, not upstream FROM scan and pin base digests separately.
  4. Emergency hotfix: break-glass deploy procedure must still log who skipped verification and for how long.

Conclusion

In this tutorial, you learned a three-layer container integrity pipeline: sign at build with Cosign, enforce immutable registry policy, and verify signatures before deploy, using digests instead of mutable tags.

Start with Layer 1 on one service, add deploy verification as Layer 3, then lock the registry as Layer 2.

References

Top comments (0)