DEV Community

Cover image for Argo CD: Previewing Pull Requests changes in SECONDS! 🥵⚡️⏰
Dag Andersen
Dag Andersen

Posted on

Argo CD: Previewing Pull Requests changes in SECONDS! 🥵⚡️⏰

#kubernetes #devops #git #argocd

TL;DR: You can now render previews of your PRs by using a cluster with Argo CD pre-installed instead of spinning up a new one each run. This results in very short preview times while maintaining accuracy.

This is a continuation of my first blog post: Rendering the TRUE Argo CD diff on your PRs. That article addresses a critical challenge in GitOps workflows: visualizing the actual impact of configuration changes when using templating tools like Helm and Kustomize.

The article shows how you can render changes to your manifests/Argo CD configuration directly on your pull requests using argocd-diff-preview

In short, it shows how you can transform a pull request like this:

and turn it into a preview like this:

Here are some examples:

3 Example Pull Requests:

Three Approaches to Preview Generation

Since its introduction in 2024, the tool now supports more ways of running it, so you can optimize it for your use case!

All three approaches are perfectly valid and you can choose the one that best fits your needs.

Approach 1: Ephemeral Clusters (Original)

This is the simplest solution, but also the slowest...

The original solution spins up ephemeral Kubernetes clusters inside your CI/CD pipeline, letting Argo CD itself render the manifests. This ensures maximum accuracy since the same engine that will deploy your changes generates the preview.

How it works:

  1. Create an ephemeral Kubernetes cluster (kind, k3d, or minikube)
  2. Install Argo CD from scratch
  3. Apply your applications (without syncing them)
  4. Wait for Argo CD to render the Applications in memory
  5. Extract the manifests from the cluster
  6. Generate the diff between the two sets of rendered manifests
  7. Post the diff to the pull request

This is the approach described in detail in the first blog post.

Why this approach is superior to alternative tools:

  • True accuracy: Uses Argo CD itself rather than tools that try to mimic its rendering logic
  • No infrastructure access required: Can work without credentials for your production Argo CD instance
  • Complete isolation: Can run in complete isolation from your production systems

However, this approach has one significant limitation: Speed. Creating a cluster and installing Argo CD takes ~60 seconds every time, making even simple configuration changes take 80+ seconds to preview.

This leads us to the next approach to running argocd-diff-preview.

Trade-offs:

  • ✅ Zero setup
  • ✅ Complete isolation
  • ✅ Works with any CI/CD (even works on your local machine)
  • ❌ Slow (~60 second overhead per run)
  • ❌ Resource-intensive (creates a new cluster for each run)

Approach 2: Cluster with Argo CD pre-installed

Instead of creating a new ephemeral cluster each time, argocd-diff-preview can now connect to a cluster with Argo CD already installed. This reduces preview times from minutes to seconds! 🤯

See it as a cluster with Argo CD pre-installed that is on standby and ready to render your manifests. Remember, the cluster only runs Argo CD and not the resources generated by the Applications. So it is fairly lightweight.

We do not recommend using your normal Argo CD instance for this. Instead, create a dedicated cluster or Argo CD instance for diff previews.

How it works:

  1. Connect to the cluster
  2. Apply your applications (without syncing them)
  3. Wait for Argo CD to render the Applications in memory.
  4. Extract the manifests from the cluster
  5. Generate the diff between the two sets of rendered manifests
  6. Post the diff to the pull request

Quick Demo:

# Create cluster and install Argo CD (one-time setup)
kind create cluster
helm repo add argo https://argoproj.github.io/argo-helm
helm install argo-cd argo/argo-cd --version 8.0.3 --create-namespace --namespace argocd-diff-preview

# Wait for Argo CD to be ready
kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=argocd-server -n argocd-diff-preview

# Clone each branch from the PR into subfolders
git clone https://github.com/dag-andersen/argocd-diff-preview base-branch --depth 1 -q 
git clone https://github.com/dag-andersen/argocd-diff-preview target-branch --depth 1 -q -b helm-example-3

# Run the tool (connects to the kind cluster)
docker run \
  --network host \
  -v ~/.kube:/root/.kube \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v $(pwd)/output:/output \
  -v $(pwd)/base-branch:/base-branch \
  -v $(pwd)/target-branch:/target-branch \
  -e TARGET_BRANCH=helm-example-3 \
  -e REPO=dag-andersen/argocd-diff-preview \
  dagandersen/argocd-diff-preview:v0.1.18 \
  --argocd-namespace=argocd-diff-preview \
  --create-cluster=false
Enter fullscreen mode Exit fullscreen mode

Concurrent Runs: Multiple PRs, No Conflicts

Each run uses a unique identifier, so multiple PRs can run without collisions.

Network & Security:

The main downside here is that you need access to the Kubernetes API from your GitHub/GitLab hosted pipeline. Some organizations will see this as a no-go... which leads us to the next way of running argocd-diff-preview...

Trade-offs:

  • ✅ Fast execution (eliminates ~60s overhead)
  • ✅ Utilizes Argo CD caching from previous runs
  • ❌ Infrastructure setup required (set up a cluster with Argo CD beforehand)
  • ❌ Cluster credentials in CI/CD pipeline

Approach 3: Cluster with Argo CD pre-installed + Self-Hosted Runner

Running argocd-diff-preview on a self-hosted runner inside a cluster that has Argo CD pre-installed combines maximum performance with enhanced security. This approach eliminates both cluster creation overhead and the need to store cluster credentials in your CI/CD pipeline.

Instead of creating a temporary cluster for each diff preview, your self-hosted GitHub Actions runner connects directly to a dedicated Argo CD instance running in the same cluster as the hosted runners. This offers fast execution (no cluster creation overhead) and enhanced network and credential security.

Setup Guide:

  1. Install Action Runner Controller (ARC) in your cluster alongside the dedicated Argo CD instance in the namespace argocd-diff-preview
  2. The runner uses a service account to connect to the host cluster and access the Argo CD instance
  3. The tool runs exactly as before, but without any credential management complexity and without creating an ephemeral cluster

Note: This setup is more complex than the previous two approaches. For complete setup instructions, see the self-hosted runner guide.

Trade-offs:

  • ✅ Fast execution (eliminates ~60s overhead)
  • ✅ Network isolation (No need to expose your cluster to the internet)
  • ✅ No cluster credentials in CI/CD pipeline because you are using a service account from within the cluster and Argo CD already has all the credentials it needs
  • ❌ Most complex setup (requires self-hosted runners + dedicated Argo CD)

Optimizing for Large Repositories

Even when using a cluster with Argo CD pre-installed, repositories with hundreds of applications can take 1+ minutes to render. The solution is selective rendering using annotations.

The most powerful optimization is the argocd-diff-preview/watch-pattern annotation. This tells the tool exactly which files each application cares about. An application will only be rendered if its watch patterns match any of the changed files in the PR.

Smart Application Selection

Setup Guide:

  1. Add the argocd-diff-preview/watch-pattern annotation to your Applications and ApplicationSets
  2. Enable automatic file change detection. Use the following flags:
    1. --auto-detect-files-changed=true
    2. --watch-if-no-watch-pattern-found=true

Example:

Use the argocd-diff-preview/watch-pattern annotation to tell the tool exactly which files each application cares about:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  namespace: argocd
  annotations:
    argocd-diff-preview/watch-pattern: "apps/my-app/.*, shared/values.yaml"
spec:
  source:
    path: apps/my-app
    # ...
Enter fullscreen mode Exit fullscreen mode

This application will only render if:

  • Any file in the apps/my-app/ directory changes
  • The shared/values.yaml file changes
  • The application's own manifest file changes (automatic)

More information about the watch-pattern annotation can be found in the application-selection documentation.

Other application selection methods:

  • Ignore applications: argocd-diff-preview/ignore: "true"
  • Filter by labels: --selector "team=frontend"
  • Filter by path: --file-regex="/team-a/"

Performance impact: Instead of rendering 100+ applications (60+ seconds), render only 6-10 relevant applications (~10 seconds).

For complete details, see the application-selection documentation.

Comparison: Which Approach to Choose?

Approach Best For Pros Cons
Ephemeral
clusters		 Getting started, Full isolation • Simple setup
• Complete isolation		 • Slow (~60s overhead)
Cluster with
Argo CD
pre-installed		 Teams prioritizing speed • Fast
• Leverages Argo CD caching		 • Requires dedicated Argo CD setup
• Need cluster credentials in CI/CD
• Infrastructure maintenance
Cluster with
Argo CD
pre-installed + self-hosted runner		 Teams prioritizing speed AND network security • Fast + secure
• Leverages Argo CD caching
• No credentials in CI/CD		 • Most complex setup
• Requires dedicated Argo CD and self-hosted runner setup
• Infrastructure maintenance

Real-World Results

At Egmont, we use the "Cluster with Argo CD pre-installed + self-hosted runner" approach with a repository containing 600+ applications. Combined with smart application selection, we achieve preview times under 10 seconds - a 20x improvement over the original "ephemeral cluster" approach.

Each pull request preview includes a stats section at the bottom.

stats section in preview

The key is combining two optimizations:

  1. Use a cluster with Argo CD pre-installed to eliminate cluster creation overhead
  2. Select only affected applications using watch patterns to minimize rendering scope

This approach maintains the accuracy that makes argocd-diff-preview superior to alternatives while delivering the preview in seconds.

Conclusion

The evolution of argocd-diff-preview proves you don't have to choose between accuracy and speed. What started as an accurate - but slow - solution has matured into a flexible tool that adapts to your team's needs - whether you prioritize simplicity, speed, or security.

Throughout this journey, one principle has remained constant: use Argo CD itself to render manifests. This ensures your previews perfectly match what will actually deploy. What's new is the speed at which this happens. By using a cluster with Argo CD pre-installed and intelligently selecting applications, we've lower preview times from minutes to seconds! ⏰⚡️

Each approach presented in this blog post has its own pros and cons. You should choose the one that best fits your needs.

I suggest that you begin with the "ephemeral cluster" approach to get familiar with the tool. As your confidence grows and speed becomes a priority, you can consider if you should switch to a faster setup using a cluster with Argo CD pre-installed. Combine this with well-placed argocd-diff-preview/watch-pattern annotations and you can achieve preview times under 10 seconds while maintaining the same accuracy you started with, even at scale.

The result? Preview rendering in under 10 seconds 🎉

For detailed setup instructions, check out:

Top comments (0)