Deploying Nginx with Argo CD on a Local Kubernetes Cluster — A Real-World GitOps Walkthrough
Most Argo CD tutorials show you a clean happy path: install, point it at a repo, watch it sync, done. That's useful for understanding the concept — but it doesn't prepare you for what actually happens when you build this on your own machine.
This post walks through a real deployment of Nginx to a local Kubernetes cluster (k3d) using Argo CD — including every infrastructure issue I hit along the way, why it happened, and how I fixed it. If you're learning GitOps, I think the debugging is actually the more valuable part.
Why GitOps, and why Argo CD
Traditional CI/CD is push-based: your pipeline authenticates to the cluster and runs kubectl apply or helm upgrade as its last
step. GitOps flips this to pull-based: a controller running inside the cluster (Argo CD, in this case) continuously watches a Git repository and reconciles the live cluster state to match what's declared there.
This has real, practical consequences:
Git becomes the source of truth. Want to know what's running in production? Read the repo, not the cluster.
Full audit trail. Every change is a commit — no more "who ran that kubectl command?"
Easy rollback. git revert is your rollback strategy.
No CI credentials in the cluster. CI only ever touches Git, never the cluster directly, tightening your security boundary considerably.
Drift detection and correction. If someone manually edits a resource, Argo CD notices and can revert it — this is called self-heal, and I'll demonstrate it later in this post.
Architecture overview
Developer → commits code → CI builds & pushes image → CI updates image tag in Git
│
▼
Config repo (Git)
│
Argo CD polls / receives webhook
│
▼
Diffs desired state vs live state
│
▼
Syncs Kubernetes cluster
Argo CD's core components: the API server (serves the UI/CLI), the repo server (clones and renders manifests), the application controller (the actual reconciliation loop), Redis (caching), and CRDs (Application, AppProject, ApplicationSet) that define what to track.
Setting up the local cluster
I used k3d — k3s running inside Docker containers — since it spins up fast and doesn't need a Linux host natively.
choco install docker-desktop kubernetes-cli k3d -y
k3d cluster create devcluster --servers 1 --agents 1 -p "8081:80@loadbalancer"
This should, in theory, "just work." In practice, I hit three separate infrastructure issues before the cluster came up healthy.
Issue 1 — Docker couldn't resolve external DNS
docker failed to pull the image 'ghcr.io/k3d-io/k3d-proxy:5.9.0':
dial tcp: lookup ghcr.io: no such host
Docker Desktop's internal DNS wasn't resolving external registries at all — a common issue caused by network config, VPNs, or a broken default resolver inside the WSL2 VM Docker runs on.
Fix: set an explicit DNS server in Docker Desktop's daemon config (Settings → Docker Engine):
{
"dns": ["8.8.8.8", "1.1.1.1"]
}
Verified with:
docker run --rm busybox nslookup ghcr.io
Issue 2 — kubelet refused to start (cgroup v1 vs v2)
Error: failed to validate kubelet configuration, error: kubelet is configured to not run on a
host using cgroup v1. cgroup v1 support is unsupported and will be removed in a future release
This one was buried under about a hundred repeating "connection refused" log lines — the real error was easy to miss. Modern k3s requires the unified cgroup v2 hierarchy, but my WSL2 environment was still exposing cgroup v1.
Fix: forced cgroup v2 via a WSL2 boot parameter.
wsl --update
wsl --shutdown
In %USERPROFILE%.wslconfig:
[wsl2]
kernelCommandLine=cgroup_no_v1=all
After restarting WSL2 and Docker Desktop, I confirmed the fix by checking for the v2 marker file:
docker run --rm busybox ls /sys/fs/cgroup
The presence of cgroup.controllers in the output confirms cgroup v2 is active (cgroup v1 exposes separate directories like cpu/, memory/, blkio/ instead).
*Issue 3 *— kubectl couldn't connect after everything looked fine
Unable to connect to the server: dial tcp 192.168.0.2:xxxxx: connectex:
No connection could be made because the target machine actively refused it.
The cluster and load balancer were both running and healthy — but kubectl was trying to reach host.docker.internal, which resolved to a LAN IP (192.168.0.2) instead of loopback. A quick Test-NetConnection -ComputerName localhost -Port confirmed the port was open on localhost — the hostname resolution was just wrong.
**Fix: **rewrote the kubeconfig to use localhost instead:
$config = Get-Content "$env:USERPROFILE.kube\config" -Raw
$config = $config -replace "host.docker.internal", "localhost"
Set-Content "$env:USERPROFILE.kube\config" -Value $config
After that:
kubectl get nodes
NAME STATUS ROLES AGE VERSION
k3d-devcluster-agent-0 Ready 19m v1.35.5+k3s1
k3d-devcluster-server-0 Ready control-plane 19m v1.35.5+k3s1
Cluster: finally healthy.
Installing Argo CD
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
Issue 4 — a CRD too large for client-side apply
The CustomResourceDefinition "applicationsets.argoproj.io" is invalid: metadata.annotations:
Too long: may not be more than 262144 bytes
kubectl apply's default client-side apply stores the entire applied configuration as an annotation on the object, and the ApplicationSet CRD is large enough to exceed Kubernetes' 262144-byte annotation limit. This is a known Argo CD installation quirk, not specific to my setup.
**Fix: **apply that CRD with server-side apply, which has the API server track field ownership instead of stuffing everything into an annotation:
kubectl apply --server-side --force-conflicts --validate=false --request-timeout=120s `
-f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/crds/applicationset-crd.yaml
Then restarted the dependent controller so it would pick up the newly available CRD:
kubectl rollout restart deployment/argocd-applicationset-controller -n argocd
All 7 Argo CD pods reached 1/1 Running after that.
Accessing the UI
kubectl port-forward svc/argocd-server -n argocd 8080:443
The initial admin password is auto-generated and stored as a base64-encoded Kubernetes Secret. Worth noting that kubectl get secret ... -o jsonpath only extracts the raw (still-encoded) value; it doesn't decode it:
$encoded = kubectl get secret argocd-initial-admin-secret -n argocd -o jsonpath="{.data.password}"
Logged into https://localhost:8080 with admin and the decoded password.
Defining Nginx declaratively
nginx/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.27
ports:
- containerPort: 80
nginx/service.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
type: ClusterIP
selector:
app: nginx
ports:
- port: 80
targetPort: 80
Pushed both to a GitHub repo — this repo is now the only thing Argo CD needs to watch.
git init
git add .
git commit -m "Initial nginx manifests"
git branch -M main
git remote add origin https://github.com//.git
git push -u origin main
Creating the Argo CD Application
Via the UI: + NEW APP, filled in the repo URL, path (nginx), destination (https://kubernetes.default.svc, namespace default), and set Sync Policy to Automatic with Self Heal enabled.
Result:
Status: Healthy ✅ Synced ✅
Confirmed at the raw Kubernetes level too:
kubectl get pods -n default
kubectl get svc -n default
An unrelated but real infrastructure lesson: resource contention
Partway through, kubectl started throwing TLS handshake timeout on every command — including basic ones like kubectl get nodes. This turned out to have nothing to do with Argo CD or k3d directly:
docker stats --no-stream
CONTAINER CPU %
k3d-devcluster-server-0 260.04%
dev-cluster-control-plane 341.70%
An unrelated, unused kind cluster (dev-cluster-control-plane) had been left running from an earlier session, consuming more CPU than my actual working cluster — combined, they exceeded Docker Desktop's allocated CPU ceiling and starved the k3s API server of the resources it needed to respond to requests.
docker rm -f dev-cluster-control-plane
CPU pressure dropped immediately, and the cluster became responsive again. A good reminder to periodically audit what's actually running in Docker Desktop, especially on a resource-constrained laptop.
Proving the GitOps loop actually works
This is the part that matters most — everything above was just getting the environment ready to demonstrate this.
- Automated sync from Git I changed replicas: 2 to replicas: 4 directly in the GitHub file editor and committed to main. Within Argo CD's poll interval, the Application flipped to OutOfSync, then automatically back to Synced as it applied the change — no kubectl command from me at any point. kubectl get pods -n default
NAME READY STATUS RESTARTS AGE
nginx-deployment-fd956d49d-8xcs8 1/1 Running 0 40m
nginx-deployment-fd956d49d-8xgcz 1/1 Running 0 46s
nginx-deployment-fd956d49d-dg9lz 1/1 Running 0 40m
nginx-deployment-fd956d49d-qzsf6 1/1 Running 0 46s
Four pods, matching Git — automatically.
- Self-heal from manual drift To prove the reverse direction, I manually scaled the deployment down, bypassing Git entirely: kubectl scale deployment nginx-deployment -n default --replicas=1 kubectl get pods -n default -w
Kubernetes terminated three pods, leaving one running. Within seconds, Argo CD's reconciliation loop detected that live state (1 replica) no longer matched Git (replicas: 4), and created three new pods to restore it — automatically:
nginx-deployment-fd956d49d-zn4lh 1/1 Running 0 14s
nginx-deployment-fd956d49d-qgn44 1/1 Running 0 10s
nginx-deployment-fd956d49d-mvfnd 1/1 Running 0 11s
That's the core GitOps guarantee, demonstrated end-to-end: whatever happens to the live cluster- a bad manual command, a crashed pod, config drift of any kind- the system converges back to what's declared in Git.
Why Argo CD is efficient (not just trendy)
Having now built this from scratch and broken it a few times along the way, here's what actually stands out about Argo CD versus a traditional push-based pipeline:
Security boundary. CI never needs cluster credentials; it only ever needs write access to Git. That's a meaningfully smaller attack surface.
Auditability. Every deployment is a Git commit with a message and an author. No more digging through CI logs to figure out what changed and when.
Consistency at scale. With the "app of apps" pattern, adding a new microservice to your GitOps setup is just a new file in a repo — not a new pipeline to write and maintain.
Self-correction. Configuration drift — from a manual hotfix someone forgot to codify, or a partial failure during a rollout — gets corrected automatically instead of silently persisting.
Two clear states per app. Sync status (does live match Git?) and Health status (are the resources actually working?) give you a fast, unambiguous signal for every deployed service.
Closing thoughts
If you're learning Kubernetes and GitOps, I'd genuinely recommend doing this the "hard way" once — build it locally, and don't route around the errors when they show up. Every issue I hit here (DNS, cgroup versions, hostname resolution, CRD size limits, resource contention) is a real thing that happens in actual infrastructure work, just usually with higher stakes. Debugging it on a local cluster, where nothing is at risk, is the cheapest place to build that instinct.
If you found this useful, I write about Kubernetes, DevOps tooling, and cloud infrastructure — feel free to follow for more hands-on breakdowns like this one.
Top comments (0)