DEV Community

Cover image for ArgoCD Silently Stuck at "Unknown" Sync Status? It Was a Kubernetes Version, Not ArgoCD
Vijaya Rajeev Bollu
Vijaya Rajeev Bollu

Posted on

ArgoCD Silently Stuck at "Unknown" Sync Status? It Was a Kubernetes Version, Not ArgoCD

#kubernetes #devops #git #argocd

The Setup

Adding a GitOps delivery layer on top of an existing 11-service Online Boutique deployment: ArgoCD ApplicationSet generating dev/staging/prod Applications, Argo Rollouts canary for the frontend service, Sealed Secrets for committing encrypted secrets. Everything ran on a single minikube cluster with three namespaces simulating the three environments — no extra cloud cost.

Every manifest applied cleanly. Every Application generated correctly. And then every single one of them sat at Sync Status: Unknown — forever.

The Investigation

No error in kubectl describe application online-boutique-dev. No crash in the ArgoCD application-controller pod. Just Unknown, indefinitely.

kubectl get applications -n argocd
NAME                     SYNC STATUS   HEALTH STATUS
online-boutique-dev      Unknown       Healthy
online-boutique-staging  Unknown       Missing
online-boutique-prod     Unknown       Missing
Enter fullscreen mode Exit fullscreen mode

Checked the application-controller logs directly — buried among routine warnings:

W... controller.go:139] slow openapi aggregation of "applicationsets.argoproj.io": 20.808634845s
E... timeout.go:140] "Post-timeout activity" ... path="/openapi/v2" result=null
Enter fullscreen mode Exit fullscreen mode

Tested the endpoint directly, ruling out a timeout:

kubectl get --raw /openapi/v2
# clean, fast 404 — not hanging, just not there
Enter fullscreen mode Exit fullscreen mode

The Reveal

Kubernetes 1.35.1 — minikube's default at the time — no longer serves /openapi/v2. ArgoCD v2.13.2's application-controller depends on that exact endpoint for its schema-based diff between live cluster state and Git. Without it, the controller can generate Applications and even sync them, but it can never actually compute whether they're in sync. It just sits at Unknown, silently, with nothing in the UI or logs pointing at the real cause.

The Fix

Pin a dedicated Kubernetes profile to a version that still serves the endpoint, without touching the original cluster:

minikube start -p v4-gitops --driver=docker --kubernetes-version=v1.30.0 --cpus=6 --memory=6500
kubectl get --raw=/openapi/v2 | head -c 100
# {"swagger":"2.0","info":{"title":"Kubernetes","version":"v1.30.0"},...}
Enter fullscreen mode Exit fullscreen mode

The Result 

NAME                  SYNC STATUS   HEALTH STATUS
online-boutique-dev   Synced        Healthy
Enter fullscreen mode Exit fullscreen mode

From there, the rest of the GitOps flow proved out live. Manually scaling a deployment outside Git:

NAME           DESIRED   READY
adservice      5         1
SYNC        HEALTH
OutOfSync   Progressing
Enter fullscreen mode Exit fullscreen mode

...and self-heal reverting it, unattended:

NAME           DESIRED   READY
adservice      1         1
SYNC     HEALTH    PHASE
Synced   Healthy   Succeeded
Enter fullscreen mode Exit fullscreen mode

And a genuinely bad image commit going Degraded, fixed the GitOps way — not with kubectl:

REVISION      SYNC        HEALTH     PHASE
14151c9a...   OutOfSync   Degraded   Running

$ git revert 14151c9a
[f0e6a50f] Revert "test(gitops): deploy invalid frontend image"

NAME                  SYNC STATUS   HEALTH STATUS   REVISION
online-boutique-dev   Synced        Healthy         f0e6a50f...
Enter fullscreen mode Exit fullscreen mode

What I Learned

1. A missing API endpoint can fail silent, not loud. No crash, no error surfaced anywhere obvious — just a status field that never resolves. Check the controller's own logs before assuming the manifests are wrong.

2. Version pinning is a legitimate fix, not a workaround to feel bad about. A dedicated profile pinned to a known-good Kubernetes version, kept separate from the primary cluster, cost nothing and solved it cleanly.

3. Self-heal and git-revert are two different rollback tools. Self-heal fixes drift — someone changed the cluster without touching Git. git revert fixes a bad commit — Git said something wrong, and Git is what has to say something right again. Neither substitutes for the other.

4. The whole exercise ran on one minikube cluster with three namespaces, not three separate clusters — zero cloud cost to prove a real multi-environment GitOps promotion model end to end.

Try It

GitHub: https://github.com/ThinkWithOps/thinkwithops-online-boutique-production

Youtube: https://youtu.be/2CJqoVwJEqY

git clone https://github.com/ThinkWithOps/thinkwithops-online-boutique-production.git
cd thinkwithops-online-boutique-production
git checkout v4.0-gitops-argocd
minikube start -p v4-gitops --driver=docker --kubernetes-version=v1.30.0 --cpus=6 --memory=6500
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/v2.13.2/manifests/install.yaml
kubectl apply -f argocd/project.yaml -f argocd/applicationset.yaml
kubectl get applications -n argocd
Enter fullscreen mode Exit fullscreen mode

Has anyone else hit a silent /openapi/v2 gap after a Kubernetes minor version bump? Curious how many other controllers quietly depend on that same endpoint.

Top comments (0)