Last month I migrated two production clusters from GitHub Actions-only deployments to a hybrid GitOps setup with ArgoCD. The trigger? A misconfigured workflow secret that exposed an AWS key for 11 minutes before our scanner caught it. Nothing happened — this time. But it made me rethink how we handle the boundary between CI and CD.
Here’s what I learned about running both tools securely in production, and when each one actually makes sense.
GitOps: Let Git Be the Only Way In
GitOps treats Git as the single source of truth for your cluster state. You define what should exist in a repo, and an agent like ArgoCD or Flux continuously reconciles reality to match. No one SSHs into production. No one runs kubectl apply by hand.
The security model here is simple: the cluster pulls config from Git. The agent runs inside the cluster with the minimum permissions needed to apply manifests. Your developers never need direct cluster access — they open a PR, it gets reviewed, merged, and the agent picks it up.
This is a massive reduction in attack surface. In a traditional CI/CD model, your pipeline needs credentials to push to the cluster. With GitOps, those credentials stay inside the cluster.
Here’s a basic ArgoCD Application manifest:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-app
spec:
source:
repoURL: https://github.com/my-org/my-app-config
targetRevision: HEAD
path: .
destination:
server: https://kubernetes.default.svc
namespace: my-app-namespace
syncPolicy:
automated:
prune: true
selfHeal: true
The selfHeal: true setting is important — if someone does manage to modify a resource directly in the cluster, ArgoCD will revert it to match Git. That’s drift detection for free.
One gotcha: make sure you enforce branch protection on your GitOps repos. I’ve seen teams set up ArgoCD perfectly, then leave the main branch unprotected. Anyone with repo write access can then deploy anything. Always require reviews and status checks.
GitHub Actions: Powerful but Exposed
GitHub Actions is a different animal. It’s event-driven — push code, open a PR, hit a schedule, and workflows fire. That flexibility is exactly what makes it harder to secure.
Every GitHub Actions workflow that deploys to production needs some form of credential. Even with OIDC federation (which you should absolutely be using), there are still risks. Third-party actions can be compromised. Workflow files can be modified in feature branches. Secrets can leak through step outputs if you’re not careful.
Here’s a typical deployment workflow:
name: Deploy to Kubernetes
on:
push:
branches:
- main
jobs:
deploy:
runs-on: ubuntu-latest
environment: production
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Configure kubectl
uses: azure/setup-kubectl@v3
- name: Deploy application
run: kubectl apply -f k8s/deployment.yaml
Notice the environment: production — that enables environment protection rules, so deployments require manual approval. Without it, any push to main goes straight to prod. I always set this up, even on small projects.
The bigger issue is that GitHub Actions workflows are imperative. You’re writing step-by-step instructions that execute on a runner with network access. Compare that to GitOps where you declare “this is what should exist” and an agent figures out the rest. The imperative model has more moving parts, and more places for things to go wrong.
Where Each One Wins on Security
After running both in production, here’s how I’d break it down:
Access control — GitOps wins. The agent pulls from Git, so your CI system never needs cluster credentials.
Secret handling — GitOps is cleaner. You pair it with External Secrets Operator or Sealed Secrets and your Git repo never contains actual credentials. GitHub Actions has encrypted secrets, but they’re injected into the runner environment at build time.
Audit trail — GitOps. Every change is a Git commit with an author, timestamp, and review trail.
Flexibility — GitHub Actions. Running test suites, building container images, scanning for vulnerabilities — these are CI tasks, and GitHub Actions handles them well.
Speed of setup — GitHub Actions. You can go from zero to deployed in an afternoon.
The Hybrid Approach (What Actually Works)
Most teams end up running both, and it’s the right call. Use GitHub Actions for CI — build, test, scan, push images. Use GitOps for CD — let ArgoCD or Flux handle what’s running in the cluster.
The boundary is important: GitHub Actions should never directly kubectl apply to production. Instead, it updates the image tag in your GitOps repo (via a PR or direct commit to a deploy branch), and the GitOps agent picks it up.
This gives you:
- Full Git audit trail for all production changes
- No cluster credentials in your CI system
- Automatic drift detection and self-healing
- The flexibility of GitHub Actions for everything that isn’t deployment
Adding Security Scanning to the Pipeline
Whether you use GitOps, GitHub Actions, or both, you need automated security checks. I run Trivy on every image build and OPA/Gatekeeper for policy enforcement in the cluster.
The exit-code: 1 setting in Trivy means the workflow fails if critical or high vulnerabilities are found. No exceptions. It’s caught real issues — including supply chain problems in base images that would have made it to prod otherwise.
What I’d Do Starting Fresh
If I were setting up a new production Kubernetes environment today:
- ArgoCD for all cluster deployments, with strict branch protection and required reviews
- GitHub Actions for CI only — build, test, scan, push to registry
- External Secrets Operator for credentials, never stored in Git
- OPA Gatekeeper for policy enforcement
- Trivy in CI, plus periodic scanning of running images
The investment in GitOps pays off fast once you’re past the initial setup. The first time you need to answer “what changed?” during a 2 AM incident and the answer is right there in the Git log, you’ll be glad you did it.
Originally published at orthogonal.info
Top comments (0)