DEV Community

Devam Parikh
Devam Parikh

Posted on

GitOps Without Surprises: Generate, Review, Then Reconcile

Rendered manifest workflow

GitOps is supposed to make production changes boring: review a pull request, merge it, and let the controller reconcile the desired state.

But there is a subtle gap in many Kubernetes delivery flows.

When a repository stores Helm charts, Helm values, Kustomize bases, or overlays, Git often contains the inputs to the desired state. The actual Kubernetes objects are created later, usually inside Argo CD at sync time.

That is convenient. It is also where surprises hide.

The problem: Git may not show what Kubernetes will receive

Helm and Kustomize are excellent authoring tools. They reduce duplication, let teams share common patterns, and make multi-environment delivery manageable.

The problem is not the tools. The problem is where rendering happens.

If rendering happens only inside the GitOps controller, reviewers may approve a change without seeing the exact final YAML. A one-line values change can modify a Deployment, Service, ConfigMap, HPA, RBAC rule, or annotation in ways that are not obvious from the pull request.

That hurts in a few places:

  • Reviews focus on abstraction changes, not final Kubernetes objects.
  • Audits require re-rendering with matching tool versions and inputs.
  • Rollbacks are harder to reason about.
  • Incident debugging starts with “what did Argo CD actually apply?”
  • Policy and API deprecation checks happen later than they should.

For platform teams, this becomes painful as more teams and environments onboard onto the same GitOps platform.

The pattern: generate, review, then reconcile

A safer GitOps flow looks like this:

  1. Developers keep authoring with Helm, Helmfile, or Kustomize.
  2. CI renders the final Kubernetes manifests.
  3. CI commits the rendered output back to the pull request branch.
  4. Reviewers inspect the exact YAML diff.
  5. Policy, security, and API deprecation checks run against rendered objects.
  6. Argo CD reconciles from generated-manifest/.

In other words:

Author with abstractions, but reconcile reviewed YAML.

This is usually called the rendered manifests pattern or hydrated manifests pattern.

Why this works well with Argo CD

Argo CD ultimately applies Kubernetes manifests. Even when you point Argo CD at a Helm chart, Argo CD renders that chart and applies the resulting YAML.

The rendered-manifest pattern moves that rendering step earlier so humans and automation can inspect the result before reconciliation.

A deployment repository can look like this:

k8s/
  demo/
    demo-api/
      Chart.yaml
      values.yaml
      staging-nam-values.yaml
      production-nam-values.yaml

generated-manifest/
  staging/
    nam/
      demo/
        demo-api/
          manifest.lock.yaml
  production/
    nam/
      demo/
        demo-api/
          manifest.lock.yaml
Enter fullscreen mode Exit fullscreen mode

Argo CD then points to the rendered path:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: demo-api-staging
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/your-deploy-repo.git
    targetRevision: main
    path: generated-manifest/staging/nam/demo/demo-api
  destination:
    server: https://kubernetes.default.svc
    namespace: demo
Enter fullscreen mode Exit fullscreen mode

Now a pull request does not only show a values file change. It also shows the resulting manifest.lock.yaml change.

That is the part reviewers actually need.

A reusable GitHub Action for this flow

I created a reusable GitHub Action for this pattern:

https://github.com/Devamparikh/gitops-render-manifests-action

It supports:

  • helm
  • helmfile
  • kustomize
  • none for repositories that already store raw YAML
  • optional shared-directory detection
  • optional commit-back to the pull request branch
  • optional Kubernetes API deprecation checks with Pluto

A basic caller workflow looks like this:

name: Render Kubernetes manifests

on:
  pull_request:
    paths:
      - "k8s/**"
      - "common-values/**"
      - ".github/workflows/render-manifests.yaml"

permissions:
  contents: write
  pull-requests: write

jobs:
  render:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout pull request branch
        uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.head.ref }}
          token: ${{ secrets.GITHUB_TOKEN }}
          fetch-depth: 0

      - name: Render manifests
        uses: Devamparikh/gitops-render-manifests-action@v0
        with:
          package_manager: helm
          shared_dirs: common-values
          target_k8s_version: v1.33.0
Enter fullscreen mode Exit fullscreen mode

For Kustomize:

- name: Render manifests
  uses: Devamparikh/gitops-render-manifests-action@v0
  with:
    package_manager: kustomize
    kustomize_load_restrictor_none: "true"
Enter fullscreen mode Exit fullscreen mode

For Helmfile:

- name: Render manifests
  uses: Devamparikh/gitops-render-manifests-action@v0
  with:
    package_manager: helmfile
    shared_dirs: common-values,charts
    parallel: "4"
Enter fullscreen mode Exit fullscreen mode

The review experience changes

Before rendered manifests, a reviewer might see:

replicaCount: 3
Enter fullscreen mode Exit fullscreen mode

After rendered manifests, the reviewer also sees:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-api
spec:
  replicas: 3
Enter fullscreen mode Exit fullscreen mode

That sounds small, but it changes the conversation.

Reviewers can ask:

  • Did this change the expected namespace?
  • Did the image tag move?
  • Did RBAC expand?
  • Did resource requests change?
  • Did labels, annotations, selectors, or topology rules change?
  • Will this API still work on the target Kubernetes version?

Those are production questions. They deserve to be answered before Argo CD syncs.

Trade-offs

This pattern is not free.

It adds generated files to Git. It can make pull requests larger. It requires CI to have the right rendering tools. It also requires a clear repository contract so teams know where source inputs live and where generated manifests go.

You should also be careful with secrets. Rendered manifests should not commit plaintext secrets. Use External Secrets Operator, Sealed Secrets, SOPS, secret references, or another secret-management pattern that fits your organization.

The pattern is worth considering when:

  • multiple teams deploy through a shared platform,
  • reviewers need stronger auditability,
  • regulated environments need explicit change records,
  • Argo CD sync-time rendering has caused surprises,
  • you want policy and deprecation checks before sync,
  • you want ApplicationSets to reconcile plain generated YAML.

For very small teams or simple services, direct Helm or Kustomize in Argo CD may still be enough.

Where this fits with Source Hydrator and other tools

Argo CD Source Hydrator is making hydrated manifests a more first-class workflow inside Argo CD itself. Kargo and OCI-based promotion patterns can also solve parts of this problem, especially around environment promotion.

The action in this post is intentionally simple: it gives teams a CI-based way to try the pattern without changing their entire platform.

Start small. Render one service. Review the manifest diff. Point Argo CD at the generated path. Then decide whether the pattern should become part of your platform contract.

Final thought

GitOps is not just “put YAML in Git.”

The stronger promise is this:

The desired state that humans review is the same desired state the controller reconciles.

That is the heart of GitOps without surprises.

Top comments (0)