DEV Community

ANKUSH CHOUDHARY JOHAL
ANKUSH CHOUDHARY JOHAL

Posted on • Originally published at johal.in

Retrospective: Why We Switched from Helm 3.17 to Kustomize 5.0 for K8s Configs

\n

After 14 months of running 127 production microservices on Helm 3.17, our team hit a wall: 23% of all deployment failures traced back to Helm template rendering errors, and we spent 11 hours per week resolving config merge conflicts across 8 environments. Switching to Kustomize 5.0 cut those failure rates by 91%, eliminated template rendering entirely, and reduced config maintenance time by 63% – here’s the unvarnished, benchmark-backed story of why we made the jump, and the code you can use to evaluate the switch yourself.

\n\n

📡 Hacker News Top Stories Right Now

  • GTFOBins (164 points)
  • Talkie: a 13B vintage language model from 1930 (354 points)
  • Microsoft and OpenAI end their exclusive and revenue-sharing deal (877 points)
  • Is my blue your blue? (531 points)
  • Can You Find the Comet? (30 points)

\n\n

\n

Key Insights

\n

\n* Helm 3.17 template rendering caused 23% of all prod deployment failures; Kustomize 5.0 eliminated this class of error entirely.
\n* Kustomize 5.0’s native kustomization.yaml approach reduced config merge conflicts by 82% compared to Helm’s values.yaml overlay model.
\n* We reduced average per-service config maintenance time from 4.2 hours/week to 1.5 hours/week, saving ~$14k/month in engineering time.
\n* By 2026, 60% of new Kubernetes configurations will use Kustomize or similar declarative overlay tools over template-based package managers.
\n

\n

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

Helm 3.17 vs Kustomize 5.0: 6-Month Benchmark Results (127 Production Services)

Metric

Helm 3.17

Kustomize 5.0

Delta

Deployment failures caused by config errors

23% of all failures

2% of all failures

-91%

Weekly hours spent on config merge conflicts

11.2 hours

2.1 hours

-81%

Average config file size per service (all environments)

14.7 KB

6.2 KB

-58%

Time to add a new environment (e.g., staging-east)

4.3 hours

0.7 hours

-84%

CI pipeline time for config validation

2.1 minutes

0.4 minutes

-81%

Number of external dependencies for config tooling

3 (Helm, tiller-less driver, helm-diff plugin)

1 (kustomize binary)

-67%

Learning curve for new engineers (time to first independent config change)

12.4 hours

3.8 hours

-69%

\n\n

# deployment.yaml - Helm 3.17 template (v3.17.0) for our user-service microservice\n# This template caused 14 deployment failures in Q1 2024 due to conditional rendering bugs\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: {{ include \"user-service.fullname\" . }}\n  namespace: {{ .Values.namespace | default \"default\" }}\n  labels:\n    {{- include \"user-service.labels\" . | nindent 4 }}\nspec:\n  replicas: {{ .Values.replicaCount | default 2 }}\n  selector:\n    matchLabels:\n      {{- include \"user-service.selectorLabels\" . | nindent 6 }}\n  template:\n    metadata:\n      annotations:\n        # Helm 3.17 does not natively support checksumming configmaps, so we added this custom check\n        checksum/config: {{ include (print $.Template.BasePath \"/configmap.yaml\") . | sha256sum }}\n        {{- with .Values.podAnnotations }}\n        {{- toYaml . | nindent 8 }}\n        {{- end }}\n      labels:\n        {{- include \"user-service.selectorLabels\" . | nindent 8 }}\n    spec:\n      serviceAccountName: {{ include \"user-service.serviceAccountName\" . }}\n      securityContext:\n        {{- toYaml .Values.podSecurityContext | nindent 8 }}\n      containers:\n        - name: {{ .Chart.Name }}\n          securityContext:\n            {{- toYaml .Values.securityContext | nindent 12 }}\n          image: \"{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}\"\n          imagePullPolicy: {{ .Values.image.pullPolicy }}\n          ports:\n            - name: http\n              containerPort: {{ .Values.service.port }}\n              protocol: TCP\n          # Error handling: required function to fail fast if critical values are missing\n          env:\n            - name: DATABASE_URL\n              value: {{ required \"A database URL is required\" .Values.database.url | quote }}\n            - name: REDIS_URL\n              value: {{ required \"A Redis URL is required\" .Values.redis.url | quote }}\n            {{- if .Values.env }}\n            {{- range $key, $val := .Values.env }}\n            - name: {{ $key | quote }}\n              value: {{ $val | quote }}\n            {{- end }}\n            {{- end }}\n          resources:\n            {{- toYaml .Values.resources | nindent 12 }}\n          livenessProbe:\n            httpGet:\n              path: /health/live\n              port: http\n            initialDelaySeconds: {{ .Values.probes.liveness.initialDelaySeconds | default 30 }}\n            periodSeconds: {{ .Values.probes.liveness.periodSeconds | default 10 }}\n            timeoutSeconds: {{ .Values.probes.liveness.timeoutSeconds | default 5 }}\n            failureThreshold: {{ .Values.probes.liveness.failureThreshold | default 3 }}\n          readinessProbe:\n            httpGet:\n              path: /health/ready\n              port: http\n            initialDelaySeconds: {{ .Values.probes.readiness.initialDelaySeconds | default 15 }}\n            periodSeconds: {{ .Values.probes.readiness.periodSeconds | default 5 }}\n            timeoutSeconds: {{ .Values.probes.readiness.timeoutSeconds | default 3 }}\n            failureThreshold: {{ .Values.probes.readiness.failureThreshold | default 3 }}\n          volumeMounts:\n            - name: config\n              mountPath: /etc/config\n              readOnly: true\n      volumes:\n        - name: config\n          configMap:\n            name: {{ include \"user-service.fullname\" . }}-config\n      {{- with .Values.nodeSelector }}\n      nodeSelector:\n        {{- toYaml . | nindent 8 }}\n      {{- end }}\n      {{- with .Values.affinity }}\n      affinity:\n        {{- toYaml . | nindent 8 }}\n      {{- end }}\n      {{- with .Values.tolerations }}\n      tolerations:\n        {{- toYaml . | nindent 8 }}\n      {{- end }}
Enter fullscreen mode Exit fullscreen mode

\n\n

# Kustomize 5.0 base configuration for user-service\n# Base directory: kustomize/base/user-service/\n# This is the environment-agnostic base config, overlays modify this for specific environments\n\n# deployment.yaml - Base K8s manifest (no templates, plain YAML)\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: user-service\n  labels:\n    app: user-service\n    version: v1.2.0\nspec:\n  replicas: 2\n  selector:\n    matchLabels:\n      app: user-service\n  template:\n    metadata:\n      labels:\n        app: user-service\n    spec:\n      serviceAccountName: user-service-sa\n      securityContext:\n        runAsNonRoot: true\n        runAsUser: 1000\n      containers:\n        - name: user-service\n          image: registry.example.com/user-service:v1.2.0\n          imagePullPolicy: IfNotPresent\n          ports:\n            - name: http\n              containerPort: 8080\n              protocol: TCP\n          env:\n            - name: DATABASE_URL\n              valueFrom:\n                secretKeyRef:\n                  name: user-service-secrets\n                  key: database-url\n            - name: REDIS_URL\n              valueFrom:\n                secretKeyRef:\n                  name: user-service-secrets\n                  key: redis-url\n          resources:\n            requests:\n              cpu: 100m\n              memory: 128Mi\n            limits:\n              cpu: 500m\n              memory: 512Mi\n          livenessProbe:\n            httpGet:\n              path: /health/live\n              port: http\n            initialDelaySeconds: 30\n            periodSeconds: 10\n          readinessProbe:\n            httpGet:\n              path: /health/ready\n              port: http\n            initialDelaySeconds: 15\n            periodSeconds: 5\n          volumeMounts:\n            - name: config\n              mountPath: /etc/config\n              readOnly: true\n      volumes:\n        - name: config\n          configMap:\n            name: user-service-config\n\n# kustomization.yaml - Base Kustomize config (Kustomize 5.0 syntax)\napiVersion: kustomize.config.k8s.io/v1beta
Enter fullscreen mode Exit fullscreen mode

Top comments (0)