DEV Community: Sandesh Pawar

Kubernetes Backup & Restore: Velero + MinIO Complete Guide

Sandesh Pawar — Fri, 27 Mar 2026 10:20:01 +0000

Kubernetes environments demand reliable backups to prevent data loss from misconfigurations or disasters. This step-by-step tutorial shows how to set up Velero with a local MinIO backend for namespace backups and restores using Helm on Minikube or any cluster.

Why Velero with MinIO?

Velero backs up Kubernetes resources like deployments, services, and volumes via the API server, storing them in object storage like MinIO (S3-compatible). It's the leading open-source tool for disaster recovery, cluster migration, and scheduled backups in 2026 production setups.

MinIO provides a lightweight, self-hosted S3 alternative ideal for development, air-gapped clusters, or cost-sensitive teams—avoiding cloud vendor lock-in.

Prerequisites

Ensure these are ready before starting:

Docker and Docker Compose installed.
Kubernetes cluster (e.g., Minikube v1.33+).
kubectl configured.
Helm v3+.
Update MinIO IP in configs (use hostname -I for host IP).

Common Mistake: Forgetting to expose MinIO's IP correctly leads to Velero connection timeouts.

Phase 1: Deploy MinIO Storage Backend

MinIO acts as your S3-compatible backup storage. Use Docker Compose for quick local setup.

Create docker-compose.yaml:

version: '3.7'
services:
  minio:
    image: minio/minio:latest
    container_name: velero-minio
    ports:
      - "9000:9000"  # API
      - "9001:9001"  # Console
    volumes:
      - minio-data:/data
    environment:
      MINIO_ROOT_USER: velero
      MINIO_ROOT_PASSWORD: Velero123StrongPass!
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3
    command: server /data --console-address ":9001"

  mc:  # MinIO Client with retry loop
    image: minio/mc:latest
    depends_on:
      minio:
        condition: service_healthy
    entrypoint: >
      /bin/sh -c "
      echo 'Waiting for MinIO health...';
      mc alias set local http://minio:9000 velero Velero123StrongPass!;
      mc mb local/backup-bucket || echo 'Bucket already exists';
      mc anonymous set public local/backup-bucket || echo 'Policy already set';
      mc ls local/backup-bucket;
      exit 0;
      "

volumes:
  minio-data:

Launch it:

docker compose up -d

Verify at http://<your-host-ip>:9001 (user: velero, pass: Velero123StrongPass!). Check backup-bucket exists.

Pro Tip: In production, use distributed MinIO with erasure coding for high availability.

Phase 2: Install Velero via Helm

Velero deploys as a cluster operator. Use VMware Tanzu Helm repo (latest stable as of March 2026).

Add repo:

helm repo add vmware-tanzu https://vmware-tanzu.github.io/helm-charts
helm repo update

Create velero-secret.yaml:

apiVersion: v1
kind: Secret
metadata:
  name: velero-secrets
  namespace: velero
type: Opaque
stringData:
  cloud: |
    [default]
    aws_access_key_id = velero
    aws_secret_access_key = Velero123StrongPass!

Apply:

kubectl create namespace velero
kubectl apply -f velero-secret.yaml

Create velero-values.yaml (update s3Url with your MinIO host IP, e.g., http://192.168.49.2:9000 for Minikube):

configuration:
  backupStorageLocation:
    - name: default
      provider: aws
      bucket: backup-bucket
      config:
        s3Url: http://YOUR_MINIO_IP:9000  # Update this!
        s3ForcePathStyle: "true"
      default: true
  defaultVolumesToRestic: true

credentials:
  existingSecret: velero-secrets

snapshotsEnabled: false  # Enable for PV snapshots in prod

initContainers:
   - name: velero-plugin-for-aws
     image: velero/velero-plugin-for-aws:v1.13.1  # Use latest compatible
     imagePullPolicy: IfNotPresent
     volumeMounts:
       - mountPath: /target
         name: plugins

Install:

helm install velero vmware-tanzu/velero -n velero -f velero-values.yaml
kubectl get pods -n velero  # Wait for Running

Common Mistake: Wrong s3Url IP—use minikube ip or host IP reachable from cluster pods.

Phase 3: Install Velero CLI

Download Velero CLI v1.18.0+ (matches server; check velero.io for latest):

wget https://github.com/vmware-tanzu/velero/releases/download/v1.18.0/velero-v1.18.0-linux-amd64.tar.gz
tar -xvf velero-v1.18.0-linux-amd64.tar.gz
sudo cp velero-v1.18.0-linux-amd64/velero /usr/local/bin/
velero version

Phase 4: Create and Verify Backup

Backup namespaces (create test-ns first if needed: kubectl create ns test-ns):

velero backup create test-backup --include-namespaces default,test-ns --wait
velero backup get
velero backup describe test-backup  # Should show Completed

Backups store in MinIO backup-bucket as tarballs with YAML manifests.
Pro Tip: Add --default-volumes-to-restic for PVC data; requires restic daemonsets.

Phase 5: Simulate Disaster and Restore

Delete test resources:

kubectl delete pod,service --all -n default  # Or specific: kubectl delete pod test -n default
kubectl delete ns test-ns
kubectl get po -A  # Confirm gone

Restore:

velero restore create --from-backup test-backup --wait
velero restore get
velero restore describe <restore-name>  # From output
kubectl get po -A  # Resources back!

Common Mistake: Restores skip existing resources by default—use --existing-resource-policy=update to overwrite.

Best Practices for Production

Aspect	Recommendation	Why It Matters
Scheduling	`velero schedule create daily --schedule="0 2 * * *"`	Automates daily backups at 2 AM.
Retention	`--ttl=30d`	Keeps 30 days; prevents storage bloat.
Monitoring	Export Prometheus metrics; alert on failures.	Catches issues before disasters.
Multi-location	Add secondary BSL for offsite DR.	Survives regional outages.
Hooks	Pre/post exec hooks for DB flush.	Ensures consistent stateful backups.

Test full E2E monthly. Velero shines for DevOps teams handling stateful apps like databases on Kubernetes.

Ready to implement? Drop a comment on your cluster setup or share your backup success! Subscribe for more Kubernetes/DevOps guides.

What is Kubernetes API Deprecation.

Sandesh Pawar — Sun, 08 Mar 2026 14:12:37 +0000

Introduction

Kubernetes evolves rapidly, and with each release, APIs are improved, stabilized, or removed. While this evolution helps improve reliability and performance, it also introduces challenges for administrators and developers maintaining Kubernetes clusters.

One common issue teams encounter during cluster upgrades is API deprecation. If deprecated APIs are still being used in manifests, Helm charts, or automation scripts, deployments can fail, applications may stop working, and in severe cases, production outages can occur.

In this guide, we will explain:

What Kubernetes API deprecation means
Why deprecated APIs can break your workloads
How to identify deprecated APIs in your cluster
A practical example of converting deprecated APIs using kubectl-convert

What is Kubernetes API Deprecation?

Kubernetes is built around an API-driven architecture. Every resource inside a Kubernetes cluster — such as Pods, Deployments, Services, and Ingress — is defined and managed using APIs.

Users interact with these APIs using:

kubectl CLI
REST API
Client libraries
Infrastructure tools like Terraform or Helm

Each Kubernetes resource is defined with an apiVersion field in the manifest.

Example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3

The apiVersion determines which version of the Kubernetes API is used to create or manage the resource.

As Kubernetes evolves, APIs go through different stages:

Alpha – Experimental and unstable
Beta – Stable but may change
GA (General Availability) – Production-ready

When an API version becomes outdated or is replaced by a better version, Kubernetes marks it as deprecated.

What Does API Deprecation Mean?

API deprecation means that an API version is still available but scheduled for removal in future Kubernetes releases.

Kubernetes maintains a strict policy regarding deprecated APIs:

Beta APIs are supported for at least 9 months or 3 releases
After that, they may be completely removed

For example:

extensions/v1beta1

This API version for Ingress resources was removed starting from Kubernetes v1.22.

If you attempt to deploy resources using a removed API version, Kubernetes returns an error.

Example error:

Error: UPGRADE FAILED: current release manifest contains removed kubernetes api(s)
error from kubernetes: unable to recognize "": no matches for kind "Ingress" in version "extensions/v1beta1"

Why Deprecated APIs Are Dangerous

Using deprecated APIs can cause serious operational issues.

1. Cluster Upgrade Failures

When upgrading Kubernetes clusters, manifests using deprecated APIs may fail to deploy.

This can break:

CI/CD pipelines
Helm upgrades
Infrastructure automation

2. Application Downtime

Applications depending on removed APIs may stop functioning after a cluster upgrade.

Even subtle API changes can introduce unexpected behavior and debugging complexity.

3. Compatibility Issues

Infrastructure tools such as:

Terraform providers
Helm charts
CI/CD pipelines

may depend on certain Kubernetes API versions. When those APIs are removed, compatibility breaks.

How to Identify API Versions in Kubernetes

You can list all supported API versions in your cluster using:

kubectl api-versions

Example output:

admissionregistration.k8s.io/v1
admissionregistration.k8s.io/v1beta1
apiextensions.k8s.io/v1
apiextensions.k8s.io/v1beta1
apiregistration.k8s.io/v1
apiregistration.k8s.io/v1beta1
apps/v1

This command helps administrators verify which APIs are currently supported by the cluster.

However, identifying which resources actually use deprecated APIs inside your cluster can be challenging.

Example: Deprecated Ingress API

Below is an example of an old Ingress manifest using a deprecated API version.

cat ingress-old.yaml

---
# Deprecated API version
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-space
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /video-service
        pathType: Prefix
        backend:
          serviceName: ingress-svc
          servicePort: 80

This API version (networking.k8s.io/v1beta1) is deprecated and removed in newer Kubernetes versions.

How to Fix Deprecated Kubernetes APIs

One of the easiest ways to migrate deprecated APIs is by using the kubectl-convert tool.

This tool automatically converts manifests from older API versions to the latest supported version.

Step 1: Download kubectl-convert

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl-convert"

Output:

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
100 55.8M  100 55.8M    0     0  55.3M      0  0:00:01  0:00:01 --:--:-- 55.3M

Step 2: Download the Checksum

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl-convert.sha256"

Step 3: Verify the Binary

echo "$(cat kubectl-convert.sha256) kubectl-convert" | sha256sum --check

Expected output:

kubectl-convert: OK

Step 4: Install the Tool

sudo install -o root -g root -m 0755 kubectl-convert /usr/local/bin/kubectl-convert

Step 5: Verify Installation

kubectl convert --help

This command displays the usage instructions for converting Kubernetes manifests.

Step 6: Convert Deprecated Manifest

The correct command is:

kubectl-convert -f ingress-old.yaml --output-version networking.k8s.io/v1

Converted output:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
  name: ingress-space
spec:
  rules:
  - http:
      paths:
      - backend:
          service:
            name: ingress-svc
            port:
              number: 80
        path: /video-service
        pathType: Prefix
status:
  loadBalancer: {}

Step 7: Save the Converted Manifest

kubectl-convert -f ingress-old.yaml --output-version networking.k8s.io/v1 > ingress-new.yaml

Verify the new file:

cat ingress-new.yaml

Step 8: Deploy the Updated Resource

k create -f ingress-new.yaml

Output:

ingress.networking.k8s.io/ingress-space created

Step 9: Verify the Deployment

k get ingress

Example output:

NAME            CLASS    HOSTS   ADDRESS   PORTS   AGE
ingress-space   <none>   *                 80      6s

Best Practices to Avoid API Deprecation Issues

To avoid disruptions caused by deprecated APIs, follow these best practices:

1. Monitor Kubernetes Release Notes

Always review release notes before upgrading clusters.

2. Regularly Audit Your Manifests

Check all YAML manifests, Helm charts, and automation scripts for deprecated APIs.

3. Use Tools for API Migration

Tools such as kubectl-convert help automate the migration process.

4. Test Upgrades in Staging Environments

Before upgrading production clusters, test upgrades in staging environments to identify deprecated API usage.

Conclusion

Kubernetes API deprecation is a normal part of the platform's evolution. However, failing to update deprecated APIs can lead to failed deployments, compatibility issues, and unexpected downtime.

Understanding how Kubernetes APIs evolve and knowing how to migrate deprecated resources is an essential skill for Kubernetes administrators and DevOps engineers.

By regularly auditing manifests and using tools like kubectl-convert, teams can ensure smooth cluster upgrades and maintain stable Kubernetes environments.

Adding Metrics to a Kubernetes Cluster for Pod and Node Resource Monitoring

Sandesh Pawar — Sun, 08 Feb 2026 11:23:09 +0000

Monitoring CPU and memory usage of pods and nodes is essential for keeping your Kubernetes cluster healthy and performant.
Without metrics, you are effectively running the cluster blind and cannot troubleshoot performance issues or scale workloads properly.

Why do we need metrics in a Kubernetes cluster?

Resource metrics help you:

See which pods and nodes are consuming the most CPU and memory.
Detect performance bottlenecks and noisy neighbors early.
Make informed scaling decisions for Horizontal/Vertical Pod Autoscalers.
Troubleshoot issues like OOMKills, throttling, and node pressure.

By default, many Kubernetes clusters do not ship with the Metrics Server installed.
You can verify this by checking for the metrics-server pod in the kube-system namespace:

kubectl get pods -n kube-system | grep metrics-server

If you do not see a metrics-server pod, it means metrics are not yet available in your cluster.

The Metrics Server is a cluster-wide aggregator that collects CPU and memory usage from kubelets on each node and exposes them through the Kubernetes Metrics API, which is what kubectl top node and kubectl top pod use under the hood.

Exploring the `kubectl top` command

Before installing anything, check what kubectl top can do using the help flag:

kubectl top --help

Once Metrics Server is running, you can use:

# Show node-level resource usage 
kubectl top nodes 
# Show pod-level resource usage in the current namespace 
kubectl top pods

If Metrics Server is not installed or not working, you will see an error similar to:

Error from server (ServiceUnavailable): the server is currently unable to handle the request (get nodes.metrics.k8s.io)

This indicates that the metrics.k8s.io API is not available in your cluster.

Installing Metrics Server in the Kubernetes cluster

Metrics Server can be installed by applying the official components.yaml manifest.

Run the following command:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

This creates the metrics-server deployment (and related resources) in the kube-system namespace.

You can then check the logs of the Metrics Server deployment:

kubectl logs -n kube-system deployment/metrics-server

If everything is working correctly, you should see logs indicating successful scraping of metrics from kubelet.

Troubleshooting: Metrics Server pod not Ready

Sometimes the metrics-server pod may stay in CrashLoopBackOff or NotReady state.

A common reason is that Metrics Server cannot establish a secure TLS connection to kubelet on the nodes, often due to certificate or hostname issues.

You have two broad options:

Fix the certificates (secure, recommended for production).
Relax TLS verification using --kubelet-insecure-tls (acceptable in labs, not recommended for production).

Option 1: Regenerate certificates securely (recommended)

If you are using a kubeadm-based cluster, you can regenerate control-plane certificates using kubeadm init phase commands, Check this link for better configuration kubelet-serving-certs.

Regenerate certificates (adjust the config path as per your environment):

sudo kubeadm init phase certs all --config /path/to/your/configuration/file.yaml

Restart kubelet on each control-plane and worker node:

sudo systemctl restart kubelet

After this, wait a few moments and re-check the Metrics Server pod status:

kubectl get pods -n kube-system | grep metrics-server

If the TLS setup is correct, the pod should move to Running and Ready.

Option 2: Use insecure TLS for Metrics Server (not recommended for production)

For development, lab, or non-critical clusters, you might decide to bypass strict TLS verification by adding the --kubelet-insecure-tls flag to the Metrics Server container arguments.

Edit the Metrics Server deployment:

kubectl edit deployment metrics-server -n kube-system

Under spec.template.spec.containers[0].args, add:

spec:   
  template:    
    spec:      
      containers:      
      - name: metrics-server        
        args:        
        - --kubelet-insecure-tls`

Save and exit the editor.

Kubernetes will restart the pod with the updated arguments.

Again, verify that the pod becomes Ready:

kubectl get pods -n kube-system | grep metrics-server

Note: Using --kubelet-insecure-tls disables certificate validation between Metrics Server and kubelet and can expose you to TLS man-in-the-middle attacks, so avoid this in production environments.

Verifying pod and node utilization with `kubectl top`

Once the Metrics Server pod is healthy and Ready, you can start querying live metrics.

Check node-level usage:

kubectl top nodes

Example output:

NAME           CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
worker-node1   120m         6%     800Mi           40%
worker-node2   90m          4%     700Mi           35%

Check pod-level usage in the current namespace:

kubectl top pods

You can also view metrics across all namespaces:

kubectl top pods -A

This gives you a quick, CLI-based view of which workloads are consuming the most resources, and is often the first step in investigating scaling or performance issues.

Reference

Official kubectl top command documentation.[kubernetes]

DEV Community: Sandesh Pawar

Kubernetes Backup & Restore: Velero + MinIO Complete Guide

Why Velero with MinIO?

Prerequisites

Phase 1: Deploy MinIO Storage Backend

Phase 2: Install Velero via Helm

Phase 3: Install Velero CLI

Phase 4: Create and Verify Backup

Phase 5: Simulate Disaster and Restore

Best Practices for Production

What is Kubernetes API Deprecation.

Introduction

What is Kubernetes API Deprecation?

What Does API Deprecation Mean?

Why Deprecated APIs Are Dangerous

1. Cluster Upgrade Failures

2. Application Downtime

3. Compatibility Issues

How to Identify API Versions in Kubernetes

Example: Deprecated Ingress API

How to Fix Deprecated Kubernetes APIs

Step 1: Download kubectl-convert

Step 2: Download the Checksum

Step 3: Verify the Binary

Step 4: Install the Tool

Step 5: Verify Installation

Step 6: Convert Deprecated Manifest

Step 7: Save the Converted Manifest

Step 8: Deploy the Updated Resource

Step 9: Verify the Deployment

Best Practices to Avoid API Deprecation Issues

1. Monitor Kubernetes Release Notes

2. Regularly Audit Your Manifests

3. Use Tools for API Migration

4. Test Upgrades in Staging Environments

Conclusion

Adding Metrics to a Kubernetes Cluster for Pod and Node Resource Monitoring

Why do we need metrics in a Kubernetes cluster?

Exploring the kubectl top command

Installing Metrics Server in the Kubernetes cluster

Troubleshooting: Metrics Server pod not Ready

Option 1: Regenerate certificates securely (recommended)

Option 2: Use insecure TLS for Metrics Server (not recommended for production)

Verifying pod and node utilization with kubectl top

Reference

Exploring the `kubectl top` command

Verifying pod and node utilization with `kubectl top`