Writing Kubernetes Manifests: From Beginner to Advanced

Kubernetes manifests are essential for defining, deploying, and managing workloads in Kubernetes clusters. As a DevOps engineer, mastering the creation of Kubernetes manifests helps you efficiently manage containerized applications, ensure scalability, and streamline deployments.

This article takes you from the fundamentals of writing Kubernetes manifests to advanced techniques and best practices. Whether you’re new to Kubernetes or looking to enhance your expertise, this guide will equip you with the knowledge to write effective and optimized manifests.

---

Table of Contents

1. What Are Kubernetes Manifests?
2. Why Learn Kubernetes Manifests?
3. Understanding Manifest Structure
4. Basic Manifest Examples
5. Intermediate Manifest Concepts
6. Advanced Manifest Techniques
7. Best Practices for Writing Manifests
8. Common Mistakes and How to Avoid Them
9. Conclusion

---

1. What Are Kubernetes Manifests?

A Kubernetes manifest is a configuration file, usually written in YAML or JSON, used to describe the desired state of resources in a Kubernetes cluster. It provides a declarative way to tell Kubernetes what you want your application and infrastructure to look like. Kubernetes then reconciles the actual state of the cluster with the desired state defined in these manifests.

Key Features:

1. Declarative Configuration:
   
   You describe the desired end-state (e.g., “I want 3 replicas of my application running”), and Kubernetes takes care of ensuring this state is achieved.

2. Portability:
   
   Kubernetes manifests can be reused across different environments (e.g., development, staging, and production). For example, you can deploy the same manifest in any Kubernetes cluster, ensuring consistent deployments.

3. Automation:
   
   Manifests integrate seamlessly with CI/CD pipelines, enabling automated application builds, tests, and deployments.

---

Example Use Case:

Imagine you have a web application that runs on multiple replicas, behind a load balancer.

• Without Kubernetes: You would manually configure servers, set up load balancing, and monitor each instance.
• With Kubernetes Manifests: All this complexity is abstracted into a few lines of YAML. Kubernetes ensures the replicas are running, load balancing is in place, and failed instances are automatically replaced.

---

2. Why Learn Kubernetes Manifests?

For DevOps engineers, Kubernetes manifests are a core skill. Understanding them unlocks powerful capabilities in managing and deploying containerized applications.

Reasons to Learn:

1. Streamlined Deployments:
   
   Automate complex setups (e.g., multi-container applications) with ease by defining resources declaratively.

2. Scalability:
   
   Modify manifests to scale your application vertically (more resources per container) or horizontally (more containers).

3. Consistency Across Environments:
   
   Use the same manifests for local testing, staging, and production to ensure consistency and reduce deployment errors.

4. Customization:
   
   Tailor manifests to specific workloads, such as deploying applications in specific namespaces or attaching storage volumes for stateful applications.

5. Integration with DevOps Workflows:
   
   Kubernetes manifests work seamlessly with CI/CD tools like Jenkins, GitHub Actions, and GitLab CI for automated deployments.

Real-World Benefits:

• Faster deployment times.
• Reduced manual intervention.
• Easier debugging of infrastructure and application issues.

---

3. Understanding Manifest Structure

Before writing manifests, it’s crucial to understand their structure. Each Kubernetes manifest follows a standard format that specifies what resource to create and its desired configuration.

Core Components of a Manifest:

1. apiVersion

Specifies the Kubernetes API version for the resource type. Each resource type (e.g., Pod, Service) belongs to a specific API group and version.

• Example:
  • v1 for Pod and Service.
  • apps/v1 for Deployment.
• Why It Matters: Using the correct API version ensures compatibility with the Kubernetes cluster version.

---

2. kind

Defines the type of resource being created. Common resource types include:

• Pod
• Deployment
• Service
• ConfigMap
• Secret

Each resource serves a specific purpose. For instance, a Pod represents a single container or a group of tightly coupled containers, while a Deployment manages replica sets and rolling updates.

---

3. metadata

Contains identifying information about the resource, such as:

• name: A unique name for the resource.
• labels: Key-value pairs used to organize and query resources.
• namespace: Isolates the resource within a specific environment (e.g., dev, prod).

Example:

metadata:
  name: my-app
  namespace: development
  labels:
    app: my-app
    environment: dev

---

4. spec

The heart of the manifest, defining the desired state of the resource. This section is resource-specific. For example:

• In a Pod, spec defines the containers, images, and ports.
• In a Service, spec defines how to expose applications (e.g., via ClusterIP, NodePort, or LoadBalancer).

---

YAML vs. JSON

Although Kubernetes supports both YAML and JSON, YAML is more commonly used because:

• It’s human-readable and easier to write.
• It supports comments, which are helpful for documentation.

Example Comparison:

YAML:

kind: Pod
apiVersion: v1
metadata:
  name: my-pod
spec:
  containers:
  - name: nginx
    image: nginx:latest

JSON:

{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "my-pod"
  },
  "spec": {
    "containers": [
      {
        "name": "nginx",
        "image": "nginx:latest"
      }
    ]
  }
}

---

4. Basic Manifest Examples

Let’s look at a few simple manifests for common Kubernetes resources.

---

4.1 Pod Manifest

A Pod is the smallest deployable unit in Kubernetes.

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  labels:
    app: web
spec:
  containers:
  - name: nginx
    image: nginx:1.23
    ports:
    - containerPort: 80

Explanation:

• Creates a single Pod named my-pod running an nginx container.
• Labels (app: web) help group and query resources.
• The container exposes port 80 internally.

---

4.2 Deployment Manifest

Deployments provide declarative updates for Pods and ReplicaSets.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.23
        ports:
        - containerPort: 80

Explanation:

• Deploys three replicas of the nginx Pod.
• The selector ensures the Pods are linked to the Deployment.
• The template defines the Pod specification for the replicas.

---

4.3 Service Manifest

Services expose Pods to the network.

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
  type: ClusterIP

Explanation:

• Exposes Pods labeled app: nginx.
• Maps port 80 of the Service to port 80 of the Pods.
• ClusterIP is the default service type, exposing the Pods within the cluster.

---

4.4 ConfigMap Manifest

ConfigMaps store configuration data in key-value pairs.

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  log_level: debug
  environment: production

Explanation:

• Stores non-sensitive application configurations, such as environment variables or logging levels.
• Accessible to Pods as environment variables or mounted volumes.

---

5. Intermediate Manifest Concepts

As you grow in Kubernetes expertise, understanding intermediate concepts like multi-container pods, resource management, and secrets becomes essential. These enhance your ability to handle more complex workloads efficiently.

---

5.1 Multi-Container Pods

A single Pod can host multiple containers that share the same lifecycle and storage. This is useful when containers need to work together closely, such as a main application container and a sidecar container for logging or monitoring.

Example:

apiVersion: v1
kind: Pod
metadata:
  name: multi-container-pod
spec:
  containers:
  - name: app
    image: my-app:latest
  - name: sidecar-logger
    image: fluentd:latest
    volumeMounts:
    - name: shared-logs
      mountPath: /var/log
  volumes:
  - name: shared-logs
    emptyDir: {}

Explanation:

• The app container runs the main application, while the sidecar-logger collects logs.
• Both containers share a temporary volume (emptyDir).

---

5.2 Resource Requests and Limits

Properly managing CPU and memory ensures that no single Pod consumes excessive cluster resources.

Example:

resources:
  requests:
    memory: "128Mi"
    cpu: "250m"
  limits:
    memory: "256Mi"
    cpu: "500m"

Explanation:

• Requests: The guaranteed minimum resources allocated to a container.
• Limits: The maximum resources a container can use.

By specifying these values, you prevent resource starvation and overloading.

---

5.3 Secrets

Secrets store sensitive data like passwords, tokens, and SSH keys securely.

Example:

apiVersion: v1
kind: Secret
metadata:
  name: db-secret
type: Opaque
data:
  username: dXNlcm5hbWU=  # Base64-encoded 'username'
  password: cGFzc3dvcmQ=  # Base64-encoded 'password'

Explanation:

• Secrets are Base64-encoded and stored securely.
• They can be injected into Pods as environment variables or mounted as files.

---

5.4 Persistent Storage

For stateful applications, persistent storage ensures data is not lost when a Pod is terminated.

Example (Persistent Volume Claim):

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

Explanation:

• The PersistentVolumeClaim requests 1GB of storage.
• This is used by Pods requiring persistent data.

---

6. Advanced Manifest Techniques

As your Kubernetes usage matures, advanced techniques enable you to optimize workloads, enhance security, and streamline operations.

---

6.1 Multi-Stage Manifests

Break your manifests into multiple stages for modularity and reusability. Use Helm charts or Kustomize for templating and environment-specific configurations.

Example with Kustomize:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- base/deployment.yaml
- base/service.yaml
patchesStrategicMerge:
- overlays/dev/replicas.yaml
- overlays/prod/limits.yaml

Explanation:

• Kustomize simplifies creating environment-specific configurations without duplicating manifests.

---

6.2 Rolling Updates

Deploy updates with zero downtime using rolling updates.

Example (Deployment):

strategy:
  type: RollingUpdate
  rollingUpdate:
    maxUnavailable: 1
    maxSurge: 1

Explanation:

• maxUnavailable: The number of Pods that can be unavailable during an update.
• maxSurge: The number of extra Pods allowed during an update.

---

6.3 Horizontal Pod Autoscaling

Automatically scale Pods based on CPU, memory, or custom metrics.

Example:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        averageUtilization: 70

Explanation:

• Scales the application between 2 and 10 replicas based on CPU usage.

---

7. Best Practices for Writing Manifests

1. Organize Resources with Namespaces:
   
   Use namespaces for logical separation of environments or applications (e.g., dev, staging, prod).

2. Use Labels and Annotations:
   
   Attach metadata to resources to enable efficient management and querying.

3. Avoid Hardcoding Values:
   
   Use ConfigMaps and Secrets to separate configuration from code.

4. Version Control Your Manifests:
   
   Store manifests in Git repositories for tracking and collaboration.

5. Validate Manifests Before Applying:
   
   Use kubectl apply --dry-run=client to check for syntax errors.

6. Use Templating Tools:
   
   Helm and Kustomize simplify creating reusable, environment-specific configurations.

---

8. Common Mistakes and How to Avoid Them

1. Incorrect Indentation in YAML:
   
   YAML is sensitive to spaces. Use proper indentation to avoid syntax errors.

2. Skipping Resource Requests and Limits:
   
   Failing to define these can lead to resource starvation or cluster overload.

3. Not Using Probes:
   
   Without liveness and readiness probes, Kubernetes cannot detect or restart failing containers.

4. Hardcoding Sensitive Data:
   
   Always use Secrets to store sensitive information like passwords or API keys.

5. Forgetting Labels and Selectors:
   
   Misaligned labels and selectors can break connections between resources like Deployments and Services.

---

9. Conclusion

Kubernetes manifests are the cornerstone of managing workloads in Kubernetes. By mastering the basics, progressing to intermediate concepts, and leveraging advanced techniques, you can efficiently deploy and manage containerized applications at scale.

As a DevOps engineer, understanding and applying these concepts will make your Kubernetes deployments robust, scalable, and maintainable. Keep experimenting, validate your manifests, and incorporate best practices to stay ahead in the rapidly evolving DevOps ecosystem.

---

👤 Author

Join Our Telegram Community || Follow me on GitHub for more DevOps content!