Day 39: Deploying Stateful Applications with StatefulSets (MongoDB)

Introduction

Today, I focused on deploying MongoDB using StatefulSets in Kubernetes. This is important for managing databases because they need stable identities and persistent storage.

Key Concepts

What are StatefulSets?

• StatefulSets are a Kubernetes feature for managing stateful applications. They differ from Deployments, which are designed for stateless applications.

Why StatefulSets Matter

• Unique Identifiers: Each pod has a unique name, making it easier to track and manage.
• Persistent Storage: Pods are linked to PersistentVolumeClaims (PVCs), ensuring data remains intact even if the pods restart.
• Ordered Management: StatefulSets ensure that pods are deployed and scaled in a specific order, which is crucial for databases.

Project Steps

1. Folder Structure

To keep things organized, I created the following folder structure for my Kubernetes configurations:

mongodb-k8s/
├── statefulset/
│   ├── mongodb-statefulset.yaml
│   └── mongodb-service.yaml
└── backups/
    └── backup-script.sh

2. Prepare YAML Configuration

• StatefulSet Definition: I created a YAML file to set up the StatefulSet for MongoDB. This included:

  • Number of replicas (instances).
  • The MongoDB container image to use.
  • Ports for communication.

• PersistentVolumeClaims: I defined PVCs to allocate storage for each MongoDB instance, ensuring data persistence.

3. Deploy MongoDB

• Apply Configuration: I used the command kubectl apply -f statefulset/mongodb-service.yaml and kubectl apply -f statefulset/mongodb-statefulset.yaml to create the StatefulSet and the service.
• Verification: After deploying, I checked the status of the pods and PVCs:
  • kubectl get pods to see if they were running.
  • kubectl get pvc to confirm the storage was allocated.

4. Implement Backup and Restore

• Automated Backups: I set up a CronJob in Kubernetes to automatically back up MongoDB every day. This job uses mongodump to create backups and stores them in a location like AWS S3.

• Restoration Process: I documented how to restore data from the backups, ensuring I could verify that the backups worked by restoring to a new MongoDB instance.

Challenges and Solutions

Challenge: Backup to Remote Storage

• Solution: I created a script (backup-script.sh) that uses mongodump to export MongoDB data and the AWS CLI to upload the backups to S3. This script runs automatically with the CronJob.

Challenge: Managing Replica Set Configurations

• Solution: I used Kubernetes init containers to automatically set up the MongoDB replica set when new pods are created. This way, it requires no manual steps.

Takeaways

Deploying MongoDB with StatefulSets helped me understand how to manage databases in Kubernetes. The automated backup and restore processes showed me the importance of keeping data safe. This experience is valuable for my growth as a DevOps engineer, combining practical deployment skills with essential data management practices.