DEV Community: devopsbg

DevOps Automation with Python: Intelligent System Monitoring with Auto Recovery

devopsbg — Wed, 10 Sep 2025 18:25:39 +0000

DevOps Automation with Python: Intelligent System Monitoring with Auto Recovery

Automation is at the core of modern DevOps culture. While powerful tools like Kubernetes, Docker, and CI/CD platforms exist, scripts remain a critical foundation for efficient infrastructure management.

In this article, we will build a Python system monitoring script that not only tracks system resources but also performs automatic recovery when issues are detected.

Why Use Scripts in DevOps?

Scripts bring several key advantages to DevOps practices:

Fast Response – A script can detect and fix issues in seconds.
Consistency – Tasks are always executed the same way, reducing human error.
Scalability – One script can manage hundreds of servers.
Documentation – A well-written script serves as executable documentation.

Full Python Monitoring Script

#!/usr/bin/env python3
"""
Intelligent System Monitoring and Auto Recovery
Author: DevOps Bulgaria Team
Version: 2.0
"""

import psutil
import subprocess
import logging
import smtplib
import json
import time
from datetime import datetime
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from typing import Dict, List, Optional
import argparse
import sys
import os

class SystemMonitor:
    def __init__(self, config_file: str):
        self.load_config(config_file)
        self.setup_logging()
        self.cooldowns = {}

    def load_config(self, config_file: str):
        try:
            with open(config_file, "r") as f:
                self.config = json.load(f)
        except Exception as e:
            print(f"Error loading config: {e}")
            sys.exit(1)

    def setup_logging(self):
        log_file = "/var/log/system_monitor.log"
        logging.basicConfig(
            filename=log_file,
            level=logging.INFO,
            format="%(asctime)s - %(levelname)s - %(message)s"
        )
        logging.getLogger().addHandler(logging.StreamHandler(sys.stdout))

    def get_metrics(self) -> Dict[str, float]:
        return {
            "cpu_percent": psutil.cpu_percent(interval=1),
            "memory_percent": psutil.virtual_memory().percent,
            "disk_percent": psutil.disk_usage("/").percent,
            "load_average": psutil.getloadavg()[0]
        }

    def check_services(self) -> Dict[str, bool]:
        results = {}
        for service in self.config.get("services", []):
            try:
                result = subprocess.run(
                    ["systemctl", "is-active", "--quiet", service],
                    check=False
                )
                results[service] = (result.returncode == 0)
            except Exception:
                results[service] = False
        return results

    def attempt_recovery(self, service: str) -> bool:
        logging.warning(f"Attempting to restart service: {service}")
        try:
            subprocess.run(
                ["systemctl", "restart", service],
                check=True
            )
            time.sleep(5)
            return subprocess.run(
                ["systemctl", "is-active", "--quiet", service],
                check=False
            ).returncode == 0
        except Exception as e:
            logging.error(f"Recovery failed for {service}: {e}")
            return False

    def send_alert(self, subject: str, body: str):
        email_conf = self.config.get("email", {})
        if not email_conf.get("enabled", False):
            return

        now = time.time()
        last_alert = self.cooldowns.get(subject, 0)
        if now - last_alert < self.config["monitoring"]["cooldown_period"]:
            return
        self.cooldowns[subject] = now

        try:
            msg = MIMEMultipart()
            msg["From"] = email_conf["username"]
            msg["To"] = ", ".join(email_conf["to_addresses"])
            msg["Subject"] = subject
            msg.attach(MIMEText(body, "plain"))

            with smtplib.SMTP(email_conf["smtp_server"], email_conf["smtp_port"]) as server:
                server.starttls()
                server.login(email_conf["username"], email_conf["password"])
                server.send_message(msg)

            logging.info(f"Alert sent: {subject}")
        except Exception as e:
            logging.error(f"Failed to send alert: {e}")

    def monitor_once(self):
        metrics = self.get_metrics()
        services = self.check_services()

        logging.info("System Metrics: " + str(metrics))

        for key, value in metrics.items():
            if value > self.config["thresholds"][key]:
                msg = f"{key} at {value}% exceeds threshold {self.config['thresholds'][key]}%"
                logging.warning(msg)
                self.send_alert("System Threshold Exceeded", msg)

        for service, is_active in services.items():
            if not is_active:
                logging.error(f"Service down: {service}")
                for _ in range(self.config["monitoring"]["recovery_attempts"]):
                    if self.attempt_recovery(service):
                        logging.info(f"Service {service} recovered successfully")
                        break
                else:
                    self.send_alert("Service Down", f"Failed to recover {service}")

    def run(self):
        interval = self.config["monitoring"]["interval"]
        while True:
            self.monitor_once()
            time.sleep(interval)

def main():
    parser = argparse.ArgumentParser(description="System Monitor")
    parser.add_argument("--config", default="monitor_config.json", help="Path to config file")
    parser.add_argument("--once", action="store_true", help="Run once and exit")
    parser.add_argument("--verbose", action="store_true", help="Enable debug logging")
    args = parser.parse_args()

    monitor = SystemMonitor(args.config)
    if args.once:
        monitor.monitor_once()
    else:
        monitor.run()

if __name__ == "__main__":
    main()

Configuration File (monitor_config.json)

{
    "thresholds": {
        "cpu_percent": 85.0,
        "memory_percent": 90.0,
        "disk_percent": 85.0,
        "load_average": 8.0
    },
    "services": [
        "nginx",
        "mysql",
        "docker",
        "redis-server"
    ],
    "email": {
        "enabled": true,
        "smtp_server": "smtp.gmail.com",
        "smtp_port": 587,
        "username": "your-email@gmail.com",
        "password": "your-app-password",
        "to_addresses": ["admin@yourcompany.com"]
    },
    "monitoring": {
        "interval": 60,
        "recovery_attempts": 3,
        "cooldown_period": 300
    }
}

Systemd Service Example

[Unit]
Description=Intelligent System Monitor
After=network.target

[Service]
Type=simple
User=root
ExecStart=/usr/bin/python3 /opt/system-monitor/system_monitor.py
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Conclusion

This script provides:

Real-time monitoring of CPU, memory, disk, and load average
Service supervision with automatic restart
Configurable thresholds and alerts through JSON
Email notifications with cooldown to avoid spam
Systemd integration for reliable background execution

Automation scripts like this are a cornerstone of DevOps practices. They ensure system resilience, reduce downtime, and free engineers from repetitive tasks.

Pro tip: Always test in a staging environment before production deployment.

Originally written in Bulgarian and translated to English for the dev.to community.

Read the original Bulgarian version: DevOps автоматизация със скриптове

Home DevOps Solution: Proxmox and Docker for Enterprise Home Lab

devopsbg — Fri, 05 Sep 2025 13:35:50 +0000

Meta Description: Learn how to build a professional Home Lab with Proxmox and Docker. Comprehensive guide with practical examples, security best practices, and real-world projects for DevOps engineers. Includes CI/CD pipeline, monitoring, and production-ready configurations.

Tags: Proxmox, Docker, Home Lab, DevOps, virtualization, containerization, CI/CD, Jenkins, Gitea, monitoring, Grafana, Prometheus, infrastructure as code, self-hosted, system administration, IT infrastructure, professional development

Creating your own Home Lab has become essential for every DevOps engineer. The combination of Proxmox and Docker offers a professional solution with enterprise capabilities. Today we'll explore how to build a system that rivals cloud platforms.

Why the Proxmox + Docker Combination is Revolutionary

Proxmox Virtual Environment (PVE) is an enterprise-grade open-source hypervisor. Docker provides containerization with minimal resources. Together they create a powerful ecosystem for home experiments and professional development.

This architecture allows you to simulate complex production environments. You can test new technologies without risk to your main work. At the same time, you develop practical skills with tools used in large companies.

Advantages Over Competitive Solutions

VMware vSphere costs thousands of dollars annually. Proxmox is free with optional premium support. Docker Desktop has licensing restrictions for business use. The Linux-based approach eliminates these problems.

Proxmox offers advanced features like live migration, high availability, and backup replication. Docker provides fast deployment and efficient resource utilization. The combination merges virtualization stability with container flexibility.

Hardware Requirements and Planning

Minimum and Recommended Specifications

Minimum for startup:

CPU: Intel i5 or AMD Ryzen 5 (4+ cores)
RAM: 16GB DDR4 (8GB for Proxmox, 8GB for VMs)
Storage: 500GB SSD for system + 1TB HDD for data
Network: Gigabit Ethernet

Recommended configuration:

CPU: Intel i7/i9 or AMD Ryzen 7/9 (8+ cores)
RAM: 32-64GB DDR4/DDR5
Storage: 1TB NVMe SSD + 4TB HDD/SSD array
Network: 10Gb Ethernet or dual 1Gb bonds

Hardware Selection for Optimal Performance

CPU with high single-thread performance is critical for Proxmox. AMD Ryzen processors offer excellent price/performance ratio. Intel processors have better virtualization support in some cases.

RAM is the most important resource for virtualization. Plan minimum 8GB for Proxmox host + 4-8GB for each virtual machine. ECC memory isn't mandatory but increases stability for 24/7 operation.

Storage architecture determines entire system performance. NVMe SSD for VM disks provides excellent responsiveness. Mechanical disks are suitable for backup storage and less critical data.

Step-by-Step Proxmox Installation

Installation Environment Preparation

Download the latest Proxmox VE ISO from the official website. Create bootable USB with tools like Rufus (Windows) or dd (Linux). Ensure UEFI/BIOS settings allow USB boot.

Enable CPU virtualization features (Intel VT-x/AMD-V). Activate IOMMU if planning GPU passthrough. Configure boot order to start first from USB device.

Installation Process and Initial Settings

Boot from USB and follow the graphical installer. Select entire disk for Proxmox or create custom partitioning scheme. Configure network parameters - IP address, gateway, DNS servers.

Create root password and enter email address for notifications. After installation the system restarts automatically. Remove USB device before boot process.

Post-Installation Configuration

Access web interface at https://[IP-address]:8006. Log in with root user and set password. First update system packages for security patches.

Configure repositories for updates. Enterprise repo requires subscription. No-subscription repo is free for testing. Add community repositories for additional packages.

# System update
apt update && apt upgrade -y

# Install useful tools
apt install -y curl wget git vim htop

Creating Virtual Machines for Docker

Template-Based Approach for Efficiency

Create base Ubuntu Server template for all Docker machines. This accelerates deployment process and guarantees consistency. Template contains pre-configured SSH, network settings, and basic packages.

Template cloning is significantly faster than full installation. You can create specialized templates for different roles - web servers, databases, monitoring systems.

Optimal VM Configuration for Containers

Configure VM with machine type "q35" for best performance. Use SCSI controller with VirtIO drivers. Activate QEMU guest agent for improved integration.

Recommended settings:

CPU: 2-4 vCPU with CPU type "host"
RAM: 4-8GB depending on workload
Disk: 40GB thin provisioned with VirtIO interface
Network: VirtIO network adapter with dedicated bridge

Network Topology and Security

Create separate Linux bridge for Docker traffic. This allows microsegmentation and traffic control. Configure VLAN tagging for isolation between different environments.

Set up firewall rules at Proxmox level. Block unnecessary ports to external network. Allow only required communication between VM and host systems.

Docker Installation and Configuration

Automatic Installation Script Approach

Docker provides convenience script for quick installation. The script automatically configures repositories and installs latest stable version. Suitable for development environments.

# Download and execute installation script
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

# Add user to docker group
sudo usermod -aG docker $USER

# Script automatically installs Docker Compose as plugin
# Check installation
docker compose version

Manual Installation for Production Environments

For critical systems I recommend manual installation. This approach offers greater control over the process. You can select specific version and configure custom settings.

# Update package index
sudo apt update

# Install prerequisites
sudo apt install -y apt-transport-https ca-certificates curl gnupg lsb-release

# Add Docker GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

# Add Docker repository
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker Engine
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

Docker Daemon Optimization

Configure Docker daemon for optimal performance in VM environment. Set up logging driver, storage driver, and resource limits. These optimizations improve performance and stability.

{
  "log-driver": "journald",
  "storage-driver": "overlay2",
  "default-ulimits": {
    "nofile": {
      "Name": "nofile",
      "Hard": 64000,
      "Soft": 64000
    }
  },
  "live-restore": true,
  "userland-proxy": false
}

Docker Compose and Portainer for Management

Docker Compose V2 installs automatically as plugin with installation script. The new version uses docker compose command without dash. This improves integration with Docker CLI and offers better performance.

Portainer - Graphical Interface for Docker

Portainer transforms Docker command line into intuitive web interface. Allows easy management of containers, networks, volumes, and images. Suitable for both beginners and experienced users.

version: '3.8'
services:
  portainer:
    image: portainer/portainer-ce:latest
    container_name: portainer
    restart: unless-stopped
    ports:
      - "9000:9000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    security_opt:
      - no-new-privileges:true

volumes:
  portainer_data:

Real Projects for Your Home Lab

Complete DevOps Stack

version: '3.8'
services:
  # Reverse Proxy
  traefik:
    image: traefik:v3.0
    container_name: traefik
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik.yml:/etc/traefik/traefik.yml
      - ./dynamic:/etc/traefik/dynamic
      - traefik-ssl:/ssl
    networks:
      - proxy

  # Code Repository
  gitea:
    image: gitea/gitea:latest
    container_name: gitea
    restart: unless-stopped
    environment:
      - USER_UID=1000
      - USER_GID=1000
      - GITEA__database__DB_TYPE=postgres
      - GITEA__database__HOST=gitea-db:5432
      - GITEA__database__NAME=gitea
      - GITEA__database__USER=gitea
      - GITEA__database__PASSWD=gitea_password
    volumes:
      - gitea-data:/data
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "3001:3000"
      - "222:22"
    depends_on:
      - gitea-db
    networks:
      - gitea
      - proxy
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.gitea.rule=Host(`git.homelab.local`)"

  gitea-db:
    image: postgres:15
    container_name: gitea-db
    restart: unless-stopped
    environment:
      - POSTGRES_USER=gitea
      - POSTGRES_PASSWORD=gitea_password
      - POSTGRES_DB=gitea
    volumes:
      - gitea-db:/var/lib/postgresql/data
    networks:
      - gitea

  # CI/CD Pipeline
  jenkins:
    image: jenkins/jenkins:lts
    container_name: jenkins
    restart: unless-stopped
    privileged: true
    user: root
    ports:
      - "8081:8080"
      - "50000:50000"
    volumes:
      - jenkins-data:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/local/bin/docker:/usr/local/bin/docker
    networks:
      - proxy
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.jenkins.rule=Host(`jenkins.homelab.local`)"

networks:
  proxy:
    external: true
  gitea:
    internal: true

volumes:
  traefik-ssl:
  gitea-data:
  gitea-db:
  jenkins-data:

Media Server Stack

version: '3.8'
services:
  # Media Server
  plex:
    image: lscr.io/linuxserver/plex:latest
    container_name: plex
    restart: unless-stopped
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - VERSION=docker
    volumes:
      - plex-config:/config
      - /media/movies:/movies
      - /media/tvshows:/tv
    ports:
      - "32400:32400"
    networks:
      - media

  # Download Manager
  qbittorrent:
    image: lscr.io/linuxserver/qbittorrent:latest
    container_name: qbittorrent
    restart: unless-stopped
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - WEBUI_PORT=8082
    volumes:
      - qbit-config:/config
      - /media/downloads:/downloads
    ports:
      - "8082:8082"
    networks:
      - media

  # Automation
  sonarr:
    image: lscr.io/linuxserver/sonarr:latest
    container_name: sonarr
    restart: unless-stopped
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - sonarr-config:/config
      - /media/tvshows:/tv
      - /media/downloads:/downloads
    ports:
      - "8989:8989"
    networks:
      - media

networks:
  media:
    external: true

volumes:
  plex-config:
  qbit-config:
  sonarr-config:

Development Environment

version: '3.8'
services:
  # Database
  postgres:
    image: postgres:15
    container_name: dev-postgres
    restart: unless-stopped
    environment:
      - POSTGRES_USER=developer
      - POSTGRES_PASSWORD=dev_password
      - POSTGRES_DB=development
    volumes:
      - postgres-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    networks:
      - development

  # Redis Cache
  redis:
    image: redis:7-alpine
    container_name: dev-redis
    restart: unless-stopped
    command: redis-server --requirepass redis_password
    ports:
      - "6379:6379"
    networks:
      - development

  # Node.js Environment
  nodejs:
    image: node:18-alpine
    container_name: dev-nodejs
    restart: unless-stopped
    working_dir: /app
    volumes:
      - ./projects/nodejs:/app
      - node-modules:/app/node_modules
    ports:
      - "3000:3000"
    networks:
      - development
    command: sh -c "npm install && npm run dev"

  # Python Environment
  python:
    image: python:3.11
    container_name: dev-python
    restart: unless-stopped
    working_dir: /app
    volumes:
      - ./projects/python:/app
      - python-packages:/usr/local/lib/python3.11/site-packages
    ports:
      - "8000:8000"
    networks:
      - development
    command: sh -c "pip install -r requirements.txt && python manage.py runserver 0.0.0.0:8000"

networks:
  development:
    external: true

volumes:
  postgres-data:
  node-modules:
  python-packages:

Docker Compose Best Practices

Structure projects with separate directories for each service. Use .env files for sensitive data. Create dedicated networks for different applications.

Version Docker Compose files in Git repository. This allows tracking changes and easy rollback during problems. Document each service with comments in YAML files.

Monitoring and Logs

Prometheus and Grafana Stack

Prometheus is industry standard for metrics collection. Grafana offers beautiful dashboards for visualization. The combination provides comprehensive monitoring of the entire system.

version: '3.8'
services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    restart: unless-stopped
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--web.console.libraries=/etc/prometheus/console_libraries'
      - '--web.console.templates=/etc/prometheus/consoles'

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    restart: unless-stopped
    ports:
      - "3000:3000"
    volumes:
      - grafana_data:/var/lib/grafana
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin

volumes:
  prometheus_data:
  grafana_data:

ELK Stack for Centralized Logging

Elasticsearch, Logstash, and Kibana form powerful platform for log analysis. Elasticsearch stores and indexes logs. Logstash processes and transforms data. Kibana provides visualization and search capabilities.

This setup allows you to search through logs of all containers from one interface. You can create custom dashboards for different metrics. Advanced alerting capabilities notify you of problems in real-time.

Backup Strategies

Proxmox Integrated Backup

Proxmox includes sophisticated backup system. Supports full and incremental backups with different retention policies. Can archive to local storage, NFS shares, or cloud services.

Set up automatic backup jobs for all critical VMs. Schedule them for off-peak hours for minimal performance impact. Test restore procedures regularly to guarantee data integrity.

Docker Volume Backup Strategies

Docker volumes contain persistent data of containers. Backup strategy depends on data type and RTO/RPO requirements. Database volumes require application-consistent backups.

# Backup Docker volume
docker run --rm -v volume_name:/data -v $(pwd):/backup ubuntu tar czf /backup/volume_backup.tar.gz -C /data .

# Restore Docker volume
docker run --rm -v volume_name:/data -v $(pwd):/backup ubuntu tar xzf /backup/volume_backup.tar.gz -C /data

Security Hardening

System Level Security

Configure fail2ban for protection against brute force attacks. Set up UFW firewall with restrictive rules. Activate automatic security updates for critical vulnerabilities.

Create separate users for different services. Avoid root access when possible. Configure SSH key authentication and disable password login.

Container Security

Use official Docker images from trusted publishers. Scan images for known vulnerabilities with tools like Trivy. Configure container runtime security with AppArmor or SELinux profiles.

Limit container capabilities to necessary minimum. Use read-only filesystems where possible. Configure resource limits to prevent DoS attacks.

Practical Examples and Use Cases

Development Environment Setup

Create isolated development environments for different projects. Each project can have its own VM with specific PHP, Node.js, or Python versions. This eliminates dependency conflicts between projects.

Docker containers allow rapid switching between different tool versions. You can test application compatibility with different database versions. Development and production environments become identical.

CI/CD Pipeline Integration

Integrate Home Lab into CI/CD processes. GitLab Runner or Jenkins can use Docker containers for build and test jobs. This significantly accelerates development cycle.

Create staging environments that mirror production setup. Automated testing can validate deployments before production release. Code quality gates guarantee maintenance of high standards.

Microservices Testing

Simulate complex microservices architectures with Docker Compose. Test service discovery, load balancing, and failure scenarios. Network policies can simulate latency and packet loss.

Service mesh technologies like Istio can be tested in controlled environment. Observability tools provide detailed insights for service interactions. Chaos engineering experiments improve system resilience.

Performance Tuning and Optimizations

VM Resource Allocation

Proper resource allocation is critical for performance. Over-provisioning leads to resource contention. Under-provisioning limits application capacity.

Monitor CPU, memory, and disk utilization regularly. Adjust resource limits based on real usage patterns. Use memory ballooning for dynamic resource adjustment.

Network Optimization

Configure multiple network bridges for traffic separation. Use VXLAN or GRE tunnels for overlay networking. QoS policies can prioritize critical traffic.

Network bonding improves throughput and provides redundancy. LACP configuration requires managed switch support. Load balancing algorithms should match traffic patterns.

Bonus Tips for Professionals

Advanced Proxmox Techniques

GPU Passthrough for ML/AI Projects
Configure GPU passthrough for TensorFlow or PyTorch containers. This provides native GPU performance for machine learning workloads. Nvidia Tesla or consumer GPUs work excellently for this setup.

ZFS Storage Optimization
Use ZFS for enterprise-grade data protection. ARC cache significantly improves read performance. L2ARC on SSD can accelerate workloads with random access patterns.

Cluster Setup for High Availability
Configure 3-node Proxmox cluster for production-like environment. Shared storage with Ceph provides distributed storage. Live migration allows zero-downtime maintenance.

Docker Production Secrets

Multi-stage Builds for Smaller Images

# Build stage
FROM node:18 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

# Production stage
FROM node:18-alpine
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY . .
EXPOSE 3000
CMD ["node", "server.js"]

Health Checks for Robust Deployments

healthcheck:
  test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
  interval: 30s
  timeout: 10s
  retries: 3
  start_period: 60s

Resource Limits for Stability

deploy:
  resources:
    limits:
      memory: 1G
      cpus: '0.5'
    reservations:
      memory: 512M
      cpus: '0.25'

Network Architecture Patterns

Ingress with SSL Termination
Traefik or Nginx automatically manage SSL certificates with Let's Encrypt. Wildcard certificates simplify multi-service setups. HTTP to HTTPS redirect improves security posture.

Service Discovery and Load Balancing
Consul or etcd provide service registry. HAProxy or Nginx can load balance between container instances. Health check integration guarantees traffic to healthy services.

Network Segmentation for Security
Create separate networks for frontend, backend, and database tiers. Bridge networks isolate traffic between different application stacks. Overlay networks allow multi-host communication.

Infrastructure as Code Approaches

Terraform for Proxmox Automation

resource "proxmox_vm_qemu" "docker-node" {
  count       = 3
  name        = "docker-node-${count.index + 1}"
  target_node = "proxmox"

  memory      = 4096
  cores       = 2

  disk {
    size     = "40G"
    type     = "virtio"
    storage  = "local-lvm"
  }

  network {
    model  = "virtio"
    bridge = "vmbr0"
  }
}

Ansible for Configuration Management

- name: Install Docker on all nodes
  hosts: docker_nodes
  become: true
  tasks:
    - name: Install Docker
      shell: curl -fsSL https://get.docker.com | sh

    - name: Add user to docker group
      user:
        name: "{{ ansible_user }}"
        groups: docker
        append: yes

    - name: Start Docker service
      systemd:
        name: docker
        enabled: true
        state: started

Performance Debugging Tips

Container Resource Monitoring

# Real-time resource usage
docker stats

# Detailed container inspection
docker inspect <container_id>

# Process tree inside container
docker exec <container_id> ps aux

Network Troubleshooting

# Check container networking
docker network ls
docker network inspect bridge

# Test connectivity between containers
docker exec -it container1 ping container2
docker exec -it container1 nslookup container2

Storage Performance Analysis

# I/O statistics
iostat -x 1

# Disk usage by container
docker system df
docker image ls --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}"

Conclusion

The combination of Proxmox and Docker creates professional-grade Home Lab environment. It offers enterprise functionality without enterprise cost. Hardware investment pays back through practical learning experience.

This setup prepares you for real-world DevOps challenges. Practical experience with these tools is invaluable for career development. Home Lab environment allows experimentation without production risk.

Successfully building this architecture demonstrates technical skills and commitment to continuous learning. It serves as proof of concept for new technologies and architectural patterns.

Start with basic configuration and gradually add complexity. Document every step for future reference. Share your experience with DevOps community for mutual benefit.

Your Home Lab is investment in professional future. It provides sandbox for innovation and platform for skill development. Possibilities are limitless - limited only by curiosity and creativity.

Originally written in Bulgarian and translated to English for the dev.to community.

Read the original Bulgarian version: Домашно DevOps решение: Proxmox и Docker за корпоративен Home Lab

Backing Up and Restoring WSL: How to Safeguard Your Linux Environment

devopsbg — Fri, 05 Sep 2025 04:32:45 +0000

WSL Backup and Restore: How to Save Your Linux Environment

Working with Windows Subsystem for Linux (WSL) is like building a digital garden. You invest weeks or months setting up your environment. You install packages, configure tools, create scripts. And suddenly – a system error, corrupted hard drive, or simply the need to reinstall Windows.

All your efforts can disappear in seconds.

That's exactly why proper WSL environment backup is critically important. In this article, we'll explore how to protect your Linux environment and how to restore it when needed.

Why WSL Backup is Important

WSL is not an ordinary virtual machine. Your Linux environment is stored in a specific format in the Windows file system. During system problems, this data can be lost forever.

Imagine the following scenario: you're working on an important project, you've configured Docker containers, set up a development environment with Node.js, Python, and many other tools. Suddenly Windows gets corrupted and you need to reinstall the system.

Without backup, you lose:

All installed packages and configurations
Project files in the Linux environment
Personalized settings
SSH keys and credentials
Hours of setup work

WSL Backup Basics

WSL provides built-in commands for exporting and importing distributions. The process is simple, but requires planning and regularity.

What exactly are we archiving?

When we talk about archiving, we're taking a "snapshot" of the WSL distribution. This includes:

File system: All your files and directories
Installed programs: All programs and packages
User settings: Your configuration files
System settings: All WSL environment settings

With one archive, you preserve the entire distribution. This is extremely convenient. You don't need to archive each file separately.

Preparation: What We Need to Know

Before we start, here are a few important things. To archive, you need to stop the distribution. This ensures the completeness of the archive. There should be no running processes. Otherwise, the archive might be corrupted or incomplete.

The first step is to check. Verify if the WSL distribution is running. This is done with the command:

wsl --list --verbose

You'll see a list of distributions and their status. If the status is Running, you need to stop it. This is done with the command:

wsl --terminate <distribution_name>

For example, if the name is Ubuntu, the command would be:

wsl --terminate Ubuntu

Then check again. Make sure the status is Stopped.

Archiving: Easy Steps

Now let's move to the essence. To archive, we'll use the wsl --export command. This command exports the distribution to a single .tar file. The TAR format is standard and very convenient for archiving.

Here's the basic syntax:

wsl --export <distribution_name> <path_to_file>

For example, let's archive our Ubuntu distribution. We want to save it on the E: drive in a WSL_Backups folder. We want to add the date to the filename. This is good practice. This way we'll know when the archive was made. Here's the command I use:

wsl --export Ubuntu "E:\WSL_Backups\UbuntuBackup$(Get-Date -Format yyyy-MM-dd).tar"

Let's break it down:

wsl --export Ubuntu: Export the distribution named Ubuntu
"E:\WSL_Backups\UbuntuBackup...": Specify the path and filename
$(Get-Date -Format yyyy-MM-dd): This code generates the current date, making filenames unique

You can execute this command from PowerShell or Command Prompt. Make sure you have write permissions. Wait for the process to complete. Depending on the distribution size, this may take time.

When the process is finished, you'll have one file. This file is the complete archive of your environment. You can copy and store it on an external drive or in the cloud, wherever is most convenient for you.

Restoration: How to Get Back in the Game

Now let's look at the reverse process. How do we restore our WSL environment? For this, we'll use the wsl --import command. This command imports a .tar file and creates a new distribution or replaces an existing one.

Here's the basic syntax:

wsl --import <new_distribution_name> <installation_path> <path_to_file>

Let's break down this command too:

wsl --import <new_distribution_name>: Set a new name for your distribution, e.g., Ubuntu_Restored
<installation_path>: The path where you want to install WSL, e.g., C:\WSL\Ubuntu_Restored
<path_to_file>: Full path to your .tar file

For example, let's restore the archive we created. We want to install it on C: drive and name the distribution Ubuntu_Backup:

wsl --import Ubuntu_Backup C:\WSL\Ubuntu_Backup "E:\WSL_Backups\UbuntuBackup2025-09-02.tar"

After executing this command, a new WSL distribution will be created. It will contain everything from your archive and be absolutely identical. You'll be able to start wsl -d Ubuntu_Backup and work as if nothing happened.

Important User Tip

When restoring, the new distribution starts as root. This can be a problem. You need to set up a default user.

To do this, you need to modify the distribution's configuration file. The file is called wsl.conf and is located in the /etc directory.

First, start the distribution as root:

wsl -d Ubuntu_Backup

Then open the wsl.conf file:

sudo nano /etc/wsl.conf

Add the following lines to the file:

[user]
default = your_username

Replace your_username with your username. Save the file and exit the editor. Then close and restart:

wsl --terminate Ubuntu_Backup
wsl -d Ubuntu_Backup

Now you'll automatically log in as your user. All your settings will be there. That's it! 🥳

Automation and Other Tips

Manual archiving is good, but why not automate it? You can create a script that you run regularly or schedule it using Windows Task Scheduler. This is a great way to ensure you always have a current archive without having to think about it.

Additional Tips:

Store archives elsewhere: Don't keep archives on the same drive. Use an external drive or cloud. This protects you from drive failures.
Archive size: The .tar file size can be large. Clean unnecessary files before archiving to save space.
Short and clear distribution name: Choose a short name to make it easier to work with.

Conclusion

WSL archiving is critical. It's not just an option – it's a necessity. With these simple commands, you can protect your work, save yourself a lot of headaches, and save time.

I hope this article has been helpful to you. Share it with your colleagues and friends. Let's take care of our work and protect it!

If you have questions, ask in the comments. Let's build a community where we help each other and share knowledge. Thank you! 🙏

Originally written in Bulgarian and translated to English for the dev.to community.

Read the original Bulgarian version: Бекъп и възстановяване на WSL: Как да запазите своята Linux среда