Automating Docker Container Restarts Based on CPU Usage: A Guide

Effectively managing Docker containers involves monitoring resource consumption and automating responses to maintain optimal performance. In this guide, we'll delve into the process of automating Docker container restarts based on CPU usage, using tools like Prometheus and Grafana. By setting up health checks, monitoring CPU metrics, and triggering alerts, you can ensure that your containers operate efficiently and reliably.

let's go through creating a Docker Compose file for a sample WordPress application and setting up Prometheus from scratch.

Step 1: Create a Docker Compose File for WordPress

Create a file named docker-compose.yml and add the following content:

version: '3'
services:
  wordpress:
    image: wordpress
    restart: always
    ports:
      - "8080:80"
    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_USER: exampleuser
      WORDPRESS_DB_PASSWORD: examplepass
      WORDPRESS_DB_NAME: exampledb
  db:
    image: mysql:5.7
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: rootpass
      MYSQL_DATABASE: exampledb
      MYSQL_USER: exampleuser
      MYSQL_PASSWORD: examplepass

Step 2: Start WordPress Container:

Run the following command in the same directory as the docker-compose.yml file:

docker-compose up -d

This will start the WordPress and MySQL containers.

Step 3: Setting Up Prometheus:

• Download and Extract Prometheus:

wget https://github.com/prometheus/prometheus/releases/download/v2.30.3/prometheus-2.30.3.linux-amd64.tar.gz
tar -xvzf prometheus-2.30.3.linux-amd64.tar.gz

• Configure Prometheus

Create a file named prometheus.yml in the Prometheus directory and add the following configuration:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'docker'
    static_configs:
      - targets: ['localhost:9323']

• Run Prometheus

Navigate to the Prometheus directory and run the following command:

./prometheus --config.file=prometheus.yml

This will start Prometheus, which will scrape targets on localhost:9323.

Step 4: Visualizing CPU Usage with Grafana:

• Install and Set Up Grafana:

Follow the official Grafana installation guide for your operating system.

• Add Prometheus as Data Source:

• Open Grafana in your browser (typically at http://localhost:3000).

• Log in with the default credentials (admin/admin).

• Add Prometheus as a data source:

• Go to Configuration > Data Sources.

• Click on "Add data source".

• Select "Prometheus".

• Set the URL to http://localhost:9090 (default Prometheus address).

• Create a Dashboard

• Create a new dashboard:

• Click on the "+" icon on the left panel.

• Choose "Dashboard".

• Click on "Add new panel".

• Choose a visualization type (e.g., "Graph").

In the "Metrics" tab, set the query to monitor CPU usage of the WordPress container:

rate(container_cpu_usage_seconds_total{container_name="wordpress"}[1m])

Adjust visualization settings and save the panel.

Step 5: Creating CPU Usage Alerts:

• Configuring Alerting Rules:

Open prometheus.yml and add alerting configuration:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'docker'
    static_configs:
      - targets: ['localhost:9323']

alerting:
  alertmanagers:
  - static_configs:
    - targets:
      - 'localhost:9093'

• Start Prometheus with Alerting:

Navigate to the Prometheus directory and run the following command:

./prometheus --config.file=prometheus.yml --alertmanager.url=http://localhost:9093

• Define Alert Rules:

Create a file named alert.rules in the Prometheus directory and add the following rule:

groups:
- name: cpu_alerts
  rules:
  - alert: HighCpuUsage
    expr: rate(container_cpu_usage_seconds_total{container_name="wordpress"}[1m]) > 0.9
    for: 15m
    labels:
      severity: warning
    annotations:
      summary: "High CPU usage detected in WordPress container"
      description: "CPU usage is above 90% for more than 15 minutes."

• Run Prometheus with Alert Rules:

Run Prometheus with the new alerting rules:

./prometheus --config.file=prometheus.yml --alertmanager.url=http://localhost:9093 --web.enable-admin-api --web.enable-lifecycle

Conclusion:

Following these steps, you've set up a Docker Compose file for a WordPress application and configured Prometheus with alerting rules to monitor CPU usage. Additionally, you've integrated Grafana to visualize the CPU metrics. This comprehensive guide demonstrates how to automate container restarts based on CPU usage using Docker, Prometheus, and Grafana.