DEV Community

Moisi Trungu
Moisi Trungu

Posted on • Originally published at flixzone.icu

Automated ViciDial Backups — Database, Recordings & Config

#devops #linux #sip #voip

Automated ViciDial Backups — Database, Recordings & Config

Master complete backup automation for ViciDial production systems, including MariaDB databases, call recordings, configuration files, and verification workflows that run unattended on schedule.

Introduction

ViciDial is a mission-critical dialing platform. Losing your database, recordings, or configuration files can bring revenue-generating operations to a halt in minutes. This tutorial walks you through building a production-grade automated backup system that protects your entire ViciDial infrastructure—not just the database, but also recordings stored in /var/spool/asterisk/monitor/ and configuration files scattered across /etc/asterisk/.

You'll learn to create a robust vicidial backup script that runs daily, compresses efficiently, maintains version history, validates integrity, and alerts you to failures before they become disasters.

Prerequisites

  • ViciDial 2.12+ (tested on 2.14.1, but compatible with 2.12 and above)
  • Asterisk 11+ running on CentOS 7/8, Rocky Linux, or Ubuntu 18.04+
  • Root or sudoer access on the ViciDial server
  • MariaDB 5.5+ or MySQL 5.7+ with direct shell access
  • Minimum 200GB free disk space for backup retention (calculate: 7 days × daily full backup size)
  • Cron daemon running (verify: systemctl status crond or systemctl status cron)
  • Basic bash scripting knowledge (we'll provide complete scripts)
  • Confirmation that /etc/asterisk/sip-vicidial.conf exists and is readable

Understanding ViciDial's Critical Data

Before backing up, know what you're protecting:

Component Location Size Criticality Frequency
Asterisk DB /var/lib/mysql/asterisk/ 500MB–5GB Critical Every backup
Call Recordings /var/spool/asterisk/monitor/ 1–50GB+ Critical Every backup
SIP Config /etc/asterisk/sip-vicidial.conf 50KB–500KB Critical Every backup
Extensions /etc/asterisk/extensions-vicidial.conf 100KB–2MB Critical Every backup
Agent Configs /usr/share/astguiclient/config.php 5–50KB Important Every backup
Logs /var/log/asterisk/ 100MB–1GB Optional Weekly

The asterisk database contains all agent accounts (vicidial_users), campaigns (vicidial_campaigns), call logs (vicidial_log), and dialing lists (vicidial_list). Losing this is unrecoverable without backups.

Section 1: Creating the Backup Directory Structure

Start by creating isolated storage for backups with proper permissions:

#!/bin/bash
# Setup backup storage directories

BACKUP_ROOT="/mnt/backups/vicidial"
MYSQL_BACKUPS="${BACKUP_ROOT}/mysql"
RECORDING_BACKUPS="${BACKUP_ROOT}/recordings"
CONFIG_BACKUPS="${BACKUP_ROOT}/configs"
LOG_BACKUPS="${BACKUP_ROOT}/logs"
ARCHIVE_BACKUPS="${BACKUP_ROOT}/archives"

# Create directory structure
mkdir -p ${MYSQL_BACKUPS}/{daily,weekly,monthly}
mkdir -p ${RECORDING_BACKUPS}/{daily,weekly}
mkdir -p ${CONFIG_BACKUPS}/{daily,weekly}
mkdir -p ${LOG_BACKUPS}/daily
mkdir -p ${ARCHIVE_BACKUPS}

# Set restrictive permissions (backup files contain sensitive data)
chmod 700 ${BACKUP_ROOT}
chmod 700 ${MYSQL_BACKUPS}
chmod 700 ${RECORDING_BACKUPS}
chmod 700 ${CONFIG_BACKUPS}
chmod 700 ${LOG_BACKUPS}

# Create backup user for cron (optional but recommended)
useradd -M -s /bin/false vicidial-backup 2>/dev/null || true
chown -R vicidial-backup:vicidial-backup ${BACKUP_ROOT}

echo "Backup directories initialized at ${BACKUP_ROOT}"
Enter fullscreen mode Exit fullscreen mode

Run this once as root to initialize the backup structure. In production, use a dedicated partition (e.g., /mnt/backups on a separate mount) to prevent backup disk issues from affecting the ViciDial system.

Section 2: Building the Core vicidial backup script

The centerpiece of your backup strategy is a comprehensive bash script that handles all components. Here's a production-grade implementation:

#!/bin/bash
#
# ViciDial Complete Backup Script
# Backs up: MySQL database, recordings, configs, logs
# Usage: ./vicidial-backup.sh [full|quick]
# Full = all components, Quick = DB + configs only
#

set -o pipefail
trap 'echo "Backup interrupted at $(date)" | mail -s "ViciDial Backup Error" admin@example.com' EXIT

# ===== CONFIGURATION =====
BACKUP_ROOT="/mnt/backups/vicidial"
MYSQL_BACKUPS="${BACKUP_ROOT}/mysql"
RECORDING_BACKUPS="${BACKUP_ROOT}/recordings"
CONFIG_BACKUPS="${BACKUP_ROOT}/configs"
ARCHIVE_BACKUPS="${BACKUP_ROOT}/archives"

MYSQL_USER="root"
MYSQL_PASSWORD=""  # Leave empty to use ~/.my.cnf
MYSQL_HOST="localhost"
MYSQL_DB="asterisk"

ASTERISK_USER="asterisk"
BACKUP_USER="vicidial-backup"

LOG_FILE="${BACKUP_ROOT}/backup.log"
BACKUP_DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_TIMESTAMP=$(date +%Y-%m-%d\ %H:%M:%S)

RETENTION_DAYS=7
COMPRESSION="pigz"  # faster than gzip; fallback to gzip
MAX_BACKUP_SIZE="50G"

# Color codes for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

# ===== FUNCTION: Logging =====
log_info() {
    echo "[${BACKUP_TIMESTAMP}] [INFO] $1" | tee -a ${LOG_FILE}
}

log_error() {
    echo -e "${RED}[${BACKUP_TIMESTAMP}] [ERROR] $1${NC}" | tee -a ${LOG_FILE}
}

log_success() {
    echo -e "${GREEN}[${BACKUP_TIMESTAMP}] [SUCCESS] $1${NC}" | tee -a ${LOG_FILE}
}

# ===== FUNCTION: Check Prerequisites =====
check_prerequisites() {
    log_info "Checking prerequisites..."

    # Check if running as root or sudoer
    if [[ ${EUID} -ne 0 ]]; then
        log_error "This script must be run as root"
        exit 1
    fi

    # Check required commands
    for cmd in mysqldump mysql tar ${COMPRESSION:-gzip} md5sum; do
        if ! command -v ${cmd} &> /dev/null; then
            log_error "Required command not found: ${cmd}"
            exit 1
        fi
    done

    # Check ViciDial directories
    for dir in /etc/asterisk /var/spool/asterisk/monitor /usr/share/astguiclient; do
        if [[ ! -d ${dir} ]]; then
            log_error "ViciDial directory not found: ${dir}"
            exit 1
        fi
    done

    # Test MySQL connectivity
    if ! mysql -h ${MYSQL_HOST} -u ${MYSQL_USER} -e "SELECT 1;" &> /dev/null; then
        log_error "Cannot connect to MySQL at ${MYSQL_HOST}"
        exit 1
    fi

    # Check backup storage space
    AVAILABLE_SPACE=$(df ${BACKUP_ROOT} | awk 'NR==2 {print $4}')
    REQUIRED_SPACE=$(($(du -sb ${RECORDING_BACKUPS} 2>/dev/null | awk '{print $1}') + 10737418240)) # +10GB buffer

    if (( AVAILABLE_SPACE < REQUIRED_SPACE )); then
        log_error "Insufficient disk space. Available: ${AVAILABLE_SPACE}KB, Required: ${REQUIRED_SPACE}KB"
        exit 1
    fi

    log_success "All prerequisites met"
}

# ===== FUNCTION: Backup MySQL Database =====
backup_mysql() {
    log_info "Starting MySQL backup of database: ${MYSQL_DB}"

    local BACKUP_FILE="${MYSQL_BACKUPS}/daily/asterisk_${BACKUP_DATE}.sql"
    local BACKUP_COMPRESSED="${BACKUP_FILE}.${COMPRESSION:-gz}"

    # Use mysqldump with optimizations for large databases
    if ! mysqldump \
        -h ${MYSQL_HOST} \
        -u ${MYSQL_USER} \
        --single-transaction \
        --lock-tables=false \
        --routines \
        --triggers \
        --events \
        --master-data=2 \
        --flush-logs \
        ${MYSQL_DB} | \
        ${COMPRESSION:-gzip} > ${BACKUP_COMPRESSED}; then

        log_error "MySQL backup failed"
        rm -f ${BACKUP_COMPRESSED}
        return 1
    fi

    # Verify backup integrity
    local BACKUP_SIZE=$(du -h ${BACKUP_COMPRESSED} | awk '{print $1}')
    local LINE_COUNT=$(${COMPRESSION:-gzip} -dc ${BACKUP_COMPRESSED} | wc -l)

    if (( LINE_COUNT < 1000 )); then
        log_error "MySQL backup appears incomplete (only ${LINE_COUNT} lines)"
        rm -f ${BACKUP_COMPRESSED}
        return 1
    fi

    # Calculate checksum
    md5sum ${BACKUP_COMPRESSED} > "${BACKUP_COMPRESSED}.md5"

    log_success "MySQL backup completed: ${BACKUP_SIZE} (${LINE_COUNT} lines)"
    echo "${BACKUP_COMPRESSED}" >> ${LOG_FILE}

    return 0
}

# ===== FUNCTION: Backup Call Recordings =====
backup_recordings() {
    log_info "Starting call recordings backup"

    local RECORDINGS_DIR="/var/spool/asterisk/monitor"
    local BACKUP_FILE="${RECORDING_BACKUPS}/daily/recordings_${BACKUP_DATE}.tar.${COMPRESSION:-gz}"

    # Find recordings modified in last 24 hours
    local RECORDING_COUNT=$(find ${RECORDINGS_DIR} -type f -mtime -1 | wc -l)
    log_info "Found ${RECORDING_COUNT} recordings from last 24 hours"

    if (( RECORDING_COUNT == 0 )); then
        log_info "No new recordings to backup"
        return 0
    fi

    # Backup with tar + compression, excluding temp files
    if ! tar \
        --exclude='*.tmp' \
        --exclude='*.lock' \
        -C ${RECORDINGS_DIR} \
        -cf - \
        $(find ${RECORDINGS_DIR} -type f -mtime -1 -printf '%f\n' | head -c 100000) 2>/dev/null | \
        ${COMPRESSION:-gzip} > ${BACKUP_FILE}; then

        log_error "Recordings backup failed"
        rm -f ${BACKUP_FILE}
        return 1
    fi

    local BACKUP_SIZE=$(du -h ${BACKUP_FILE} | awk '{print $1}')
    md5sum ${BACKUP_FILE} > "${BACKUP_FILE}.md5"

    log_success "Recordings backup completed: ${BACKUP_SIZE}"
    echo "${BACKUP_FILE}" >> ${LOG_FILE}

    return 0
}

# ===== FUNCTION: Backup Configuration Files =====
backup_configs() {
    log_info "Starting configuration files backup"

    local BACKUP_FILE="${CONFIG_BACKUPS}/daily/configs_${BACKUP_DATE}.tar.gz"

    # Tar all critical config files
    if ! tar \
        --gzip \
        -cf ${BACKUP_FILE} \
        /etc/asterisk/sip-vicidial.conf \
        /etc/asterisk/extensions-vicidial.conf \
        /etc/asterisk/voicemail-vicidial.conf \
        /usr/share/astguiclient/config.php \
        /etc/asterisk/manager.conf \
        /etc/asterisk/iax.conf \
        2>/dev/null; then

        log_error "Config backup failed"
        rm -f ${BACKUP_FILE}
        return 1
    fi

    local BACKUP_SIZE=$(du -h ${BACKUP_FILE} | awk '{print $1}')
    md5sum ${BACKUP_FILE} > "${BACKUP_FILE}.md5"

    log_success "Config backup completed: ${BACKUP_SIZE}"
    echo "${BACKUP_FILE}" >> ${LOG_FILE}

    return 0
}

# ===== FUNCTION: Cleanup Old Backups =====
cleanup_old_backups() {
    log_info "Cleaning up backups older than ${RETENTION_DAYS} days"

    local DELETED_COUNT=0

    for backup_dir in ${MYSQL_BACKUPS}/daily ${RECORDING_BACKUPS}/daily ${CONFIG_BACKUPS}/daily; do
        if [[ -d ${backup_dir} ]]; then
            DELETED_COUNT=$(find ${backup_dir} -type f -mtime +${RETENTION_DAYS} -delete -print | wc -l)
            log_info "Deleted ${DELETED_COUNT} backups from ${backup_dir}"
        fi
    done

    log_success "Cleanup completed"
}

# ===== FUNCTION: Generate Backup Report =====
generate_report() {
    log_info "Generating backup report"

    local REPORT_FILE="${BACKUP_ROOT}/backup_report_${BACKUP_DATE}.txt"

    cat > ${REPORT_FILE} << EOF
=====================================
ViciDial Backup Report
Generated: ${BACKUP_TIMESTAMP}
=====================================

BACKUP SUMMARY:
$(du -sh ${MYSQL_BACKUPS}/daily ${RECORDING_BACKUPS}/daily ${CONFIG_BACKUPS}/daily 2>/dev/null)

TOTAL BACKUP SIZE:
$(du -sh ${BACKUP_ROOT} | awk '{print $1}')

MYSQL BACKUPS (Last 7 Days):
$(ls -lh ${MYSQL_BACKUPS}/daily/ 2>/dev/null | tail -7)

RECORDING BACKUPS (Last 7 Days):
$(ls -lh ${RECORDING_BACKUPS}/daily/ 2>/dev/null | tail -7)

CONFIG BACKUPS (Last 7 Days):
$(ls -lh ${CONFIG_BACKUPS}/daily/ 2>/dev/null | tail -7)

BACKUP INTEGRITY CHECKS:
$(for f in ${MYSQL_BACKUPS}/daily/*.md5; do
  if md5sum -c ${f} &>/dev/null; then
    echo "✓ $(basename ${f%.*})"
  else
    echo "✗ FAILED: $(basename ${f%.*})"
  fi
done)

EOF

    log_success "Report generated: ${REPORT_FILE}"
    cat ${REPORT_FILE} >> ${LOG_FILE}
}

# ===== FUNCTION: Send Alert Email =====
send_alert() {
    local subject="$1"
    local body="$2"
    local recipient="admin@example.com"

    echo "${body}" | mail -s "${subject}" ${recipient}
}

# ===== MAIN EXECUTION =====
main() {
    local BACKUP_MODE="${1:-full}"

    log_info "========== ViciDial Backup Started =========="
    log_info "Backup Mode: ${BACKUP_MODE}"

    check_prerequisites || exit 1

    case ${BACKUP_MODE} in
        full)
            backup_mysql || exit 1
            backup_recordings || exit 1
            backup_configs || exit 1
            cleanup_old_backups
            generate_report
            log_success "========== Full Backup Completed =========="
            ;;
        quick)
            backup_mysql || exit 1
            backup_configs || exit 1
            log_success "========== Quick Backup Completed =========="
            ;;
        *)
            log_error "Unknown backup mode: ${BACKUP_MODE}"
            exit 1
            ;;
    esac
}

main "$@"
Enter fullscreen mode Exit fullscreen mode

Save this as /root/vicidial-backup.sh and make it executable:

chmod 750 /root/vicidial-backup.sh
Enter fullscreen mode Exit fullscreen mode

Testing the Backup Script

Run the script manually first to ensure it works:

/root/vicidial-backup.sh full
Enter fullscreen mode Exit fullscreen mode

Check the log file:

tail -50 /mnt/backups/vicidial/backup.log
Enter fullscreen mode Exit fullscreen mode

Verify backup files were created:

ls -lh /mnt/backups/vicidial/mysql/daily/
ls -lh /mnt/backups/vicidial/configs/daily/
Enter fullscreen mode Exit fullscreen mode

Section 3: Scheduling with Cron

Create a cron job to run the backup script automatically every day at 2 AM:

cat > /etc/cron.d/vicidial-backup << 'EOF'
# ViciDial Automated Backup
# Runs daily full backup at 2 AM
# Weekly archives on Sunday at 3 AM

0 2 * * * root /root/vicidial-backup.sh full >> /var/log/vicidial-backup.log 2>&1
0 3 * * 0 root /root/vicidial-backup.sh full && cd /mnt/backups/vicidial && tar -czf archives/weekly_$(date +\%Y\%m\%d).tar.gz mysql/daily configs/daily

EOF

chmod 644 /etc/cron.d/vicidial-backup
Enter fullscreen mode Exit fullscreen mode

Verify the cron job was added:

crontab -l | grep vicidial
# or
cat /etc/cron.d/vicidial-backup
Enter fullscreen mode Exit fullscreen mode

To test cron functionality without waiting 24 hours, manually run at a scheduled time:

# Run backup in 2 minutes
echo "0 $(date -d '+2 minutes' +%H:%M) * * * root /root/vicidial-backup.sh full" | at now
# Check scheduled jobs
atq
Enter fullscreen mode Exit fullscreen mode

Section 4: Backup Verification & Integrity Checks

Always verify your backups can be restored. Here's a validation script:

#!/bin/bash
# Backup Verification Script - Run weekly

BACKUP_ROOT="/mnt/backups/vicidial"
VERIFY_LOG="${BACKUP_ROOT}/verify.log"

verify_mysql_backup() {
    local backup_file="$1"

    echo "[$(date)] Verifying MySQL backup: ${backup_file}" >> ${VERIFY_LOG}

    # Check file integrity with md5
    if [[ -f "${backup_file}.md5" ]]; then
        if md5sum -c "${backup_file}.md5" &>> ${VERIFY_LOG}; then
            echo "[$(date)] ✓ MD5 check passed" >> ${VERIFY_LOG}
        else
            echo "[$(date)] ✗ MD5 check failed!" >> ${VERIFY_LOG}
            return 1
        fi
    fi

    # Try to restore to a temporary database for verification
    local temp_db="verify_test_$(date +%s)"

    if gunzip -c "${backup_file}" | mysql -u root &>> ${VERIFY_LOG}; then
        echo "[$(date)] ✓ Backup structure is valid" >> ${VERIFY_LOG}
    else
        echo "[$(date)] ✗ Backup restore test failed" >> ${VERIFY_LOG}
        return 1
    fi

    return 0
}

verify_config_backup() {
    local backup_file="$1"
    local temp_dir=$(mktemp -d)

    echo "[$(date)] Verifying config backup: ${backup_file}" >> ${VERIFY_LOG}

    if tar -tzf "${backup_file}" &>> ${VERIFY_LOG}; then
        echo "[$(date)] ✓ Config tar is valid" >> ${VERIFY_LOG}
        rm -rf ${temp_dir}
        return 0
    else
        echo "[$(date)] ✗ Config tar is corrupted" >> ${VERIFY_LOG}
        return 1
    fi
}

# Find most recent backups and verify
echo "[$(date)] ===== Backup Verification Run =====" >> ${VERIFY_LOG}

LATEST_MYSQL=$(ls -t ${BACKUP_ROOT}/mysql/daily/*.sql.gz 2>/dev/null | head -1)
LATEST_CONFIG=$(ls -t ${BACKUP_ROOT}/configs/daily/*.tar.gz 2>/dev/null | head -1)

if [[ -n ${LATEST_MYSQL} ]]; then
    verify_mysql_backup ${LATEST_MYSQL}
fi

if [[ -n ${LATEST_CONFIG} ]]; then
    verify_config_backup ${LATEST_CONFIG}
fi

# Check backup age (warn if > 24 hours old)
LATEST_BACKUP_TIME=$(ls -t ${BACKUP_ROOT}/mysql/daily/ | head -1 | xargs -I {} stat -c %Y ${BACKUP_ROOT}/mysql/daily/{})
CURRENT_TIME=$(date +%s)
BACKUP_AGE=$(( (CURRENT_TIME - LATEST_BACKUP_TIME) / 3600 ))

if (( BACKUP_AGE > 24 )); then
    echo "[$(date)] ⚠ WARNING: Last backup is ${BACKUP_AGE} hours old!" >> ${VERIFY_LOG}
fi
Enter fullscreen mode Exit fullscreen mode

Save as /root/verify-vicidial-backups.sh and run weekly:

chmod 750 /root/verify-vicidial-backups.sh

# Add to cron (Sunday at 4 AM)
echo "0 4 * * 0 root /root/verify-vicidial-backups.sh" >> /etc/cron.d/vicidial-backup
Enter fullscreen mode Exit fullscreen mode

Section 5: Restoring from Backup

When disaster strikes, know how to recover. Here's the restoration procedure:

Restore MySQL Database 

#!/bin/bash
# Restore MySQL from backup
# Usage: ./restore-mysql.sh /path/to/backup.sql.gz

BACKUP_FILE="$1"

if [[ ! -f ${BACKUP_FILE} ]]; then
    echo "Backup file not found: ${BACKUP_FILE}"
    exit 1
fi

echo "Preparing to restore from: ${BACKUP_FILE}"
echo "This will DROP the current 'asterisk' database. Continue? [y/N]"
read -r response

if [[ ! "${response}" =~ ^[yY]$ ]]; then
    echo "Restore cancelled"
    exit 0
fi

# Stop ViciDial services to prevent connection conflicts
systemctl stop vicidial asterisk 2>/dev/null

# Create backup of current database before restore
mysqldump asterisk > /tmp/asterisk_backup_$(date +%s).sql

# Drop and restore
mysql -u root << EOF
DROP DATABASE IF EXISTS asterisk;
CREATE DATABASE asterisk DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
EOF

# Restore from backup
gunzip -c ${BACKUP_FILE} | mysql -u root asterisk

# Verify restoration
if mysql -u root -e "SELECT COUNT(*) FROM asterisk.vicidial_users LIMIT 1;" &>/dev/null; then
    echo "✓ Database restored successfully"
    systemctl start asterisk vicidial
else
    echo "✗ Restore verification failed"
    exit 1
fi
Enter fullscreen mode Exit fullscreen mode

Restore Configuration Files 

# Extract config from backup
tar -tzf /mnt/backups/vicidial/configs/daily/configs_YYYYMMDD_HHMMSS.tar.gz

# Restore specific config
tar -xzf /mnt/backups/vicidial/configs/daily/configs_YYYYMMDD_HHMMSS.tar.gz \
    -C / \
    --strip-components=1 \
    etc/asterisk/sip-vicidial.conf

# Or restore all configs
tar -xzf /mnt/backups/vicidial/configs/daily/configs_YYYYMMDD_HHMMSS.tar.gz -C /

# Reload Asterisk configuration
asterisk -rx "reload"
Enter fullscreen mode Exit fullscreen mode

Restore Call Recordings 

# List recordings in backup
tar -tzf /mnt/backups/vicidial/recordings/daily/recordings_YYYYMMDD_HHMMSS.tar.gz | head -20

# Extract all recordings
tar -xzf /mnt/backups/vicidial/recordings/daily/recordings_YYYYMMDD_HHMMSS.tar.gz \
    -C /var/spool/asterisk/monitor/

# Fix ownership
chown -R asterisk:asterisk /var/spool/asterisk/monitor/
Enter fullscreen mode Exit fullscreen mode

Section 6: Monitoring & Alerting

Add health checks that email alerts if backups fail:

#!/bin/bash
# Backup Health Monitor - Run hourly

BACKUP_ROOT="/mnt/backups/vicidial"
ALERT_EMAIL="admin@example.com"
ALERT_THRESHOLD=86400  # 24 hours in seconds

check_backup_freshness() {
    local backup_dir="$1"
    local backup_name="$2"

    if [[ ! -d ${backup_dir} ]]; then
        send_alert "CRITICAL" "${backup_name} backup directory missing: ${backup_dir}"
        return 1
    fi

    local latest_backup=$(find ${backup_dir} -type f -name "*.gz" -o -name "*.sql" | sort -r | head -1)

    if [[ -z ${latest_backup} ]]; then
        send_alert "CRITICAL" "No backups found in ${backup_name}"
        return 1
    fi

    local backup_time=$(stat -c %Y "${latest_backup}")
    local current_time=$(date +%s)
    local backup_age=$((current_time - backup_time))

    if (( backup_age > ALERT_THRESHOLD )); then
        send_alert "WARNING" "${backup_name} backup is ${backup_age} seconds old"
        return 1
    fi

    return 0
}

send_alert() {
    local severity="$1"
    local message="$2"

    echo "${message}" | mail -s "[${severity}] ViciDial Backup Alert" ${ALERT_EMAIL}
    echo "[$(date)] ${severity}: ${message}" >> ${BACKUP_ROOT}/health.log
}

# Run checks
check_backup_freshness "${BACKUP_ROOT}/mysql/daily" "MySQL"
check_backup_freshness "${BACKUP_ROOT}/configs/daily" "Config"
check_backup_freshness "${BACKUP_ROOT}/recordings/daily" "Recordings"
Enter fullscreen mode Exit fullscreen mode

Add to cron (runs every hour):

echo "0 * * * * root /root/backup-health-monitor.sh" >> /etc/cron.d/vicidial-backup
Enter fullscreen mode Exit fullscreen mode

Section 7: Offsite Backup Replication

For true disaster recovery, replicate backups to remote storage:

#!/bin/bash
# Replicate backups to remote server via SFTP

BACKUP_ROOT="/mnt/backups/vicidial"
REMOTE_HOST="backup-server.example.com"
REMOTE_USER="vicidial-backup"
REMOTE_PATH="/backups/vicidial/"
SFTP_KEY="/root/.ssh/id_rsa_vicidial_backup"

replicate_to_remote() {
    local local_file="$1"
    local remote_file="$2"

    if ! scp -i ${SFTP_KEY} -q ${local_file} ${REMOTE_USER}@${REMOTE_HOST}:${REMOTE_PATH}${remote_file}; then
        echo "[$(date)] Failed to replicate ${local_file}" >> ${BACKUP_ROOT}/replicate.log
        return 1
    fi

    echo "[$(date)] Replicated ${local_file}" >> ${BACKUP_ROOT}/replicate.log
    return 0
}

# Replicate today's backups
for backup in $(find ${BACKUP_ROOT}/mysql/daily -type f -mtime -1); do
    replicate_to_remote "${backup}" "mysql/$(basename ${backup})"
done

for backup in $(find ${BACKUP_ROOT}/configs/daily -type f -mtime -1); do
    replicate_to_remote "${backup}" "configs/$(basename ${backup})"
done
Enter fullscreen mode Exit fullscreen mode

Add to cron (runs at 3 AM daily):

echo "0 3 * * * root /root/replicate-backups.sh" >> /etc/cron.d/vicidial-backup
Enter fullscreen mode Exit fullscreen mode

Section 8: Troubleshooting

Issue: "mysqldump: command not found"

Solution: Install MySQL client tools:

# CentOS/Rocky
yum install -y mariadb mysql

# Ubuntu/Debian
apt-get install -y mariadb-client mysql-client
Enter fullscreen mode Exit fullscreen mode

Issue: "Permission denied" when writing to backup directory

Solution: Check ownership and permissions:

ls -ld /mnt/backups/vicidial
chown -R vicidial-backup:vicidial-backup /mnt/backups/vicidial
chmod 750 /mnt/backups/vicidial
Enter fullscreen mode Exit fullscreen mode

Issue: "Backup file is incomplete" or corrupted tar

Solution: The backup was interrupted. Check:

# Verify backup file size is reasonable
ls -lh /mnt/backups/vicidial/mysql/daily/

# Check if disk was full
df -h /mnt/backups

# Review backup log for errors
tail -100 /mnt/backups/vicidial/backup.log | grep -i error
Enter fullscreen mode Exit fullscreen mode

Issue: Cron job not running

Solution: Verify cron is enabled and check logs:

# Check if crond is running
systemctl status crond  # CentOS/Rocky
systemctl status cron   # Ubuntu/Debian

# Check cron logs
tail -50 /var/log/cron  # CentOS/Rocky
grep CRON /var/log/syslog | tail -50  # Ubuntu/Debian

# Test cron environment
env -i /bin/sh -c /root/vicidial-backup.sh
Enter fullscreen mode Exit fullscreen mode

Issue: "Database already exists" error during test restore

Solution: Clean up test databases before each verification:

mysql -u root -e "DROP DATABASE IF EXISTS verify_test_*;"
Enter fullscreen mode Exit fullscreen mode

Issue: Backups consuming too much disk space

Solution: Reduce retention or increase compression:

# Change RETENTION_DAYS in vicidial-backup.sh from 7 to 3
sed -i 's/RETENTION_DAYS=7/RETENTION_DAYS=3/' /root/vicidial-backup.sh

# Switch to xz compression for better ratio (slower)
sed -i 's/COMPRESSION="pigz"/COMPRESSION="xz"/' /root/vicidial-backup.sh
Enter fullscreen mode Exit fullscreen mode

Issue: MySQL backup is much smaller

Top comments (0)