DEV Community

Alan Varghese
Alan Varghese

Posted on

Automate Your Linux Updates Like a Pro (with Rollbacks!) 🚀

#bash #devops #automation #linux

Running sudo apt-get update && sudo apt-get upgrade -y is fine for your local dev machine. But when you're managing multiple Ubuntu or Debian servers, manual updates are a recipe for disaster.

One day, a kernel update will break a driver, or a library conflict will brick a service. If you don't have a history of what was installed or a way to roll back, you're in for a long night of troubleshooting.

I built System Update Manager, a production-ready Bash tool that treats system updates like a CI/CD pipeline.

The Problem with "Yolo-Upgrading"

  1. No Audit Trail: Did that server crash because of an update 2 hours ago or a config change?
  2. The "Point of No Return": Once apt finishes, knowing exactly which versions were upgraded is a pain.
  3. Concurrency Issues: Running multiple update scripts at once can lock the dpkg database.

My Solution: System Update Manager

This isn't just a wrapper for apt. It’s a full management suite designed for reliability.

Key Features 🛠️

  • SQLite History Tracking: Every update attempt is logged in a local database. You can see the start time, duration, exit codes, and exactly which packages changed.
  • Pre-update Snapshots: Before touching a single byte, it snapshots your package list.
  • Automated Rollback Scripts: If an update fails, the tool generates a standalone .sh script that uses apt-get install --allow-downgrades to bring your system back to its previous state.
  • Native Automation: It ships with Systemd timer and Cron configurations out of the box.
  • Lock Management: Uses file-based locking to ensure only one instance is running.

How It Works

The script follows a strict lifecycle for every update:

  1. Pre-flight Check: Validates root privileges and disk space.
  2. Locking: Acquires a lock file to prevent concurrent runs.
  3. Snapshot: Records current package versions in SQLite.
  4. Update: Executes apt-get update and upgrade with configurable retries.
  5. Recovery: If a failure occurs, it immediately generates a rollback script.
  6. Logging: Records the outcome and duration.

A Peek at the Status UI

I wanted the CLI to be readable at a glance:

╔══════════════════════════════════════════════════════╗
║          SYSTEM UPDATE MANAGER - STATUS              ║
╠══════════════════════════════════════════════════════╣
║ Status: SUCCESS                                     ║
║ Start Time:  2024-03-24 03:00:00                    ║
║ End Time:    2024-03-24 03:05:23                    ║
║ Exit Code:   0                                      ║
╚══════════════════════════════════════════════════════╝
Enter fullscreen mode Exit fullscreen mode

Getting Started

If you want to try it out (or use it in your own homelab/prod environment), here is the quick start:

# Clone it
git clone https://github.com/yourusername/automated_system_update_manager.git
cd automated_system_update_manager

# Check for updates without changing anything
./update_manager.sh --check

# Run a manual install
sudo ./update_manager.sh --install
Enter fullscreen mode Exit fullscreen mode

Automation via Systemd

The cleanest way to run this is via a Systemd timer. It’s more modern than Cron and integrates perfectly with journalctl.

sudo cp systemd/update-manager.* /etc/systemd/system/
sudo systemctl enable --now update-manager.timer
Enter fullscreen mode Exit fullscreen mode

Now your system will update daily at 3 AM, and you can sleep soundly knowing there's a rollback script waiting if things go south.

Lessons Learned

Building this project reminded me that Bash is still a superpower. By combining simple Unix tools like grep, awk, and sqlite3, you can build incredibly robust infrastructure tools without the overhead of a heavy Python or Go binary.

What do you use to manage updates on your servers? Let me know in the comments! 👇

Top comments (0)