DEV Community

Cover image for Automated Backups Made Easy: & Lessons Learned From This Project
Stanley Chinedu Ogada
Stanley Chinedu Ogada

Posted on

2 2 2 2 2

Automated Backups Made Easy: & Lessons Learned From This Project

Table of Contents:

  1. Introduction
  2. Project Overview
  3. Discoveries and Resolutions
  4. Conclusion

Introduction:

In the ever-evolving world of technology, ensuring the safety of our data is paramount. Recently, I completed a project that automates backup tasks, aiming to simplify the process and enhance data security. While the project proved successful, I encountered a stumbling block that led me on a troubleshooting journey. In this blog post, I'll share the project overview and the valuable lessons learned from resolving a crucial issue.

Project Overview:

The project (GitHub link: Auto-Backup Project) automates the addition of backup tasks to the crontab for daily, weekly, and monthly backups. All compressions (.tar.gz) are stored in a designated backup directory, and SSH key authentication is required for remote hosts. The script intelligently manages backup files, retaining only the most recent ones to optimize storage usage.

Discoveries and Resolutions:

While developing the project, a significant challenge surfaced when incorporating the rsync command into the script for scheduled execution via crontab. Here are the two major discoveries that solved the issue:

Use Full Paths in Crontabs:

  • Crontab entries should include the full path to scripts. Correct example:

     * * * * * /usr/local/bin/backup.sh
    

Specify Full Paths for Rsync and SSH Key

  • In the script, provide the full path to the rsync program and explicitly specify the SSH key for the remote system.

     /usr/bin/rsync -a -e "ssh -i /home/user/.ssh/vps-1-private-key" /home/user/backups user@vps-1.com
    
  • To troubleshoot, add verbose logging with the -vv option to the rsync in the script and redirect the crontab output to a file for monitoring.

     * * * * * /usr/local/bin/backup.sh >> /home/user/.backup.cron.log 
    
    

Conclusion:

Developing the Auto-Backup Project was a rewarding experience, and overcoming the challenge highlighted the importance of attention to detail. By sharing these discoveries, I hope to assist fellow developers facing similar issues. Remember, the devil is in the details, and providing full paths can make all the difference in the world of scheduled tasks. Explore the GitHub repository for more details and project documentation.

Check out a related problem posted on StackOverFlow

Also, connect with me on LinkedIn

Happy hacking!

Image of Docusign

🛠️ Bring your solution into Docusign. Reach over 1.6M customers.

Docusign is now extensible. Overcome challenges with disconnected products and inaccessible data by bringing your solutions into Docusign and publishing to 1.6M customers in the App Center.

Learn more

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay