DEV Community: SkySilk Cloud

How To Use Rsync to Backup Your VPS Files Off-Site

SkySilk Cloud — Thu, 08 Apr 2021 07:08:43 +0000

Sync is a simple yet powerful tool for system administrators. It allows you to easily copy and synchronize files from one server to another or from your local machine to a VPS server (off-site). Rsync is a smart tool that allows you to use a special algorithm so you can synchronize files as they are edited or added.

There are two ways you can utilize the Rsync command to keep a backup of your production server on a remote backup server.

The sync method
The snapshot method

Both methods can be automated using a Cron job. However, you need to first set up the production server to login as a root in the backup server without a passphrase.

Prerequisites

A server running Ubuntu 18.04 – server A
A backup server running ubuntu 18.04 – server B

Deploy Ubuntu 18.04

Setting Up the Production Server to Login Into the Backup Server

1 ssh root@server-A-IP

Create a keygen on your production server. (Make sure you create a passphraseless key).

1 ssh-keygen -t rsa -b 4096 -v

Copy the keys in your new backup server so that the production server is able to access the backup server as root.

1 ssh-copy-id server-B-IP

Once you run the above command, it will ask you for the root password of your backup server. This is a one-time process. You will type Server B’s root password only once.

Test the authentication by logging in to a remote server inside the production server’s SSH shell.

1 ssh root@server-B-IP

You will be logged in to the backup server as root without needing a password.

On successful authentication, try a few commands with sudo privileges. Once you’re done testing a few commands, logout from the backup server after creating a directory where you’ll be storing the backup.

For this tutorial, we will use /root/backup as the backup location. Create this directory with the below command.

1 cd /root | mkdir backup


1 exit

The Sync Method

In this method, the rsync command will first look for changes on the production server. The Rsync will process files that match one of the following conditions:

It is a new file added in the “source”
The file has recently been edited and does not match the file in the destination folder except the file name.
The file is removed from the “source”

So, before running the sync command, create a few files on “source”
Login to your production server as root.

1 ssh root@server-A-IP

Create files named 1.html index.html and hello.html with the below command.

1 cd /var/www/html | touch 1.html index.html hello.html

To run the active sync once, run the following command on your production server.

1 rsync -azvP --delete /var/www/html/ root@serverB-ip:/root/backup/

The command returns a message like this.

sending incremental file list

./

1.html

              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=2/4)

hello.html

              0 100%    0.00kB/s    0:00:00 (xfr#2, to-chk=1/4)

index.html

              0 100%    0.00kB/s    0:00:00 (xfr#3, to-chk=0/4)

sent 222 bytes  received 76 bytes  596.00 bytes/sec

total size is 0  speedup is 0.00

The next time you run this command again, the response should be something like this.

sending incremental file list

sent 107 bytes  received 12 bytes  238.00 bytes/sec


total size is 0  speedup is 0.00

As you can see, rsync was smart enough to detect that there are no changes in the “source”. That is why, rsync did not transfer any file to the “destination”.

Test the sync by creating 2.html and deleting hello.html followed by the same rsync command.

1 cd /var/www/html | touch 2.html | rm -r hello.html


1 rsync -azvP --delete /var/www/html/ root@serverB-ip:/root/backup/

This time, rsync recognized the changes and synchronized the file that was added. Also, rsync deleted hello.html in the “destination” directory as it does not exist in the “source” anymore.

sending incremental file list

deleting hello.html

./

2.html

              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=1/4)

sent 150 bytes  received 52 bytes  134.67 bytes/sec

total size is 0  speedup is 0.00

To facilitate the synchronization, create a bash script so that you don’t have to type the whole command everytime you want to synchronize the files.

1 cd /root


1 nano sync.sh

In the nano editor, paste the following and save.

#!/bin/sh

#Automating the rsync to smart sync html folder to backup server

rsync -azvP –delete /var/www/html/ root@serverB-ip:/root/backup/

Save and exit the file using ctrl+X followed by Y and Enter.

Now, everytime you want to manually synchronize the files to the backup server, run the following command.

1 /root/sync.sh

Setting up Cronjob to Activate the Sync

Once you’re convinced that the Rsync command works as required, automate it with a new Cronjob.

1 crontab -e

Choose the editor you want to use. Add the following line at the end of this file.

1 30 20 * * 1 /root/sync.sh

The above cron will run every monday at 8:30 PM and sync all the files from source directory on the production server to the destination directory on your backup server.

Save the file using ctrl+X followed by Y and Enter.

The Snapshot Method

The sync method is not a scalable and effective method to keep backup of a production site.

When you delete a file from the source destination, it will be automatically deleted from the destination directory as well when the cron runs. You can not retrieve the deleted file afterwards.

To tackle this problem, you can create snapshots. You can set it up in the following manner.

Set up the rsync command to synchronize the files every morning at 9:30 AM
Create a snapshot of the site weekly on every monday at 10:30 AM.

This way, you will retain a copy of the source directory in a separate folder on the backup server.

Instead of creating a snapshot and further transferring it to the backup server, create a bash script to do everything at once with a single line of code.

Create a script

1 cd /root


1 nano snapshot.sh

In the nano editor, paste the below content and change the following values.

!/bin/sh

# Create a timestamp

date=date "+%Y-%m-%d-t-%H-%M-%S"


# Define the directory to retain on your production server


SOURCE=/var/www/html/

# Specify the destination folder on your backup VPS

DEST=/root/snapshots

# Execute rsync followed by cleanup

rsync -arvt -t \

  –delete \

  –link-dest=../current \

  $SOURCE root@server-B-IP:$DEST/backup-$ddate \

  && ssh root@server-B-IP \

  “rm -rf $DEST/current \

  && ln -s $DEST/backup-$date $DEST/current \

  && find $DEST -maxdepth 1 -type d -mtime +30 -exec rm -r {} +”

Change server-B-IP with your backup server’s public IP. Save the file and exit using ctrl+X followed by Y and Enter.

Run this script with the following command

1 /root/snapshot.sh

When you run the command, it does the following:

It creates a snapshot of the “source” directory and further transfers it to the backup server in the specified destination folder
It creates a soft link “current” in the destination directory to the latest snapshot inside the directory.
It also checks for snapshots that are older than 30 days and further deletes it to conserve space on your backup server.

Automating the Snapshots and Sync with Cron job

To create an efficient backup of your files and sync the current version on the backup server, you can set up 2 cron jobs.

Edit the crontab

1 crontab -e

Choose the editor you want to use and add the following line at the end of this file.

# synchronize the files from the production server to the backup server every morning at 9:30 am

30 9 * * * /root/sync.sh



# create a snapshot of the source directory weekly on every Monday

30 10 * * 1 /root/sync.sh

Save the file and exit using ctrl+X followed by Y and Enter.

Conclusion

At this stage you have set up a daily off-site active sync along with weekly off-site snapshots.

You can do much more with Rsync command itself and automate it with bash scripts and cron jobs.

What Professions Will Cloud Computing Make Redundant?

SkySilk Cloud — Tue, 30 Mar 2021 13:36:22 +0000

An increasing number of businesses, and the IT sector, are relying on automation for growth. The recurring question becomes, what organizational changes will cloud computing bring?

Although businesses and institutions desire to embrace cloud computing as a disruptive technology, they don't desire to change their organizational structures. The fact remains that you can’t adopt disruptive technologies without affecting your organizational structure. And the individuals who perform various tasks in your organization.

Effects of Disruptive Technology on IT Professions

Disruptive technologies influence IT professions in two ways: profession transition and profession loss. Employees dread the two; with the latter dreaded the most. Cloud computing is not a commerce craze anymore. It’s here for the long haul, and it’s changing IT professions.

A study by the Center for Business and Economic Research at Ball State University found that 85% of jobs lost in the U.S. from 2000 to 2010 were attributed to technological change due to automation.

It’s predicted in the next 5-10 years that cloud computing will hit IT professions the hardest. Will cloud computing eliminate jobs? No, the embracing of cloud computing will move IT professions instead of eliminating them.

IT professions will see decreased internal demand. Fresh IT professions will increase with the support of remote cloud networks. The internal IT professions that are anticipated to see reduced demand in the foreseeable future are:

System Administrators and Database Administrators - Cloud computing remotely streamlines networks. This will create a reduced demand for internal system and database administrators. IT experts with this background will still be needed by cloud providers, as more institutions bank on the cloud for management.
Help Desk Support - Like with preferred management posts, the function of help desk personnel will not be made entirely redundant. It’s presumed that the pair of helpdesk and system support personnel roles will lessen in-house as companies migrate to the cloud. They will evolve within remote service providers.
Infrastructure and Network Professionals - These positions will be absorbed by cloud providers, as there will be an increase in demand for network specialist to maintain cloud networks.

The Increase of the Cloud Job Market

With the exponential growth in the cloud computing industry, there will be increased demand for cloud computing specialists. The demand will continue to increase for the foreseeable future. The projection is that 80% of professions predicted to be formed in 2025 don’t even exist.

As more organizations and businesses spur benefit from container platforms, infrastructure as code solutions, software-defined networking, storage, continuous integration delivery, and AI, the demand for people with skills and profound technical understanding of the cloud will grow. Job market growth for the IT sector will be in cloud services.

Cloud computing profiles accessible in the current job market are a result of employer demand propelling innovation. This is the principle of providing new business apps and end-user services.

Sought-after Cloud Computing Professions

Cloud computing’s impact on IT jobs is evident. The high paying cloud computing jobs that have increased in demand are:

Project Manager Possessing IT Proficiency - They are required to restate an institution’s cloud computing concepts and their impact on various projects. The average base pay is $88,397
Cloud Architect/Cloud Developer/Cloud System Administrator - They will oversee an institution’s cloud computing blueprint from deployment and management to support of cloud apps. The average base pay is $64,892
Cloud Consultants - They will carry out an institution’s technical analysis and evaluation. They will also give cloud technology recommendations to bolster productivity and capability. The average base pay is $86,619
Business Intelligence Analyst - They will work together with most IT department personnel to boost performance and production. The average base salary is $85,277
Software Developer - They will be tasked with updating, maintaining, and creating software. The average base pay is $76,526. A software engineer intern earns on average $30,000.

So if you are asking yourself, is cloud computing a good career for me? Yes, it is lucrative but you’ll have to have passion, dedication, discipline, and the drive to succeed.

Closing Remarks…

Cloud computing continues to grow at remarkable speed as organizations and businesses continue spending on cloud services like SaaS, PaaS, IaaS, and FaaS. The endorsement of cloud technology platforms gives institutions immeasurable opportunities for innovation.

However, in order to stay at pace with this innovation, and the rapid batch of services provided by cloud providers, all businesses and organizations will require skilled personnel. They will offer these professionals lucrative and rewarding careers in the cloud computing industry for years to come.

In-demand professionals will be required year-after-year and the question that will be left unanswered is, will demand outpace the supply of these professionals? Only time will tell.

We’re thrilled to be here with you!

SkySilk Cloud — Wed, 03 Mar 2021 11:08:50 +0000

Hi everyone! We’re thrilled to be here at Dev.to

We’re SkySilk, a cloud computing company. We offer simple cloud computing solutions for beginners, aspiring developers, experienced developers, or anyone who’s fascinated with developing.

Our purpose is to meet the growing public demand for virtualized hardware through cloud utilization. We believe cloud computing is the future. It will continue to grow in every aspect of our day-to-day lives; from communication and interaction with others to learning, shopping, banking, traveling, and more...thus shaping our future.

That’s why we created a cloud server management service that’s not only secure but cost-effective. Along the way, we’ve managed to stay up-to-date with the latest technology ensuring our customers get the best user experience. We want to become a global leader in Cloud Hosting, Virtualization Management, and Enterprise-level Data Security.

Currently, it's becoming a necessity to own a cloud. The conventional IT infrastructure is outpaced and the ability to effectively utilize the cloud is priceless.

What We Stand For

We want to make deployment as simple as flipping a switch. And we’ve managed to do that as developers or teams can deploy their applications seamlessly. We’ve done this by investing and spending countless hours improving our platform. Though there are still a lot of improvements to be made we’re on the right track.

Everything we do is aimed at creating a platform for developers and teams to deploy and scale their applications quickly and efficiently. Our joy is having a part to play in your next award-winning project that’s going to change the world.

This has been our driving force from our inception. In the end, we want everyone to be able to harness the power of the cloud. To some, this might seem too ambitious but we’re driven and motivated to make it happen

Why We Do What We Do

It’s our obligation to spread cloud computing technology for the greater good of humanity. Your own or your team’s long-awaited project(s) shouldn’t be stored away on a local hard drive because you couldn’t deploy due to lack of funds. We enable you to showcase your creations to the world and bring them to life.

We pride ourselves on being able to offer enterprise-grade services to those who just dreamed about it. Your success is our success. Our platform is your platform.

“Cloud is about how you do computing, not where you do computing” - Paul Maritz, Computer Scientist.

How Happy We are to Contribute to this Channel

It gives us great pleasure to be able to showcase our expertise and experience in various subjects and hands-on scenarios. We will bring our unique visioning ability to all community members.

We’re experienced in many areas related to cloud computing, including deployment, monitoring, and management. For example, we’ve helped individuals and businesses alike with premium resources to scale upwards and NEVER downwards.

The Sky's the Limit

We have a strong conviction that there are no limits whatsoever to achieving your ultimate goal. The Sky represents abundance and anyone can own their own cloud. Ours is to give you a robust and secure platform to do your thing. Don’t worry about downtime or DDoS. Just do your magic and deploy seamlessly.

We’re going to have lots of fun and lots to share that will be meant to give you value and insight into the development world. Also, if you want additional support on any of your projects or just share value that you feel is important, we have an active community that’s willing to listen, learn and also advise you on Discord. Every attendee:

Has a right to be heard by every member
Is free to propose subjects to cover, ask questions, chip in on every conversation
Is welcome to invite anyone who wishes to participate

We ensure:

Every attendee is heard
There is a friendly, conducive, and productive environment ALWAYS (but we do joke from time-to-time)
There is an aura of love

If you’ve got any questions, suggestions, remarks, shoot away. We’re honored to be able to assist anyone and also take notes from you. When you’re around, do add us so that we can always learn and share from each other.

We’ll also be delighted to get back to you on any comments you left down below.
Anyone can own a cloud. If you’ve been taken for granted by people not wanting to listen to what you have to say, you’ve got a chance to own and have your voice here. We want to build one of the most helpful, satisfying, informative, and coolest cloud computing communities around.

Once again, it’s an honor to be here and we’re confident that you’re going to get the very best from us. We stick by whatever we say and never go back on any of our promises or commitments.

Do come and sail with us in the cloud.