AWS Series:Automating File Backups to AWS S3(Windows Server)

#aws #backup #devops #cloud

This tutorial is for windows server but should work in principle for other operating systems

INSTALL AWS CLI

install the aws cli using the install instructions specified here

After successful installation setup default user profiles,access id and access key by running the following command

aws configure
AWS Access Key ID [None]: <AccessID>
AWS Secret Access Key [None]: secretkey
Default region name [None]: us-east-1
Default output format [None]: json

if you dont have an ACCESS KEY, ACCESS ID follow the link below to create one here

If you don't specify the region when running aws configure, you'll have to specify the region your s3 bucket is in when running s3 commands.

S3 COMMANDS

When using s3 commands one path argument must be specified. paths can be one of two types, localpath or an S3Uri The localpath represents the local directory, the S3Uri represents a URI of your S3 bucket*

example
localpath - C://Desktop/backups
S3URI: - s3://yourbucket/yourkey

S3 operations generally take this pattern.

aws s3 s3command source_file/folder destinationpath

The operations can be from a localpath —→ S3URI, S3URI —→ S3URI ,S3URI —→ localpath

Common S3 commands include *cp,mv,rm,sync* etc

Sample command

aws s3 cp "C://Desktop/backups/myfile.txt" s3://your-bucket-name

This command copies the file myfile.txt to the bucket your-bucket-name no region was specified because the region was specified during aws configure you can use --region flag to specify the region your bucket is in.

aws s3 sync "C://Desktop/backups/" s3://your-bucket-name

The above commands copies all contents of the folder backups to the bucket your-bucket-name

The sync command copies files only if

if the size of the local file is different than the size of the s3 object,
The last modified time of the local file is newer than the last modified time of the s3 object,
The local file does not exist under the specified bucket and prefix

Using filters flag with s3 commands (--exclude, --include)

--excludes removes specific files or folders when an s3 command is run

--includes adds specific files or folders when an s3 command is run

the --exclude and the --include can be used multiple times in the same command

aws s3 sync "C://Desktop/backups/" s3://your-bucket-name --exclude "*" --include "*.jpg" --include "*.png" --include "*.txt"

The following command excludes all files in the folder backups and includes all files in the backups folder that have the extensions jpg,txt,png

You can test your settings by adding the --dryrun flag to your command

aws s3 sync "C://Desktop/backups/" s3://your-bucket-name --exclude "*" --include "*.jpg" --include "*.png" --include "*.txt" --dryrun

By default all files are included during s3 operations and the last applied filter takes precedence

Create Backup with Batch Files

Create batch file
- open an empty txt file using notepad
- type in the s3 command you want to execute,here i want to sync my backups folders to an s3 bucket aws s3 sync "C://Desktop/backups/" s3://your-bucket-name
- save the txt file as a batch file(.bat)
Open windows task scheduler,
- Select Create Task
- Add task name and description
- Add trigger (One time,Daily,Weekly,Monthly)
- Add actions
- Save your task.