DEV Community: m4r4v

APIOLOGY a PHP API light framework

m4r4v — Fri, 22 Sep 2023 23:03:35 +0000

Just released a php api framework, to make things easy in order to handle backend requests in any host using php.

It can be found here

What I loved about it is the way it can handle http headers, including Origin, Methods and Authorization.

At the moment it just have BasicAuth and ApiKey available.

The versioning works great, no need to reinvent the wheel. To create a version is as easy as the following:

Add version name and status to settings
Create a folder with the Version Number i.e: v2
Copy resources.php from v1 folder and paste it inside the new version folder.
Use a Namespace according to the version i.e.: Apiology\Api\Resources\V2

for testing purposes, it has instruction on how to run a docker container, instructions and be read here

Love to have a review about it and improvements are always welcome.

Thanks

How to access android files from linux terminal

m4r4v — Fri, 22 Jan 2021 18:08:17 +0000

Hi there,
If you ever wanted to know how to access our android phone files from terminal, without using any third party app and just with a USB cable and GNome, here is how:

We need to check the GNOME Virtual file system (gfvs) to see it is available.

run the following command:

ls /run/user/1000/gvfs/

You will get an Media Transfer Protocol (mtp), for more about this, check this link: MTP

'mtp:host=PHONE_MANUFACTURER_AND_MODEL'

Now use it as it would be any other directory on your system

cd /run/user/1000/gvfs/mtp:host=PHONE_MANUFACTURER_AND_MODEL

Done, you can access your phone directory and do whaterver you want.

Bye!

How to fix audio from ubuntu 20.04 focal fossa "dummy-output"

m4r4v — Fri, 08 Jan 2021 14:23:51 +0000

Hello, I came across an issue from the latest update. I couldn't listen to any sound from my earphones it was only showing dummy-output as the default Output Device, which it was absolutely not right.

To fix it, I did the following (It must be done as root, so used sudo for it):

sudo vim /etc/modprobe.d/alsa-base.conf

At the end of the file I added:

options snd-hda-intel dmic_detect=0

Then I did:

sudo vim /etc/modprobe.d/blacklist.conf

At the end of the file I added:

blacklist snd_soc_skl

I didn't even had to reboot, but, somehow it is always good to reboot the system after editing configuration files.

Note: I am using an acer swift laptop (intel) so please make sure that this works for you. I also use vim but it can also be done in your text editor of your preference

Happy Coding!

Bash Script to Install FireFoxDeveloper Edition

m4r4v — Fri, 11 Sep 2020 15:49:33 +0000

I am a big fan of firefox and since it came out with the finest distribution for developers firefox developer edition it really made my web dev life easie.

Installing it in Linux is not that easy as it is in windows or in mac, but that's the fun about linux hehe, at least it is for me.

I've seen many articles and tutorials where people explain how to install it, but most of them it just get rid of firefox and replace it with the firefox developer edition. I just like to have both browsers the released and stable version and the unstable developer version.

So far I've been doing the installation manually and every time they have a new release I had to run the same command line again and again, got boring to me and I just created my automated script for it to do it on its own.

I uploaded the script to my github update-firefox-developer-linux for everyone to use it.

let me show it to you here:

#!/bin/bash

# Source reference: https://www.rogerpence.com/posts/a-bash-script-to-install-firefox-dev-edition

function installFFDE {
    cd ~/Downloads
    ffdFile= firefox-*.tar.bz2
    tar -xvf $ffdFile
    mv firefox firefox-developer
    sudo cp -r firefox-developer/ /opt/
    echo "files were copied into /opt/"
    rm -r firefox-developer && rm -r $ffdFile
    echo "All files were removed, your're good to go"
}

function downloadAndInstallFFDE {
    curl -o releases.txt https://download-installer.cdn.mozilla.net/pub/devedition/releases/
    echo "Downloaded..."
    latest=$(grep -o '[0-9][0-9]\.[0-9][a-z][0-9]' releases.txt | tail -1)  
    echo "getting latest release...\nthe latest release is:\nfirefox-$latest.tar.bz2" 
    echo "got latest version"
    ffdFile=firefox-$latest.tar.bz2
    rm releases.txt
    curl -o $ffdFile https://download-installer.cdn.mozilla.net/pub/devedition/releases/$latest/linux-x86_64/en-US/$ffdFile
    echo "got latest version. Proceeding to install it"
    tar -xvf $ffdFile
    mv firefox firefox-developer
    sudo cp -r firefox-developer/ /opt/
    echo "files were copied into /opt/"
    rm -r firefox-developer && rm -r $ffdFile
    echo "All files were removed, your're good to go"
}

FILE=firefox-*.tar.bz2

if [ -e $FILE ]; then
    installFFDE
else 
    echo "installation file does not exists, will start downloading it now..."
    downloadAndInstallFFDE
fi

hope you all like it, let me know any improvement suggestions and happy to help.

Don't forget to Follow, ❤️ or 🦄
Happy Coding!!!

Level 1 Completed

m4r4v — Wed, 29 Jul 2020 23:47:13 +0000

Hi everyone,
I got the foobar challenge and just wanted to share with you all that I just completed level 1.

It was a nice challenge, I have no idea why I love to challenge myself.

If you have done this challenge, please share your experience

Take care everyone, stay safe and happy coding!!!

rsync - 10 examples in 11 days (Day 11)

m4r4v — Sun, 26 Jul 2020 19:36:32 +0000

Day 11

10 examples in 11 days

Log Files

Well it seems we have come to the end of the series. Have in mind that rsync has a lot more to offer so please read the man page to get and use the best of rsync.

I can't end this series without talking about a great option. It's about the --log-file=FILE option.

Sometimes you may want or need a log file, let's say you want to check out what was done, or like what happened to me, our manager always made us send every morning an email with the log of all sync done the day before (we made a script for that hehe). But let's face it, it's a good practice to create a log file almost for everything (I do it all the time).

So, having rsync create its own log file after every sync huh?, yup, it's as easy as it sounds, let me show you what the syntax can look like:

rsync [--options] [Source] [Destination] --log-file=[Where the log file will be stored]

If you read Day 10 great and if you haven't, please do because today's option will use a lot of the arguments shown there. I will still use the folders I've been using so far Users and Sync.

Now, to get the picture, I will do the following:

I will empty out all files inside Sync
I will run rsync with the log file option
I will show what the log file looks like

iamgroot@laptop:~$ rm -r Sync/ && rsync -a Users/ Sync --log-file=Sync/logfile.txt && ls -la Sync/ && cat Sync/logfile.txt
total 16
drwxrwxr-x  2 iamgroot iamgroot 4096 jul 26 13:48 .
drwxrwxr-x 38 iamgroot iamgroot 4096 jul 26 02:15 ..
-rw-rw-r--  1 iamgroot iamgroot   30 jul 26 13:48 file1.txt
-rw-r--r--  1 iamgroot iamgroot  217 jul 26 14:16 logfile.txt
2020/07/26 14:16:35 [13004] building file list
2020/07/26 14:16:35 [13004] .d..t...... ./
2020/07/26 14:16:35 [13004] >f+++++++++ file1.txt
2020/07/26 14:16:35 [13004] sent 163 bytes  received 43 bytes  total size 30

You can see what the logfile.txt looks like, let's go through it

2020/07/26 14:16:35 [13004] building file list: Rsync always shows this when it starts

2020/07/26 14:16:35 [13004] .d..t...... ./ The directory of the source using the -i option

2020/07/26 14:16:35 [13004] >f+++++++++ file1.txt The file sync displayed using the -i option

2020/07/26 14:16:35 [13004] sent 163 bytes received 43 bytes total size 30 End of the sync

As I said, rsync log file uses by default the -i option that I explained in Day 10.

But what if we want to add some different arguments to the log file?

Well this is what I love about rsync, it always gives us something else. In this case we have the --log-file-format option. Is like the --out-format but explicitly for the log file.

Let's say I only want the length of the file in bytes, then we can use the %l argument like this:

iamgroot@laptop:~$ rsync -a Users/ Sync --log-file=Sync/logfile.txt --log-file-format='%l' && cat Sync/logfile.txt
2020/07/26 14:23:52 [14572] building file list
2020/07/26 14:23:52 [14572] 4096
2020/07/26 14:23:52 [14572] 30
2020/07/26 14:23:52 [14572] sent 163 bytes  received 43 bytes  total size 30

Let's compare what we had before and what we've got now:

Before: 2020/07/26 14:16:35 [13004] .d..t...... ./

Now: 2020/07/26 14:23:52 [14572] 4096

Before: 2020/07/26 14:16:35 [13004] >f+++++++++ file1.txt

Now: 2020/07/26 14:23:52 [14572] 30

The default options are %o %h [%a] %m (%u) %f %l", and a "%t [%p]" and all the options available are:

option	description
%a	the remote IP address (only available for a daemon)
%b	the number of bytes actually transferred
%B	the permission bits of the file (e.g. rwxrwxrwt)
%c	the total size of the block checksums received for the basis file (only when sending)
%C	the full-file checksum if it is known for the file. For older rsync protocols/versions, the checksum was salted, and is thus not a useful value (and is not displayed when that is the case). For the checksum to output for a file, either the --checksum option must be in-effect or the file must have been transferred without a salted checksum being used. See the --checksum-choice option for a way to choose the algorithm.
%f	the filename (long form on sender; no trailing "/")
%G	the gid of the file (decimal) or "DEFAULT"
%h	the remote host name (only available for a daemon)
%i	an itemized list of what is being updated
%l	the length of the file in bytes
%L	the string "-> SYMLINK", "=> HARDLINK", or "" (where SYMLINK or HARDLINK is a filename)
%m	the module name
%M	the last-modified time of the file
%n	the filename (short form; trailing "/" on dir)
%o	the operation, which is "send", "recv", or "del." (the latter includes the trailing period)
%p	the process ID of this rsync session
%P	the module path
%t	the current date time
%u	the authenticated username or an empty string
%U	the uid of the file (decimal)

Now I will give you all a nice tip, like I did in Day 06, and I really suggest you do this if you work in a team. Add the logged user $USER who is performing the sync and also the file that was sync. Let me show you how:

iamgroot@laptop:~$ rsync -a Users/ Sync --log-file=Sync/logfile.txt --log-file-format='%i --File: %n -- User: '$USER && cat Sync/logfile.txt
2020/07/26 14:33:03 [15332] building file list
2020/07/26 14:33:03 [15332] .d..t...... --File: ./ -- User: iamgroot
2020/07/26 14:33:03 [15332] >f+++++++++ --File: file1.txt -- User: iamgroot
2020/07/26 14:33:03 [15332] sent 163 bytes  received 43 bytes  total size 30

Isn't this awesome?

OK, that'll be it for today and this is the end of the rsync series. I really hope you all enjoyed it. If you have any doubt don't hesitate asking me, will be more than happy to help.

Thanks for reading, keep it safe!!

Don't forget to Follow, ❤️ or 🦄

iamgroot@laptop:~$ exit

👍

rsync - 10 examples in 11 days (Day 10)

m4r4v — Sun, 26 Jul 2020 06:10:09 +0000

Day 10

10 examples in 11 days

Sync --i, --itemize-changes

Now in Day 10 I will talk about a flag that can't be out of this series. I am talking about the -i option.

This option allows us to know what changes were made. This option comes very handy when it used along with the --dry-run flag.

Here is what the man page says about it:

-i,--itemize-changes       output a change-summary for all updates

But how is the output will look like?

Whenever we use the -i we get a string of characters YXcstpoguax (not all of them) on which each one has its own meaning:

source: man rsync

Y and X

Y/X	Description
Y	Is the type of update
X	Is the file-type

Y	Description
<	a file is being transferred to the remote host (sent)
>	a file is being transferred to the local host (received)
c	means that a local change/creation is occurring for the item (such as the creation of a directory or the changing of a symlink, etc.).
h	the item is a hard link to another item (requires--hard-links.
.	the item is not being updated (though it might have attributes that are being modified)
*	means that the rest of the itemized-output area contains a message (e.g. "deleting")

X	Description
f	for a file
d	for a directory
L	for a symlink
D	for a device
S	for a special file (e.g. named sockets and fifos).

Attributes cstpoguax

Attributes	Description
c	Means either that a regular file has a different checksum (requires --checksum) or that a symlink, device, or special file has a changed value. Note that if you are sending files to an rsync prior to 3.0.1, this change flag will be present only for checksum-differing regular files.
s	Means the size of a regular file is different and will be updated by the file transfer.
t	Means the modification time is different and is being updated to the sender’s value (requires --times). An alternate value of T means that the modification time will be set to the transfer time, which happens when a file/symlink/device is updated without --times and when a symlink is changed and the receiver can’t set its time.
p	Means the permissions are different and are being updated to the sender’s value (requires --perms).
o	Means the owner is different and is being updated to the sender’s value (requires --owner and super-user privileges).
g	Means the group is different and is being updated to the sender’s value (requires --group and the authority to set the group).
u	Slot is reserved for future use.
a	Means that the ACL information changed.
x	Means that the extended attribute information changed.

Let's see an example using a file inside our already known Users and Sync folders:

Note: I will run this with the -n flag which is he short for --dry-run

iamgroot@laptop:~$ rsync -vhrin Users/ Sync
sending incremental file list
>f+++++++++ file1.txt

sent 69 bytes  received 19 bytes  176.00 bytes/sec
total size is 12  speedup is 0.14 (DRY RUN)

As you can see, we get >f+++++++++ file1.txt that means:

> : a file is being transferred to the local host (received)
f : file not a directory
+++++++++ : no other attributes (because we don't have anything in our Sync directory)

If I sync our folders and after doing it, I modify the file and sync it again, I get:

f.sT...... file1.txt.

I got an s meaning that the file size changed and I also got a T instead of the t attribute. The T means that the modification time will be set to the transfer time.

If I change the owner and the group to root, and run my rsync as super-user, I would get:

>f.st.og... file1.txt

Verifying Owner and Group of the file so you can see the reuslt above

iamgroot@laptop:~$ sudo chown root ~/Users/file1.txt && sudo chgrp root ~/Users/file1.txt && sudo rsync -vhaio /home/iamgroot/Users/ /home/iamgroot/Sync
sending incremental file list
.d..t...... ./
>f.st.og... file1.txt

sent 186 bytes  received 38 bytes  448.00 bytes/sec
total size is 58  speedup is 0.26
iamgroot@laptop:~$ ls -la Sync/file1.txt
-rw-rw-r-- 1 root root 58 jul 26 01:20 Sync/file1.txt

I will end up this post, mentioning that is very useful to use the --out-format='arguments' with this, just for reference I will show you an example:

iamgroot@laptop:~$ rsync -vhrin --out-format='%l %n %i' /home/iamgroot/Users/ /home/iamgroot/Sync
sending incremental file list
58 file1.txt >f+++++++++

sent 67 bytes  received 19 bytes  172.00 bytes/sec
total size is 58  speedup is 0.67 (DRY RUN)

The final output is 58 which is the length in bytes and the argument is the %l. We asked for the file name (file1.txt) using the %n argument and the %i to get the changes >f+++++++++

I think you get the idea, if not or if you have any doubt don't hesitate asking me, will be more than happy to help.

Ok, that'll be it for today.

Thanks for reading, keep it safe!!

Follow, ❤️ or 🦄

rsync - 10 examples in 11 days (Day 09)

m4r4v — Sat, 25 Jul 2020 06:34:01 +0000

Day 09

10 examples in 11 days

Sync Local to Remote over SSH and Limit Bandwith

Today is going to be a simple but useful example. I will explain how to limit the bandwidth.

Let me set a very common scenario. Sometimes we need to upload a very large file or multiple files to a remote instance. We know that this task may take a long time to transfer up to our remote instance, this is ok if we are the only one who is using the internet or if we don't care having rsync consuming all of our bandwidth. But if we are in a shared internet connection or with a limited bandwidth, we don't want to cause a downtime to our connectivity.

The solution is to limit the bandwitdh, this way rsync can transfer everything correctly and we can still work in something else without loosing connectivity performance.

The option to do this is --bwlimit=KBPS. KBPS stands for KiloBytes Per Second. Let's check what the man page says about this option:

--bwlimit=RATE

This option allows you to specify the maximum transfer rate for the data sent over the socket, specified in units per second. The RATE value can be suffixed with a string to indicate a size
multiplier, and may be a fractional value (e.g. "--bwlimit=1.5m"). If no suffix is specified, the value will be assumed to be in units of 1024 bytes (as if "K" or "KiB" had been appended).
See the --max-size option for a description of all the available suffixes. A value of zero specifies no limit.

For backward-compatibility reasons, the rate limit will be rounded to the nearest KiB unit, so no rate smaller than 1024 bytes per second is possible.

Rsync writes data over the socket in blocks, and this option both limits the size of the blocks that rsync writes, and tries to keep the average transfer rate at the requested limit. Some
"burstiness" may be seen where rsync writes out a block of data and then sleeps to bring the average rate into compliance.

Due to the internal buffering of data, the --progress option may not be an accurate reflection on how fast the data is being sent. This is because some files can show up as being rapidly sent
when the data is quickly buffered, while other can show up as very slow when the flushing of the output buffer occurs. This may be fixed in a future version.

The man page is very explicit on how we can limit the bandwidth of our rsync process.

What I'll do?

I will backup a file that is 5,3MB in size, and will limit the bandwith to 400Kbps. These means it will take up to 10 or 12 seconds to upload my file.

iamgroot@laptop:~$ rsync --progress --bwlimit=400 -vhre ssh /home/iamgroot/backup-to-upload/backup_iamgroot.tar.gz iamgroot@222.222.222.222:backups/
sending incremental file list
backup_iamgroot.tar.gz
          5.30M 100%  407.18kB/s    0:00:12 (xfr#1, to-chk=0/1)

sent 5.30M bytes  received 35 bytes  321.43K bytes/sec
total size is 5.30M  speedup is 1.00

here is an animated gif of rsync while transfering:

Everything was completely uploaded, let me show you:

iamgroot@remote-instance:~$ ls -l --b=KB  backups/backup_iamgroot.tar.gz | cut -d " " -f5
5303kB

Simple right?.

Well that's it for today's example, hope you enjoyed reading.

Thanks for reading, keep it safe!!

Follow, ❤ or 🦄

rsync - 10 examples in 11 days (Day 08)

m4r4v — Thu, 23 Jul 2020 18:40:17 +0000

Day 08

10 examples in 11 days

Sync Remote to Local over SSH

Yesterday on Day 07 I showed how we can sync files from our local machine to our remote machine using ssh. Today is the other way around. I will request a file from remote to local.

The syntax it's almost the same one from yesterday, the difference between yesterday's and the the one I am using today is that with the one below, I will not remove my remote file and the yesterday's source will be our destination and our destination will be yesterday's source. Let's remember the syntax and let me describe it to you so you can see by your self what I am talking about:

Note: As yesterday I will not show the local nor remote machines IP. Instead I will always display 111.111.111.111 IP as remote and 222.222.222.222 as local

iamgroot@laptop:~$ rsync (1)--progress (2)-vhre (3)ssh (4)iamgroot@111.111.111.111:(5)[remote folder]/(6)[remote file] (7)[local folder to download]

Let's see what the above syntax does:

--progress: shows progress while synching
I used -vhre options

-v: verbose
-h: human readable
-r: recursive
-e: allows us to use ssh as remote shell and execute its commands

ssh: invoking the ssh service to be used as remote shell.
ssh connection parameters
remote folder with full path
remote file to download
local folder where the remote file will be downloaded

Note: If the local folder doesn't exist it will be created

I created a folder called download-folder in my remote instance and inside of it I created a file called file_download.txt. I will download this file into my ~/Sync folder that we have been using.

iamgroot@remote-instance:~$ mkdir download-folder && touch download-folder/file_download.txt && echo "Hello World I will be downloaded" >> download-folder/file_download.txt

Now let's sync from remote to local

iamgroot@laptop:~$ rsync --progress -vhre ssh iamgroot@111.111.111.111:download-folder/file_download.txt ~/Sync
receiving incremental file list
file_download.txt
             12 100%   11.72kB/s    0:00:00 (xfr#1, to-chk=0/1)

sent 43 bytes  received 113 bytes  34.67 bytes/sec
total size is 12  speedup is 0.08

Let's check if it got downloaded

iamgroot@laptop:~$ ls Sync/
file1.txt  file2.txt  file3.txt  file_download.txt

Nice right?

As you can see so far, rsync is a great tool to keep your files sync locally and remotely.

We are almost ending this series, hope you all readers are enjoying it, let me know if you have any doubt and I will be more than happy to help.

Thanks for reading, keep it safe!!

Follow, ❤ or 🦄

rsync - 10 examples in 11 days (Day 07)

m4r4v — Thu, 23 Jul 2020 07:23:28 +0000

Day 07

10 examples in 11 days

Sync Local to Remote over SSH

Today I will talk about how we can sync files from our local machine to our remote machine using ssh.

I have a remote machine where I will be connecting to. I use ssh with a public and private key to connect to it and this way I will not have to be asked for a password.

Note: I will not show the local nor remote machines IP. Instead I will always display 111.111.111.111 IP as remote and 222.222.222.222 as local

I have a folder called ~/backup-to-upload/. Inside of this folder I store the latest backup file as a .tar.gz. This file will be sitting there waiting for me to rsync it into my remote machine. Once I have upload the tar backup, it gets deleted leaving the ~/backup-to-upload/ folder empty.

Let's see the syntax:

iamgroot@laptop:~$ rsync (1)--progress (2)--remove-source-files (3)-vhre (4)ssh (5)[backup file to upload] (6)iamgroot@111.111.111.111:(7)[remote backup folder]/

Let's see what the above syntax does:

--progress: shows progress while synching
--remove-source-files: after transfer it will remove source file
I used -vhre

-v: verbose
-h: human readable
-r: recursive
-e: allows us to use ssh as remote shell and execute its commands

ssh: invoking the ssh service to be used as remote shell.
backup file to upload with full path
ssh connection
remote folder

Note: If the remote folder doesn't exist it will be created

Let's see what I have at my remote machine before we run the rsync command

iamgroot@remote-instance:~$ ls
scripts

Now let's upload our tar file to the remote machine

iamgroot@laptop:~$ rsync --progress --remove-source-files -vhre ssh /home/iamgroot/backup-to-upload/backup.tar.gz  iamgroot@111.111.111.111:backups/
sending incremental file list
backup.tar.gz
            195 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/1)

sent 295 bytes  received 43 bytes  75.11 bytes/sec
total size is 195  speedup is 0.58

Checking if the tar backup file got removed from source

iamgroot@laptop:~$ ls -a backup-to-upload/
.  ..

Empty!!!

Now let's check on the remote machine and first I will check if the folder backups was created and then if it has our backup.tar.gz

iamgroot@laptop:~$ ssh iamgroot@111.111.111.111
Last login: Thu Jul 23 00:38:46 2020 from 222.222.222.222
iamgroot@remote-instance:~$ ls
backups  scripts
iamgroot@remote-instance:~$ ls -a backups/
.  ..  backup.tar.gz

As you can see, it not just created the backups folder, it uploaded our tar file.

Awesome, another example accomplished!

Ok, that'll be for today's example, thanks for reading, ping me if you need any help!!!

Follow, ❤ or 🦄

rsync - 10 examples in 11 days (Day 06)

m4r4v — Tue, 21 Jul 2020 23:52:40 +0000

Day 06

10 examples in 11 days

Add Linux User + Unix date stamp to backup folder

Today's article is a short article I will not talk about an option of rsync, is just a useful syntax that we can attach to the backup destination folder.

In Day 04 I talked about backups and what I will mention today comes very handy when doing backups while synching.

So, Adding a USER + UNIX TIMESTAMP?

Yes, as you read, adding a linux user can be useful to know who did it and adding a timestamp let's us know when.

Let me show with an example:

Let's see what my Source, Destination and Backup directories have

iamgroot@laptop:~$ ls -F ~/Users
file1.txt  file2.txt  file3.txt
iamgroot@laptop:~$ ls -F ~/Sync
iamgroot@laptop:~$ ls -F ~/backups

Source ~/Users has file1.txt, file2.txt and file3.txt
Destination ~/Sync is empty
Backups ~/backups is empty

Let's now add the current linux User (\$USER) separated by a dash unix timestamp to the backup folder

iamgroot@laptop:~$ rsync -vhrb --backup-dir='/home/iamgroot/backups/'$(date +$USER-%s) --suffix='.bak' ~/Users/ ~/Sync
sending incremental file list
file1.txt
file2.txt
file3.txt

sent 276 bytes  received 73 bytes  698.00 bytes/sec
total size is 36  speedup is 0.10

Let's check each folder again

iamgroot@laptop:~$  ls -F ~/Users && printf -- '-%.0s' {1..10}; echo "" && ls -F ~/Sync && printf -- '-%.0s' {1..10}; echo "" && ls -F ~/backups/ && ls -F ~/backups/iamgroot-1595372929/
file1.txt  file2.txt  file3.txt
----------
file1.txt  file2.txt  file3.txt
----------
iamgroot-1595372929/
file1.txt.bak  file2.txt.bak  file3.txt.bak

Obviously you can add $(date +$USER-%s) as suffix or using how ever you want.

Cool!!!. Now you know a different way to perform a single task

Ok, that'll be for today's example, thanks for reading!!!

Follow, ❤ or 🦄

rsync - 10 examples in 11 days (Day 05)

m4r4v — Tue, 21 Jul 2020 02:23:42 +0000

Day 05

10 examples in 11 days

Delete

Today is the turn of the --delete option. I use this option a lot, when it comes to backups or to mantain source and destination files up to date.

Eventually you sync too much files and sometimes you just delete some files at source without removing them in destination. You know rsync does not remove any file at destination no matter if they do not exist at source, it just sync files/directories using blocks on files/directories that exist on both sides.

This is when --delete come handy. If you remove a file/s on soure and if you use this option, it will remove those files that doesn't exist in source at destination.

In our last article about backups I used ~/Users folder as source on which I had a file called file1.txt and we used it to sync and backup to our destination ~/Sync folder. I will create another file and will call it file2.txt and remove our file1.txt file.

iamgroot@laptop:~$ rm ~/Users/file1.txt && touch ~/Users/file2.txt && echo "Hello World" >> ~/Users/file2.txt && ls -F ~/Users/
file2.txt

Now, I will sync our source and destinations folders and check what we have in both folders

iamgroot@laptop:~$ ls -F ~/Users && ls -F ~/Sync
file2.txt
file1.txt  file2.txt

As you can see, in our source folder we only have file2.txt and in our destination we still have file1.txt. Here is when we should use the --delete option. Using it, will remove files in destination missing at source

iamgroot@laptop:~$ rsync -vhr --delete ~/Users/ ~/Sync && ls -F ~/Users && ls -F ~/Sync
sending incremental file list
deleting file1.txt
file2.txt

sent 124 bytes  received 48 bytes  344.00 bytes/sec
total size is 12  speedup is 0.07
file2.txt
file2.txt

Now we only have file2.txt in both folders. Nice!!

I do have to point out something. We do have few more options to play with:

--delete-before	receiver deletes before transfer, not during (this is the default)
--delete-during	receiver deletes during the transfer
--delete-delay	find deletions during, delete after
--delete-after	receiver deletes after transfer, not during
--delete-excluded	also delete excluded files from dest dirs

Another great option that we must include in this article is the --remove-source-files which removes from source synchronized files. Be aware that this option don't remove directories. This option is very useful when doing backups (tar, zip files), since backups are usually compressed files we barely want to keep those files at source after synching. Let's try it out, I will create a folder called ~/Users/backup, then I will create some files inside ~/Users/backup/ and at the end I will back them up using tar. After all this I will perform a sync and will check if it gets removed

iamgroot@laptop:~$ mkdir ~/Users/backup && touch ~/Users/backup/tobackup{1..5}.txt && tar -cvzf ~/Users/backup.tar.gz ~/Users/backup/ && ls -F ~/Users
tar: Removing leading `/' from member names
/home/iamgroot/Users/backup/
/home/iamgroot/Users/backup/tobackup3.txt
/home/iamgroot/Users/backup/tobackup4.txt
/home/iamgroot/Users/backup/tobackup5.txt
/home/iamgroot/Users/backup/tobackup2.txt
/home/iamgroot/Users/backup/tobackup1.txt
backup/  backup.tar.gz  file2.txt

Performing rsync with --remove-source-files option

iamgroot@laptop:~$ rsync -vhr --remove-source-files ~/Users/backup.tar.gz ~/Sync && ls -F ~/Users
sending incremental file list
backup.tar.gz

sent 292 bytes  received 43 bytes  670.00 bytes/sec
total size is 196  speedup is 0.59
backup/  file2.txt

Ok, that'll be for today's example, thanks for reading!!!

Follow, ❤ or 🦄