DEV Community: Zaki Arrozi Arsyad

docker : basic commands

Zaki Arrozi Arsyad — Sun, 12 Jul 2020 16:55:47 +0000

Docker info

check the version or if docker is already installed

$ docker --verison

show system information regarding docker installation

$ docker info

show stats of runnign container

$ docker stats

show container log

$ docker logs <CONTAINER_ID>

show processes of container

$ docker top <CONTAINER_ID>

show detailed info of container

$ docker inspect <CONTAINER_ID>

show modified files in a container

$ docker diff <CONTAINER_ID>

show mapped ports in a container

$ docker port <CONTAINER_ID>

Help / manual

list available commands

$ docker

manual for specific command

$ docker <COMMAND> --help

Run a new container

create a new container

// create from latest version
$ docker create <IMAGE>

// create from specific version
$ docker create <IMAGE[:TAG]>

start a container

$ docker start <CONTAINER_ID>

run (create and start) a new container

// create from latest version
$ docker run <IMAGE>

// create from specific version
$ docker run <IMAGE[:TAG]>

run in the background

$ docker run -d <IMAGE>

run and assign it a name

$ docker run --name <CONTAINER_NAME> <IMAGE>

run and map a port

$ docker run -p <HOST_PORT>:<CONTAINER_PORT> <IMAGE>

run and map all port

$ docker run -P <IMAGE>

run and assign a hostname

$ docker run --hostname <HOSTNAME> <IMAGE>

run and add a dns entry

$ docker run --add-host <HOSTNAME>:<IP> <IMAGE>

run and map a local dir

$ docker run -v <HOST_DIR>:<TARGET_DIR> <IMAGE>

run and change the entry point

$ docker run -it --executable <COMMAND> <IMAGE>

Manage Container

show list of all running container

$ docker ps

show list of all container

$ docker ps -a

stop a running container

$ docker stop <CONTAINER_ID>

kill a running container

$ docker kill <CONTAINER_ID>

start a stopped container

$ docker start <CONTAINER_ID>

restart a container

$ docker restart <CONTAINER_ID>

suspend a container

$ docker pause <CONTAINER_ID>

resume a container

$ docker unpause <CONTAINER_ID>

rename a container

$ docker rename <OLD_NAME> <NEW_NAME>

start shell inside a running container

$ docker exce -it <CONTAINER_ID> <COMMAND>

copy a file from host to container

$ docker cp <SOURCE> <CONTAINER_ID>:<DESTINATION>

copy a file from container to host

$ docker cp <CONTAINER_ID>:<SOURCE> <DESTINATION>

delete a container

$ docker rm <CONTAINER_ID>

destroy (kill + rm) a running container

$ docker rm -f <CONTAINER_ID>

delete all stopped containers

$ docker container prune

create an image out of container

$ docker commit <CONTAINER_ID>

Manage Images

download an image

$ docker pull <IMAGE[:TAG]>

upload an image to repository

$ docker push <IMAGE[:TAG]>

show list of images

$ docker images

build an image

$ docker build <DIRECTORY>

build and tag the image

$ docker build -t <IMAGE[:TAG]> <DIRECTORY>

tag an image

$ docker tag <SOURCE_IMAGE[:TAG]> <TARGET_IMAGE[:TAG]>

delete an image

$ docker rmi <IMAGE>

delete dangling images

$ docker image prune

delete all unused images

$ docker image prune -a

save an image to tar file

$ docker save <IMAGE> > <FILE.tar>

load an image from tar file

$ docker load -i <FILE.tar>

docker : installation

Zaki Arrozi Arsyad — Sun, 12 Jul 2020 16:55:13 +0000

Run commands to install docker in an ubuntu machine

Docker Engine

// download the script
$ curl -fsSL get.docker.com -o get-docker.sh

// create permission
$ chmod +x get-docker.sh

//run the script 
$ sh get-docker.sh

// check if it is successfully installed
$ docker --version

Docker Compose

Docker Compose helps us to running multi-container Docker applications with a YAML file configuration.

// download docker-compose
$ sudo curl -L "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

// Apply docker-compose
$ sudo chmod +x /usr/local/bin/docker-compose

// check if it is successfully installed
$ docker-compose --version

Docker Machine

Docker Machine lets us install Docker Engine on virtual hosts, and manage the hosts with docker-machine commands.

// install docker-machine
$ base=https://github.com/docker/machine/releases/download/v0.16.0 &&
  curl -L $base/docker-machine-$(uname -s)-$(uname -m) >/tmp/docker-machine &&
  sudo mv /tmp/docker-machine /usr/local/bin/docker-machine &&
  chmod +x /usr/local/bin/docker-machine

// check if it is successfully installed
$ docker-machine --version

note : adding our user to the “docker” group, so that we can use docker as a non-root user

$ sudo usermod -aG docker your-user

git : basic usage

Zaki Arrozi Arsyad — Mon, 22 Jun 2020 17:23:33 +0000

First thing first, we need to make sure if we already have git installed in our machine, by running one of these commands

git
git --verison

Cloning a repo

I prefer to use SSH instead of HTTP.
If you haven't set up the key, please read this post

$ git clone git@github.com:zakiarsyad/js-snake.git

Check remote repository

Remote repository is versions of our project that are hosted on the Internet.

$ git remote -v
origin git@github.com:zakiarsyad/js-snake.git (fetch)
origin git@github.com:zakiarsyad/js-snake.git (push)

By default we will work with the origin remote.

We can have more than one remote, if we need to pulling from other repositories.

Check status

$ git status
On branch master
nothing to commit, working tree clean

This command tells us which branch we are in. If there are any changes in our code, those will be displayed here as well, as long as we don't commit the changes yet.

The main branch in git is master. Basically we don't work directly in master branch.
We might need another branch like development or other environment to test the code before we put our code in the main branch. We also might need a new branch every time we build a new feature or fixing a bug.

Create a new branch

$ git checkout -b "our-branch"

Run this command to create a new branch from branch we are in previously.

This command will create a branch from our origin remote repository. If we We also can branch out directly from other remote repository.

Add changes

$ git add <file>

After creating changes in our code, we need to add it to be committed in git. So that we can track our changes later.

We can add the file one by one, or just run git add . to add all changes we have made.

As long as we don't commit the changes yet, we can unstage our changes to be committed by running this command.

$ git restore --staged <file>

Or use git restore --staged . to unstage all changes.

Commit changes

$ git commit -m "put-our-message-here"

Commit is like when we save our changes. Every time we save, git creates an unique ID that tells us what changes were made when and by who. We also need to put the message which is a brief description of what changes were made.

The commit ID or commit hash is a unique identifier for changes that have made. If needed, in the future we can easily revert the code to any version anytime in one click.

View log

$ git log

We can easily track our changes by using this command. All changes in our branch are listed with its commit ID and message.

Push changes

$ git push origin our-branch

We'll need to put our code in our online repository.

Create a Pull Request

To combine our changes to master branch or other base branch, we need to create a Pull Request.

Open a pull request from our branch to the base. Put title and brief description about what we have done. Then ask others to review.

Merge a Pull Request

If everything is ok, then it time to update the base branch by merging our changes.

git : getting started with git

Zaki Arrozi Arsyad — Sun, 21 Jun 2020 17:20:48 +0000

Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency.

Git

Git is the most commonly used for version control. Git help us to makes collaboration easy. Even we work by our self, we will love it.

Git runs in our local machine. We also can use online host such as Github, Gitlab, and Bitbucket.

Installing git

We can easily download git from this Download page.
For MacOs, I prefer to use homebrew instead. Just simply type this command brew install git.

And in linux, I will do apt-get install git.

Run git in our terminal to check if it's successfully installed.

Connecting to github with SSH

There are two ways if we want to communicate with github.

First one is using HTTP, and the second one is using SSH. Actually we don't need anything else if we want to clone a github repository using HTTP, but we will need some configurations if we want to use SSH.

This example is explaining in linux machine, need a little adjustment if we run MacOs or Windows.

Generating a new SSH key

Run this terminal command to create a new SSH key

$ ssh-keygen -t rsa -b 4096 -C "our_email@example.com"

By default, the key will saved in this path ~/.ssh/id_rsa.pub

Adding the SSH key to our github account

Copy the id_rsa.pub file generated by previous command.

Go to our github account. Find Settings - SSH and GPGP keys - New SSH key.
Create a Title and paste our Key there, then hit Add SSH Key.

We can run this command to test if our key is successfully configured.

$ ssh -T git@github.com
Hi zakiarsyad! You've successfully authenticated, but GitHub does not provide shell access.

Setting our git username

# setting our username
$ git config --global user.name "zakiarsyad"

# confirm that we set userneame correctly
$ git config --global user.name
zakiarsyad

Setting our commit email address

# setting our username
$ git config --global user.email "zakiarsyad@mail.com"

# confirm that we set userneame correctly
$ git config --global user.email
zakiarsyad@mail.com

git is ready to use in our local machine.

linux : bash script conditions

Zaki Arrozi Arsyad — Sat, 20 Jun 2020 18:06:29 +0000

Basic usage of if statement in bash scripting

if [ CONDITION ]; then
    FIRST_COMMANDS
elif [ CONDITION ]; then
    SECOND_COMMANDS
else
    THRID_COMMANDS
fi

Always use space between the brackets in the condition.
Always terminate the line with ; before adding a new keyword

if [ CONDITION ]; then
Use ! to invert the condition

if [ ! CONDITION ]; then
Combine conditions using -a for AND, and -o for OR

There are three syntaxes for define the condition

1. Single-bracket

Support 3 types of conditions

File-based condition

[ -a file ] or [ -e file ] : if the file is exists
[ -d directory ] : if the directory is exists and it's a directory
[ -r file ] : if the file is exists and is readable to the script
[ -w file ] : if the file is exists and is writeable to the script
[ -x file ] : if the file is exists and is executable to the script
[ -s file ] : if the file is exists and has a size of more than 0 bytes
[ file1 -ef file2] : if the files refer to the same device/inode number
[ file1 -nt file2] : if the file1 was changed more recently than file2, or if file1 exists and file2 doesn't
[ file1 -nt file2] : if the file1 was changed longer ago than file2, or if file1 exists and exists
[ -N file ] : if the file is exists and was modified after the last read
[ -g file ] : if the file is exists and is a set-group-ID
[ -u file ] : if the file is exists and is a set-user-ID
[ -G file ] : if the file is exists and is owned by the effective group ID
[ -O file ] : if the file is exists and is owned by the user executing the script
[ -b file ] : if the file is exists and is a block special
[ -c file ] : if the file is exists and is a character special
[ -f file ] : if the file is exists and is a regular file
[ -h file ] or [ -L file ] : if the file is exists and is a symbolic link
[ -k file ] : if the file is exists and has its sticky bit set
[ -p file ] : if the file is exists and is a named pipe
[ -S file ] : if the file is exists and is a socket
[ -t file ] : if the file descriptor is exists and refers to an open terminal

String-based condition

[ STRING1 == STRING2 ] : if equal
[ STRING1 != STRING2 ] : if not equal
[ -n STRING ] : if has length of more than zero
[ -z STRING ] : if an empty string
[ STRING =~ PATTERN ] : if match the regex pattern

Arithmetic condition

[ NUM1 -eq NUM2 ] : if equal
[ NUM1 -ne NUM2 ] : if not equal
[ NUM1 -gt NUM2 ] : if greater than
[ NUM1 -ge NUM2 ] : if greater than equal
[ NUM1 -lt NUM2 ] : if less than
[ NUM1 -le NUM2 ] : if less than equal

2. Double-bracket

This is the enhanced version of single-bracket. All features in single-bracket syntax are compatible here. We can start using this syntax.

The differences :

Can use regex

[[ STRING =~ PATTERN ]] : if match the regex pattern
Can use && and || instead of -a and -o

3. Double-parenthesis

This is another syntax for arithmetic condition

(( NUM1 == NUM2 )) : if equal
(( NUM1 != NUM2 )) : if not equal
(( NUM1 > NUM2 )) : if grater than
(( NUM1 >= NUM2 )) : if grater than equal
(( NUM1 < NUM2 )) : if less than
(( NUM1 <= NUM2 )) : if less than equal

We also can quickly test a condition like this

# using &&
[ $foo == "bar" ] && echo true

# using ||
[ $foo == "bar" ] || echo true

linux : bash script

Zaki Arrozi Arsyad — Sat, 20 Jun 2020 16:16:14 +0000

We can run the bash script directly in the terminal.

$ echo "This is a bash script."
This is a bash script.

We also can store the script in a file, so that we can run it every time we want without typing the script in terminal.

# !/bin/bash
echo "This is a bash script."

We need to update the file permission if we want to run the bash file. We can run this command, and specify the file permission

$ chmod 700 FILE
$ ./FILE
This is a bash script.

We can run linux command in bash file and run the bash script in the cli.

The bash file will very useful if we want to use some variable or put logics in the script.

VARIABLE

Positional Parameter Variable

# !/bin/bash
echo "This is a $1 $2."

we can give a value for $1 and $2 after the run command.

$ ./FILE bash script

User Defined Variable

# !/bin/bash
# define a string value, print as is
print1='script'

# define a string value, we can add variable inside the double quotes
print2="bash $print1"

# define a number
print3=3

# define a command
print4=`clear`

echo $script

Environment Variable

Usually we use .env file to store the environment variable

PORT=80
ENV=development

operation for variable

# using let
let sum=$number1+$number2

# using expr
sum=`expr $number1 + $number2`
length=`expr length $string`

declaring variable type

# declare an intrger
declare -i number
number=100

IF STATEMENT

if [ CONDITION ]; then
    TRUE_COMMANDS
else
    FALSE_COMMANDS
fi

WHILE LOOPING

while [ CONDITION ]
do
    COMMANDS
done

FOR LOOPING

for VARIABLE in LIST
do
    COMMANDS
done

linux : package and service management

Zaki Arrozi Arsyad — Sat, 20 Jun 2020 16:03:29 +0000

Package management

First, we need to update our repository

$ apt-get update

Install a package

$ apt-get install APPNAME

Check if a package is already installed

$ dpkg -l | grep APPNAME

Remove a package

$ apt-get remove APPNAME

Search packages by a keyword

$ apt-cache search “KEYWORD”

Service management

Check if a service is running

$ systemctl status APPNAME

Start a service

$ systemctl start APPNAME

Stop a service

$ systemctl stop APPNAME

Restart a service

$ systemctl restart APPNAME

Check autostart enable

$ systemctl is-enabled APPNAME

Enable autostart

$ systemctl enable APPNAME

Disable autostart

$ systemctl disable APPNAME

linux : user management and permission

Zaki Arrozi Arsyad — Thu, 11 Jun 2020 08:01:23 +0000

This is one of the best part of linux, we can grant access for all services, directories, files, and even commands that can be executed for a specific user or group.

User management

Check who am i

$ whoami

Check the user id

$ id USERNAME

Check the group of user

$ groups USERNAME

Check who is currently logged in

$ w

Check the last logged in

$ last

Change user

$ su

Add a new user

$ useradd USERNAME

Set password for a user

$ passwd PASSWORD

Change user name

$ usermod -l USERNAME NEW_USERNAME

Add a user to a group

$ usermod -a -G GROUPNAME USERNAME

Change user primary group

$ usermod -g GROUPNAME USERNAME

Remove user from a group

$ gpasswd -d USERNAME GROUPNAME

Delete a user

$ userdel -r USERNAME

Lock a user

$ passwd -l USERNAME

Unlock a user

$ passwd -u USERNAME

Group management

Add a new group

$ groupadd GROUPNAME

Change group name

$ groupmod -n GROUPNAME NEW_GROUPNAME

Delete a user

$ groupdel GROUPNAME

Permission

After create a user or group, we can grant access for specific user or group.

Permission format :

rwxrwxrwx

First 3 characters are the user permission, next 3 characters are the group permission, and the last 3 are other permission.

r is for read with value 4
w is for write with value 2
x is for execute with value 1
- is for no permission with value 0

We can also use the value of the characters to define the permission. Simply sum the values of the permissions for each role, and use only 3 numbers to define the permission.

# rwxrwxrwx
777

# rwxr-x-rx
755

# rwx------
700

We can see the files or directories permission by run ls -l.

$ ls -la
drwxrwxrwx 1 root root  96 Jun 11 07:58 directory
-rwxr-xr-x 1 zaki zaki  96 Jun 11 07:58 file.txt

The first column is the permission, the third column is the user, the fourth column is the group name.

linux : rsync vs scp

Zaki Arrozi Arsyad — Thu, 11 Jun 2020 03:13:05 +0000

rsync is a command for move and synchronizing files. This command help us to manage files or directories effectively when backing up data on a regular basis. rsync compares any differences and only transfer those differences.

scp is used for copy files and directories in secure way. Basically, It is a plain linear copy. scp reads the source file and writes it to the destination.

`rsync`

Basic usage :

$ rsync OPTION SOURCE DESTINATION

We need to check if we have rsync, because some machines might not have this command.

Not passing the OPTION

It will sync between two directories, but only files in the main directory, not include files in the subdirectories.

# Sync all files in original directory to duplicate directory
$ rsync SOURCE/* DESTINATION/

-r or -recursive

Sync all files in all the main directory and subdirectories

# Sync all files in original directory and its subdirectories
$ rsync -r SOURCE/ DESTINATION/

-a or -archive

This option allow us to copy almost everything, including file permissions, user & group ownership and timestamps.
-v or -verbose

It gives us a summary about the data transferred.
-h or --human-readable format

We will get the result in human readable format.
-z or -compress

Compress file data during the transfer
-e

To specify the type of protocol we use. We can choose using ssh or rsh

# Sync using ssh protocol
$ rsync -avhze ssh SOURCE/ DESTINATION/

--progress

To show the progress while transferring the data

# Show the progress during transfer
$ rsync -avhze ssh --progress SOURCE/ DESTINATION/

--update

To updated more recently on the local filesystem. Files that don't exist are copied. Files that already exist, but have a newer timestamp are also copied.

# Update only if there is a newer version
$ rsync -avhze ssh --progress --update SOURCE/ DESTINATION/

--remove-source-files

To remove files that successfully transferred.

# Sync using ssh protocol
$ rsync -avhze ssh --remove-source-files SOURCE/ DESTINATION/

--delete

Delete files that have been deleted in the original directory

# Update duplicate directory if there are files deleted in original directory
$ rsync -avhze ssh --delete SOURCE/ DESTINATION/

--include

This option will include files that we specify in the parameter

# Include files that matched with include paramaeter
$ rsync -avhze ssh --include 'KEYWORD' SOURCE/ DESTINATION/

--exclude

This option will exclude files that we specify in the parameter

# Exclude files that matched with exclude paramaeter
$ rsync -avhze ssh --exclude 'KEYWORD' SOURCE/ DESTINATION/

--chown

Change file permission in the destination directory

# Copied files permisson are changed form user to group
$ rsync -avhe ssh --chown=USER:GROUP SOURCE/ DESTINATION/

--dry-run

This option perform a trial run and will not make any changes, but gives us the same result as a real run. If the results are as expected, then we can remove the --dry-run

# Perfom dry run
$ rsync --dry-run -avhze ssh --delete SOURCE/ DESTINATION/

--max-size

Set the maximum files to transfer. This option will only transfer files that are equal or smaller than the max size.

# Set max size 200k
$ rsync -avhze ssh --max-size='200k' SOURCE/ DESTINATION/

--bwlimit

We can set the bandwidth limit when we sync data from a machine to another machine.

# Set bandwidth rate 5000KBytes per second
$ rsync -avhze ssh --bwlimit=5000 SOURCE/ DESTINATION/

`scp`

Basic usage :

$ scp OPTION SOURCE DESTINATION

Copy a single file

# Copy a single file
$ scp SOURCE_FILE.txt DESTINATION/

Copy multiple files

# Copy a multiple file
$ scp FILE1.txt FILE2.txt FILE3.txt DESTINATION/

Copy .pem file

# Copy aws key to vm 
$ scp -i file.pem file.pem DESTINATION/

-r to copy a directory recursively

# Copy all files in a directory
$ scp -r SOURCE/ DESTINATION/

-P to specify the port

By default, scp will use port 22, but we can use another port for security reason

# Use port 2249
$ scp -P 2249 SOURCE/ DESTINATION/

-v for verbose

To show detailed information of the scp process

# Use port 2249
$ scp -v SOURCE/ DESTINATION/

-l limit the bandwidth usage

# Set the bandwidth limit
$ scp -l 400 SOURCE/ DESTINATION/

-q for quite mode

Run the command in quite mode, and only display critical errors
-p for preserves modification times, access times, and modes from the original file
-u to delete source files after the transfer is successful

Both of the commands are similar, lets choose what we use based on our case

linux : find

Zaki Arrozi Arsyad — Wed, 10 Jun 2020 02:26:44 +0000

We can search files or directories in linux using find

Basic usage :

find DIRECTORY OPTION WHAT_TO_FIND

DIRECTORY :

We can specify where we want to run the find command.

By default, if we don't specify the directory it will call the current directory ./

# find FILE.txt in home directory
$ find /home -name "FILE.txt"

# find all .txt files in the current directory
$ find -name "*.txt"

OPTIONS :

-type f : find files

# find matched files
$ find -type f -name "FILE.txt"

by default find is finding files. we can also use -type f for file type

-type d : find a directories

# find matched directories
$ find -type d -name "DIRECTORY"

-iname : find with case insensitive

# find a FILE.txt or file.txt
$ find -iname "FILE.txt"

-print : display the path name

We can use -print option or not with the sam result.

# find files
$ find -name "FILE.txt" -print

# find directories
$ find -type d -name "DIRECTORY"

-size : find by size

# find files smaller than 10MB
$ find -size -10M

# find files larger than 10MB
$ find -size +10M

# find files exactly 3 10MB
$ find -size 10M

# find files between 10MB and 1GB
$ find -size +10M -size -1G

-empty or -size 0 : find empty files or directories

# find an empty files
$ find -type f -empty

# find an empty directories
$ find -type d -size 0

-exec CMD : execute a command after

# execute grep after find is done
$  find -name "FILE.txt" -exec grep "KEYWORD" {} \;

-delete or -exec rm {} \; : delete matched files or directories

# delete an mathed files
$ find -name "*.bak" -delete

# delete an empty directories
$ find -type d -size 0 -exec rmdir {} \;

not or ! : exclude the matched files or directories

# find all except the matched files
$ find -not -name "FILE.txt"

-mtime : find by modified time

we also can use -mmin if we want to specify the time interval to minutes

# find files modified within the last 3 days
$ find -name "FILE.txt" -mtime -3

# find files modified more than 3 days
$ find -name "FILE.txt" -mtime +3

# find files modified exactly 3 days
$ find -name "FILE.txt" -mtime 3

# find files modified between 3 and 10 days
$ find -name "FILE.txt" -mtime +3 -mtime -10

-atime : find by accessed time

we also can use -amin if we want to specify the time interval to minutes

# find files accessed within the last 3 days
$ find -name "FILE.txt" -atime -3

# find files accessed more than 3 days
$ find -name "FILE.txt" -atime +3

# find files accessed exactly 3 days
$ find -name "FILE.txt" -atime 3

# find files accessed between 3 and 10 days
$ find -name "FILE.txt" -atime +3 -atime -10

-ctime : find by changed time

we also can use -cmin if we want to specify the time interval to minutes

# find files changed within the last 3 days
$ find -name "FILE.txt" -atime -3

# find files changed more than 3 days
$ find -name "FILE.txt" -atime +3

# find files changed exactly 3 days
$ find -name "FILE.txt" -atime 3

# find files changed between 3 and 10 days
$ find -name "FILE.txt" -atime +3 -atime -10

-perm : find by permission

# find files with permission 777
$ find -perm 777

-user : find by user access

# find matched files with user root
$ find -user root -name "FILE.txt"

-group : find by group access

# find matched files with group developer
$ find -group developer -name "FILE.txt"

find hidden files

$ find -name ".*"

linux : archiving & compression

Zaki Arrozi Arsyad — Sat, 30 May 2020 17:12:09 +0000

Archiving is the process of combining multiple files and directories (same or different sizes) into one file. Compression is the process of reducing the size of a file or directory.

Common type of compression:

Gzip : with extension .tar.gz. It has fastest compression time.
Bzip : with extension .tar.bz2
Xz : with extension .tar.xz. It has smallest size compression.

Common programs for archiving files:

tar
zip
gzip
bzip2
xz

We will talk about how we use tar command. It has a lot of option that we can use:

-c or --create : create a new file archive
-f or --file : filename

# create an archive from multiple files
tar -cf ARCHIVE.tar FILE1.txt FILE2.txt FILE3.txt

# create an archive from a directory
tar -cf ARCHIVE.tar DIRECTORY

# using long command
tar --create --file ARCHIVE.tar DIRECTORY

-X or --exclude : exclude a file

# exclude a file when creating an archive
tar -cf ARCHIVE.tar DIRECTORY --exclude=FILE.txt

-x or --extract : extract an archive

# extract an archive file
tar -xf ARCHIVE.tar

# using long command
tar --extract --file ARCHIVE.tar

-z : zip or unzip using gzip

create a compressed archive

# create an archive with gzip compression
tar -czf ARCHIVE.tar.gz DIRECTORY

# using gzip
gzip ARCHIVE.tar.gz DIRECTORY

extract a compressed archive

# extract an archive with gzip compression
tar -xzf ARCHIVE.tar.gz

# using gunzip --decompress
gunzip -d ARCHIVE.tar.gz

-j : zip or unzip using bzip2

create a compressed archive

# create an archive with bzip compression
tar -cjf ARCHIVE.tar.bz2 DIRECTORY

# using bzip2
bzip2 ARCHIVE.tar.bz2 DIRECTORY

extract a compressed archive

# extract an archive with bzip compression
tar -xjf ARCHIVE.tar.bz2

# using bunzip2 --decompress
bunzip2 -d ARCHIVE.tar.bz2

-J : zip or unzip using xz

create a compressed archive

# create an archive with xz compression
tar -cJf ARCHIVE.tar.xz DIRECTORY

# using xz
xz ARCHIVE.tar.xz DIRECTORY

extract a compressed archive

# extract an archive with xz compression
tar -xJf ARCHIVE.tar.xz

# using unxz --decompress
unxz -d ARCHIVE.tar.xz

zip and unzip

create a compressed archive

# create using zip
zip ARCHIVE.zip DIRECTORY

extract a compressed archive

# extract using unzip
unzip ARCHIVE.zip

-t or --list : list the contents of an archive without extracting it

# display contents of an archive
tar -tf ARCHIVE.tar

# display contents using gzip
gzip -l ARCHIVE.tar.gz

# display contents using unzip
unzip -l ARCHIVE.zip

-r or --append : append files to the end of an archive

# append files to an archive
tar -rf ARCHIVE.tar FILE.txt

# using --append
tar --append -file ARCHIVE.tar FILE.txt

-u or --update : Append files which are newer than the corresponding copy in the archive

# append files to an archive
tar -uf ARCHIVE.tar FILE.txt

# using --append
tar --update -file ARCHIVE.tar FILE.txt

delete : delete files inside an archive

# delete from the archive
tar --delete --file ARCHIVE.tar FILE.txt

-C or --directory : change the directory

# change location of unarchived files
tar -xf ARCHIVE.tar -C DIRECTORY

-v or --verbose : see the detailed output
We can combine this options with all others
--wildcards : matching on the filenames within an archive

# extract a particular type of file
tar -xf ARCHIVE.tar --wildcards "*.txt"

linux : regex

Zaki Arrozi Arsyad — Wed, 27 May 2020 16:30:31 +0000

Regex or regular expression is a text pattern. Basically we use regex for helping us search, update, or manage text. We can combine regex with grep, sed, or awk command.

Basic usage :

grep -E REGEX_PATTERN FILE.txt

We'll talk about how we can create a regex pattern. Thing to note, regex is case sensitive. I will use grep for simple examples. It will display text with matched pattern.

1. Basic matcing

. is search for anything, or replace . with any character

# matched with anything
grep -E . FILE.txt

# matched with 'linux' or 'lanux' or etc
grep -E l.nux FILE.txt

\d is for digit in 0 - 9

# matched with all digits
grep -E '\d' FILE.txt

\D is for non-digit

# matched with all non digit
grep -E '\D' FILE.txt

\w is for word ( letter, digit, and _ )

# matched with all word
grep -E '\w' FILE.txt

is for space

# matched with space
grep -E ' ' FILE.txt

is for space

# matched with space
grep -E ' ' FILE.txt

\t is for tab

# matched with tab
grep -E '\t' FILE.txt

\r is for return

# matched with return
grep -E '\r' FILE.txt

\n is for new line

# matched with new line
grep -E '\n' FILE.txt

\s is for whitespace

# matched with whitespace. Include space, tab, return, and new line
grep -E '\s' FILE.txt

\S is for non whitespace

# matched with non whitespace
grep -E '\S' FILE.txt

2. Classes

[ ] -> matched any character inside the square bracket.

[linux] matched anything inside the square bracket

# matched with any characters inside the square bracket
grep -E '[linux]' FILE.txt

# matched with linux and xinux
grep -E '[lx]inux' FILE.txt

[^linux] matched any characters except characters inside the square bracket

# matched with any characters except l, i, n, u, and x
grep -E '[^linux]' FILE.txt

[a-z] matched anything in the range of characters

# matched with all lowercase
grep -E '[a-z]' FILE.txt

# matched with all uppercase
grep -E '[A-Z]' FILE.txt

# matched with all digit
grep -E '[0-9]' FILE.txt

3. Boundaries

\b is for boundaries
\B is for non boundaries
^ is for the beginning of the line

# matched with anything started with l
grep -E '^l' FILE.txt

$ is for the end of the line

# matched with anything ended with x
grep -E '^x' FILE.txt

4. Disjunction

| is for or

# matched with linux and unix
grep -E 'linux|unix' FILE\.txt

5. Quantifier

* is for zero or more repetition

# matched with lnux, linux, liinux, etc
grep -E 'li*nux' FILE.txt

+ is for one or more repetition

# matched with linux, liinux, etc, but doesn't matched with lnux
grep -E 'li+nux' FILE.txt

? is for zero or one instances

# matched with lnux and linux
grep -E 'li?nux' FILE.txt

{n} is for exactly n instances

# matched with linuxlinuxlinux
grep -E '(linux){3}' FILE.txt

# matched with linuxxx
grep -E 'linux{3}' FILE.txt

{n,} is for at least n instances

# matched with linuxxx with 3 or more x
grep -E 'linux{3,}' FILE.txt

{m,n} is for between m and n instances

# matched with linuxx with 2 until 4 x
grep -E 'linux{2,4}' FILE.txt

By default, quantifier are greedy. example for word linuxlinuxlinux

greedy

# matched with all stacko
print stackoverflow | grep -E 's.*o'

lazy

# matched with all stackoverflo
print stackoverflow | grep -E 's.*?o'

6. Special characters

{ } [ ] ( ) ^ $ . | * + ? \ - inside the square bracket must be escaped with \

# matched with period
grep -E '[\.]' FILE.txt

# matched with backslash
grep -E '[\\]' FILE.txt

7. Given two criteria

we can combine 2 or more pattern

# matched with text beginning with a digit and have linux word
grep -E '^[1-9]' FILE.txt | grep -E 'linux'