DEV Community: Taleeb Midi

Change Directory Up Faster With Back up Command

Taleeb Midi — Mon, 14 May 2018 00:31:14 +0000

Donovan Brown wrote a good article showcasing how you can use a custom PowerShell function to navigate up directories without the need to type multiple cd .. . I find his idea interesting and could be of important time saver if you spend a lot of time working with PowerShell.

Unfortunately (or fortunately depends how you see it) nowadays I don't use PowerShell a lot, and when I do my usage is limited to few PowerCLI commands, but I do spend a considerable time interacting with Linux Shell, so the natural thing to do is to port Donovan's idea into a Bash script. The Bash script is going to be different from the PowerShell one, but the concept remains the same.

What we need?

A function that take an integer as an argument.
A help function to display a basic help menu when wrong argument entered.

The help function:

I like to keep this as simple as possible, since our function will take only one line for the argument and the second line to indicate that this a help menu:

    function bu_usage () {
       cat <<-EOF
        Usage: bu [N]
                N        Where N is the number of level to move back to. this argument must be an integer.
                h help   displays this basic help menu.
        EOF
    }

I used -EOF to allow indentation, this work better with tabs than with spaces.

The back up function

Now that our help function is out of the way we can start building the backup function. we know the function will take one argument, and this argument must be an integer:

function bu () {
    FUNCTIONARG=$1
    # Make sure the provided argument is a positive integer:
    if [[ ! -z "${FUNCTIONARG##*[!0-9]*}" ]]; then
        for i in $(seq 1 $FUNCTIONARG); do
            STRARGMNT+="../"
        done
        CMD="cd ${STRARGMNT}"
        eval $CMD
    else
        bu_usage
    fi
}

How this works? the functions starts first by making sure the argument is a valid integer, if not it will call the help function we created earlier. Then using a sequence of numbers from 1 to the argument, we will append ../ string to STRARGMNT with each iteration. When the for loop is complete we run cd command with all the final appended up directories. This will give us:

argument	command
1	cd ..
2	cd ../..
3	cd ../../..

How to use this?

I usually add functions like these to a .functions file under my home directory, the function file get sources by .bash_profile.
To source .functions or other dotfiles add this for loop to your .bash_profile, files must be comma separated:

for file in ~/.{functions}; do
    [ -r "$file" \] && \[ -f "$file" \] && source "$file";
done;
unset file;

Create .functions and add this content to it:

 function bu () {
    function bu_usage () {
       cat <<-EOF
        Usage: bu [N]
                N        Where N is the number of level to move back to. this argument must be an integer.
                h help   displays this basic help menu.
        EOF
    }


    # unset variables
    STRARGMNT=""
    FUNCTIONARG=$1

    # Make sure the provided argument is a positive integer:
    if [[ ! -z "${FUNCTIONARG##*[!0-9]*}" ]]; then
        for i in $(seq 1 $FUNCTIONARG); do
            STRARGMNT+="../"
        done
        CMD="cd ${STRARGMNT}"
        eval $CMD
    else
        bu_usage
    fi
}

This is slightly different from the previous functions we created. The help menu is a nested function inside the main bu function. we also created empty STRARGMNT to unset the variable each time the function runs.

When you are done source .bash_profile or close and reopen your terminal for the change to take effect

how to run?
bu help or bu -h for help menu.
bu 2 to go two folders up, this is the equivalent of running cd ..\..\
you can see bu in action here(external link):

Getting started with HAProxy: Install from code source

Taleeb Midi — Mon, 14 May 2018 00:24:27 +0000

intro

In this post, which is taken from my personal blog, I will demonstrate how we can install HAProxy from source code. According to Wikipedia, HAProxy was written and still maintained by Willy Tarreau since 2000. HAProxy is free open source software (FOSS), that provides a high availability load balancer and proxy server for TCP (Transmission Control Protocol) and HTTP (Hypertext Transfer Protocol) based applications that spreads requests across multiple servers. It is written in C language and thus the reputation for being fast and efficient.

If you are curious, this HAProxy Page offers more details about its features.

Installation

In this guide , we will be installing the latest stable version of 1.8 (as of May 09, 2018). the current HAProxy available under RHEL is 1.5.18 and 1.8.8 on Debian/Ubuntu. before taking the next step, make sure you have gcc, pcre-static and pcre-devel installed:

# CentOS RHEL:
sudo yum -y install make gcc perl pcre-devel zlib-devel

# Debian/Ubuntu:
sudo apt install make gcc perl pcre-devel zlib-devel

Enable IP forwarding and binding to non-local IP addresses:

# echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
# echo "net.ipv4.ip_nonlocal_bind = 1" >> /etc/sysctl.conf
# sysctl -p
net.ipv4.ip_forward = 1
net.ipv4.ip_nonlocal_bind = 1

Download the source code:

wget https://www.haproxy.org/download/1.8/src/haproxy-1.8.8.tar.gz

Extract the file and change directory:

tar xvzf haproxy-1.8.8.tar.gz && cd haproxy-1.8.8

Compile the program:

make TARGET=linux2628 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1

This is the list of TARGETs:

linux22 for Linux 2.2
linux24 for Linux 2.4 and above (default)
linux24e for Linux 2.4 with support for a working epoll (> 0.21)
linux26 for Linux 2.6 and above
linux2628 for Linux 2.6.28, 3.x, and above (enables splice and tproxy)
solaris for Solaris 8 or 10 (others untested)
freebsd for FreeBSD 5 to 10 (others untested)
netbsd for NetBSD
osx for Mac OS/X
openbsd for OpenBSD 5.7 and above
aix51 for AIX 5.1
aix52 for AIX 5.2
cygwin for Cygwin
haiku for Haiku
generic for any other OS or version.
custom to manually adjust every setting

You can also compile the program with following options:
USE_PCRE=1 to use libpcre, in whatever form is available on your system (shared or static).
USE_ZLIP=1 to use zlib compression Library.
USE_OPENSSL=1 to add native support for SSL using the GNU makefile.

Install HAProxy:

sudo make install

With that, we should now have the chosen HAProxy version installed. now we need to setup HAProxy.

Setup:

Adding HAProxy user:

id -u haproxy &> /dev/null || useradd -s /usr/sbin/nologin -r haproxy

Copy HAProxy binary located in the extracted HAProxy directory to /usr/sbin/:

cp haproxy /usr/sbin/

Create Manual page:

wget -qO - https://raw.githubusercontent.com/horms/haproxy/master/doc/configuration.txt | gzip -c > /usr/share/doc/haproxy/configuration.txt.gz

Create Systemd service script to manage HAProxy, use the following at your discretion:

sudo wget https://gist.githubusercontent.com/tmidi/1699a358533ae876513e2887fec6fbe2/raw/6c07ce39adc56c731d2bbeb88b90d8bbc636f3ea/haproxy.service -O /lib/systemd/system/haproxy.service

or create /lib/systemd/system/haproxy.service with the following content:

[Unit]
Description=HAProxy Load Balancer
Documentation=man:haproxy(1)
Documentation=file:/usr/share/doc/haproxy/configuration.txt.gz
# allows us to do millisecond level restarts without triggering alert in Systemd
StartLimitInterval=0
StartLimitBurst=0
After=network.target syslog.service
Wants=syslog.service

[Service]
Environment="CONFIG=/etc/haproxy/haproxy.cfg" "PIDFILE=/run/haproxy.pid"
# EXTRAOPTS and RELOADOPS come from this default file
EnvironmentFile=-/etc/default/haproxy
ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q
ExecStart=/usr/sbin/haproxy -W -f $CONFIG -p $PIDFILE $EXTRAOPTS
ExecReload=/usr/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS $RELOADOPTS
ExecReload=/bin/kill -USR2 $MAINPID
KillMode=mixed
Restart=always
Type=forking

[Install]
WantedBy=multi-user.target

Enable and start haproxy.service:

systemctl enablle haproxy.service

systemctl start haproxy.service

Conclusion

This conclude conclude HAProxy installation steps.This guide however covers only the installation steps, in future post I will demonstrate how to configure HAProxy to load balance three backend web servers while using sticky session and SSL Pass-through.

If you find this guide helpful, please share it with your friends and colleagues. I am on Twitter and LinkedIn, let's connect or let me know if you have any question or any suggestions.

Ansible: Directory Layout

Taleeb Midi — Fri, 16 Feb 2018 04:48:26 +0000

According to Ansible's Best Practices, There are many possible ways to organize playbook content, and that the usage of such layout should fit your needs. The only thing I highly recommend is using roles instead of tasks, this will give your flexibility and better organization of your code.

Ansible provide two examples of directory layouts. the first one is pretty simple and the one I go to when I am working on a small environement with a sample production and staging inventory files:

production                # inventory file for production servers
staging                   # inventory file for staging environment

group_vars/
   group1                 # here we assign variables to particular groups
   group2                 # ""
host_vars/
   hostname1              # if systems need specific variables, put them here
   hostname2              # ""

library/                  # if any custom modules, put them here (optional)
module_utils/             # if any custom module_utils to support modules, put them here (optional)
filter_plugins/           # if any custom filter plugins, put them here (optional)

site.yml                  # master playbook
webservers.yml            # playbook for webserver tier
dbservers.yml             # playbook for dbserver tier

roles/
    common/               # this hierarchy represents a "role"
        tasks/            #
            main.yml      #  <-- tasks file can include smaller files if warranted
        handlers/         #
            main.yml      #  <-- handlers file
        templates/        #  <-- files for use with the template resource
            ntp.conf.j2   #  <------- templates end in .j2
        files/            #
            bar.txt       #  <-- files for use with the copy resource
            foo.sh        #  <-- script files for use with the script resource
        vars/             #
            main.yml      #  <-- variables associated with this role
        defaults/         #
            main.yml      #  <-- default lower priority variables for this role
        meta/             #
            main.yml      #  <-- role dependencies
        library/          # roles can also include custom modules
        module_utils/     # roles can also include custom module_utils
        lookup_plugins/   # or other types of plugins, like lookup in this case

    webtier/              # same kind of structure as "common" was above, done for the webtier role
    monitoring/           # ""
    fooapp/               # ""

I always use the following command to create the above directory structure and get started as soon as possible:

  
mkdir -p group_vars host_vars library module_utils filter_plugins
mkdir -p roles/common/{tasks,handlers,templates,files,vars,defaults,meta,library,module_utils,lookup_plugins}
touch production staging site.yml roles/common/{tasks,handlers,templates,files,vars,defaults,meta}/main.yml

When I have a more complex inventory, with multiple groups and children, I opt for this alternative directory layout:

inventories/
   production/
      hosts               # inventory file for production servers
      group_vars/
         group1           # here we assign variables to particular groups
         group2           # ""
      host_vars/
         hostname1        # if systems need specific variables, put them here
         hostname2        # ""

   staging/
      hosts               # inventory file for staging environment
      group_vars/
         group1           # here we assign variables to particular groups
         group2           # ""
      host_vars/
         stagehost1       # if systems need specific variables, put them here
         stagehost2       # ""

library/
module_utils/
filter_plugins/

site.yml
webservers.yml
dbservers.yml

roles/
    common/               # this hierarchy represents a "role"
        tasks/            #
            main.yml      #  <-- tasks file can include smaller files if warranted
        handlers/         #
            main.yml      #  <-- handlers file
        templates/        #  <-- files for use with the template resource
            ntp.conf.j2   #  <------- templates end in .j2
        files/            #
            bar.txt       #  <-- files for use with the copy resource
            foo.sh        #  <-- script files for use with the script resource
        vars/             #
            main.yml      #  <-- variables associated with this role
        defaults/         #
            main.yml      #  <-- default lower priority variables for this role
        meta/             #
            main.yml      #  <-- role dependencies
        library/          # roles can also include custom modules
        module_utils/     # roles can also include custom module_utils
        lookup_plugins/   # or other types of plugins, like lookup in this case

With this structure, I can put each inventory file with its group_vars\host_vars in separate directory. To quickly spin up this directory layout, I use the following commands:

  
mkdir -p inventories/{production,staging}/{group_vars,host_vars}
touch inventories/{production,staging}/hosts
mkdir -p group_vars host_vars library module_utils filter_plugins
mkdir -p roles/common/{tasks,handlers,templates,files,vars,defaults,meta,library,module_utils,lookup_plugins}
touch site.yml roles/common/{tasks,handlers,templates,files,vars,defaults,meta}/main.yml

I usually don't do any work on roles/common, to create a new role I just duplicate common to a new directory, this way I have a role template that I can use whenever I want to create a new role.