DEV Community: Aniket Pal

Shell Scripting For DevOps: Quick Beginner's Guide 💎

Aniket Pal — Mon, 18 Dec 2023 14:45:11 +0000

We all have fears – water, heights, weird smells, you name it. Me? I've got a special fear called UI change phobia. It's like my internet journey hits a speed bump every time software decides to change the UI.

Take Twitter, for example. Imagine the "Tweet" button doing a daily dance, from the right bottom to the top left. Sounds like a game, right? Now, picture this happening not just on social media but also in complicated places like the AWS Dashboard. Not so fun now, is it?

If you are someone who enjoys watching, I am curating short videos for the entire blog. If you don't want to miss out, make sure to subscribe.

Getting back to the blog. See, for the expert's UIUX Researchers' layouts, it's like playing interior designer for the internet. Changing the layout gives them analytics and hence a way to curate the best. They ultimately want to make things comfy for us in the long run. But guess who ends up feeling lost in the meantime? Yep, us!

Personally, it's exhausting to keep up with these changes. Imagine having 10 servers to check every day, and each time you press what feels like a gazillion buttons to find the logs. And just when you've got there you realise the product team has shifted the logs options to test their logic map. It's like a bad game of hide-and-seek, and I wouldn't blame anyone for getting a bit frustrated.

And, this is where the magic of Shell Scripting swoops in. Commands barely do the cha-cha, and even if they decide to change, there's a golden rule – it's well-documented, and the logic stays put. Enter the superhero of consistency: your very own Shell Script 🔥

Image Credits: Wallpaper Flare

Imagine this: every morning, you wake up, grab your cup of coffee, and you run a simple script. You toss in the resource name, and voilà, like clockwork, the script fetches the logs you desire. It's like having a reliable assistant that doesn't flinch at button shuffles.

I'm a firm believer in DRY (Don't Repeat Yourself), and you should be too! Why press a gazillion buttons when a simple script can do the heavy lifting? So, let's embrace the power of Shell Scripting, where the only dance moves are the ones you decide. After all, in the ever-changing UI landscape, a bit of scripting stability goes a long way.

You are good if you understand the need for Shell Scripting and are motivated enough to learn 🚀✨

Types of Shells 🐚

Shells serve as crucial interfaces between users and the operating system kernel, enabling command-line communication. It acts as an intermediary, interpreting user commands and translating them into instructions that the kernel can execute.

Steve Bourne aka Father of ShellImage Credits: Faces of OpenSource

Shells can broadly be classified into 5 main categories, each designed for specific tasks and preferences. We will understand a bit about them and then move on with the commands.

1️⃣ Bourne Shell (sh)

Steve Bourne is credited with developing the inaugural UNIX shell known as sh in 1979. Renowned for its swift operation, sh gained popularity for its speed. However, being the first shell, it lacks convenient features like recalling previous commands with an up arrow or handling logical and arithmetic operations.

2️⃣ C Shell (csh)

Developed by Bill Joy at the University of California, the C shell, denoted as csh, stands out for its programming-oriented features. It includes built-in support for arithmetic operations and adopts a syntax reminiscent of the C programming language.

Addressing a gap in earlier Linux shells, such as the Bourne shell, the C shell introduced command history, allowing users to track and reuse previous commands efficiently. It gained popularity for the incorporation of "aliases," providing a flexible and customizable way to define shortcuts for frequently used commands.

3️⃣ Korn Shell (ksh)

Crafted by David Korn as an enhancement to the traditional sh the Korn shell, denoted as ksh, stands as a superset of sh. It incorporates in-built support for arithmetic operations while introducing interactive features akin to the C shell.

A notable feature of ksh is its versatility—it seamlessly executes scripts written for both sh and csh. Beyond this compatibility, ksh introduces advanced functionalities such as string, array, and function manipulation, drawing parallels to the expressive capabilities of the C programming language.

ksh gained widespread acclaim for its speed, being recognized as the fastest interpreter during its initial release. This combination of compatibility and advanced features contributed to Ksh's popularity in the Unix and Linux communities.

4️⃣ GNU Bourne Again Shell (bash)

Widely known as Bash, the GNU Bourne-Again shell was crafted for compatibility with the Bourne shell. It incorporates features from various Linux shells, such as the Korn shell and C shell.

Bash stands out by automating the recall of previous commands and allowing easy editing with arrow keys—an enhancement over the Bourne shell, providing a more user-friendly and efficient command-line experience.

5️⃣ Z Shell (zsh)

Zsh, the default on MacOS, is a cutting-edge shell known for pioneering features. It introduced command completion, allowing users to swiftly complete commands with the Tab key. Dynamic filename generation, based on specified conditions, enhances file management efficiency. The creators also championed plugin support, providing a flexible architecture for users to customize their shell environment. This adaptability has made Zsh one of the most feature-rich and user-friendly shells available.

Completing the historical journey through the development of shells provides valuable insights into the evolution of these crucial components. For instance, understanding the progression from the initial Bourne shell to the advanced features of shells like Zsh sheds light on the iterative improvements and innovations. Let's quickly check out the Shell and corresponding path.

Shell	Path
sh	/bin/sh
csh	/bin/csh
ksh	/bin/ksh
bash	/bin/bash
zsh	/bin/zsh

Having covered the major shells in use, our focus now turns to Zsh and Bash. These two shells take center stage in modern software development and automation lifecycles, making them pivotal tools in contemporary computing. Before we dive in,

Bash and Zsh are Interpreters

Incase, you are confused between Interpreters and compilers. An interpreter reads and executes code line by line, providing immediate feedback on errors. On the other hand, a compiler translates the entire code before execution, creating a standalone executable file, which can be likened to translating an entire book before reading it. Let's start with learning basic commands.

UNIX Starters 🏁

Even if you don't want to automate or configure corn jobs, these commands will help you work around files and everyday UI tasks. Well, incase you have never seen a terminal,

Currently, I'm on MacOS, utilizing Zsh. Let's dive back into learning commands. Instead of lengthy descriptions, I'll present them in a table for clarity. As we delve deeper, you'll gain a better understanding of each command and its functionalities.

Command	Description	Example
`ls`	Lists files and directories in the current location.	`ls` or `ls -l` for detailed listing
`cd`	Changes the current directory.	`cd Documents`
`pwd`	Displays the present working directory.	`pwd`
`cp`	Copies files or directories.	`cp file.txt destination/`
`mv`	Moves or renames files and directories.	`mv file.txt newfile.txt`
`rm`	Removes/deletes files or directories.	`rm file.txt`
`mkdir`	Creates a new directory.	`mkdir new_directory`
`rmdir`	Removes an empty directory.	`rmdir empty_directory`
`cat`	Concatenates and displays the content of files.	`cat file.txt`
`grep`	Searches for a pattern in files.	`grep pattern file.txt`
`chmod`	Changes file permissions.	`chmod +x script.sh`
`ps`	Displays information about active processes.	`ps aux`
`kill`	Sends a signal to terminate a process.	`kill -9 process_id`
`ssh`	Connects to a remote server securely.	`ssh username@remote_host`
`echo`	Displays text or variables.	`echo "Hello, World!"`

Congrats on mastering some commands! But hey, the adventure continues. If things are a bit fuzzy, no worries – just give those commands a spin. Now, let's chat about two command champs that tend to stump folks: chmod and grep.

Change Mode 🎨

As a system administrator, one of the most important tasks is to control access management. Let's say, you create a file expenses.txt, you don't want everyone to change the content of the file so you give the needed group read access. But how to do that? Here enters chmod.

The chmod command is a powerful tool in UNIX for changing file permissions. It allows users to control access to their files by specifying who can read, write, and execute them. In UNIX, permissions are represented by three categories: owner(u), group(g), and others(o), for every group we can apply a certain permission👇

Permission	Symbol	Explanation
Read	`-r`	Allows reading the file.
Write	`-w`	Permits modifying the file.
Execute	`-x`	Grants the ability to execute.

Let's write the command for the above expenses.txt file, wherein we would give the owner the read and write access and group the access of read.



chmod u=rw,g=r expenses.txt



chmod u+rw g+r expenses.txt

After running any of the commands the file access would be modified. To change it further, refer to the table above and run the command with the needed flag and parameters.

Global Regular Expression Print 🗂️

One of the most difficult tasks is to search from the UI. The grep command is a powerful text-search utility that allows users to search for specific patterns or expressions within files. It's incredibly handy for sifting through large amounts of text to find relevant information. The basic syntax is:



grep [options] pattern [files...]

pattern: The text or regular expression we want to search for.
file: The file or files we want to search within.

Let's use grep command to find any word that ends it salary from the expenses.txt file. Such that it searches for developerSalary, managerSalary, internSalary and so on.



grep "Salary$" expenses.txt

The regex symbol $ indicates the end of the line. So, "Salary$" implies a search for every term that ends with Salary. Well, you can use an array of regex symbols and expressions to make your search more efficient. Incase, you don't know much about regex and want me to explain do let me know in the comments.

Variables in Shell 🔡

In UNIX, variables are like containers for information. They make scripts clearer by giving meaningful names to values, allowing data reuse, and helping handle dynamic information, like user input or command results. Variables also play a role in passing information between different parts of a script.

Before we start writing scripts let's get our fundamentals clear with writing variables in the command line itself.



greetings="Hello, World" && echo "$greetings"

Over here, we store the string Hello World in the variable named greetings, and then with the help of the echo command we print it. echo $greetings will also work fine but adding quotation marks " prevents possible spaces in the variable causing havoc in data. So, as a good practice remember to always add quotes in the variable.

If you have worked with any object-oriented programming language, I am pretty sure you understand the importance of scope. In simple terms scope of a variable defines it's effective boundaries. In Shell Scope is defined into 3 main categories.

1️⃣ Local Scope: Local variables are specific to the current shell session or script. They have a limited scope and are not visible to external commands or scripts. Example to make the meaning more clear,



EDITOR=vim
crontab -e

In the particular shell session whenever we run crontab -e, it will run the vim editor. When we exit the shell session it will open the default editor.

2️⃣ Environment Scope: Environment variables are accessible to any child process spawned from the current shell. They provide a way to share information between the shell and its subprocesses.



export EDITOR=vim
crontab -e

When we apply the command export it is considered an environment variable. Whenever we spin up crontab -e it will open Vim as the editor. Environment Variables are used for setting values that lives for multiple sessions unless manually removed. For example, whenever we set path we use environment variable

3️⃣ Command Scope: Available for one instance of command, once execution is done instance doesn't persist.



EDITOR=nano crontab -e

Over here, the EDITOR variable is limited to the command. Even if we run crontab in the current shell session, it would open the default editor.

The three variables are important and unique in their own way. We use them as per the need. But don't you think using export, not export becomes pretty confusing with variable scope definition? Well, yes so we generally use Declare to play around with variables.

Note: All predefined variables are in UPPERCASE

Declare 🤌

To remove the confusion with the usage of export, not export, and set properties to variables declare is used for simplification. This command is particularly useful for declaring variables with specific properties. The basic syntax is:



declare [options] variable=value

The options flag lets us decide the property of the variable.By default, the variables are stored in strings. The key options available are:

Option	Description
`-i`	Declares the variable as an integer.
`-a`	Declares the variable as an array.
`-r`	Makes the variable read-only.
`-x`	Exports the variable to the environment.

Let's use the options for further understanding



declare -i n=10

The i flag defines the variable n as an integer and stores the value in n. To make it read only just add -ir instead of -i. Declare command is certainly flexible, play around with the options for further clarity.



declare -a arr
arr[0]=10
arr[1]=97

Arrays are the most used data structure and you can simply create one with declare. To print any index value do echo ${arr[1]}. Incase, you want to create an associative array i.e. key-value pair-based array use -A instead of -a. Remember associative arrays are for the latest version of shells so you may not get the support for it in older versions.

Variable can be referenced as $n

While writing scripts we pass value of variables with $n, where n being the number of variable. We will get back to it for further clarity.

First Script ✨

Let's write our first shell script.



vim myscript

When we execute the command above we create a file myscript.sh, by default .sh extension is added. Shell considers all files created from the terminal to be a script file unless mentioned. By vim we indicate to open the vim editor to work on the file. To start writing in the vim editor use escape key + 'i'.



#!/bin/bash
firstname="$1"
lastname="$2"
echo First Name is "$name" and Surname is "$lastname"

To run this script we simply get out of our vim editor.In case, you have not used Vim for the time being just press, the escape key and then use :wq. It will save the current state of the script and exit from the vim editor.

#!/bin/bash tells the terminal when we execute our script it should use bash to execute it. Incase you want to use zsh simply use #!/bin/zsh. Shebang is used for telling which script we are using. Let's change the file permission:



chmod +x myscript

As we discussed chmod before, I am confident you know what it means. Over here we permit us to execute the shell script.Now, let's run it:



./myscript

Well, we are done with executing out first shell script 🎉 We get the output. First Name is <space> and Surname is <space>. But, well we expected firstname and surname instead of blank space right? So, let's pass our values.



./myscript Aniket Pal

And now we have our desired output, ie. First Name is Aniket and Surname is Pal. We use $1,$2 instead of writing the variable name. We could have used



./myscript firstname="Aniket" lastname="Pal"

But, it creates a problem in remembering what exact variable name we have used. Imagine an example when you wrote a login script and you passed username and password as parameters. It will be difficult to remember what was the exact variable name you wrote while creating the script. Thus, in this case, we use $1, $2 and so on.

But their is another problem. Suppose you write a lot of variables and don't remember the exact number at which you ended. So, when you try to initialize a new variable, you need to lookup. So, we use the shift keyword.



#!/bin/bash
firstname="$1"
shift
lastname="$1"
echo First Name is "$name" and Surname is "$lastname"

So, even when you have hundreds of variables, you just need to use shift and then use $1. Isn't this amazing? Well, there is something cooler than this.

Why $1, not $0?

When passing arguments to scripts they populate variables. $0 represents the script itself. Say, for example:



./namescript.sh aniket alice bob

Arguments	Variable Number
./namescript.sh	$0
aniket	$1
alice	$2
bob	$3

Incase, you want to get the total number of arguments use $#, to get argument list use $* and get the arguments in an array use $@. For the above script use:

Option	Output
$#	3
$*	"aniket alice bob"
$@	["aniket" "alice" "bob"]

That's all for variables. Now, let's dive into Conditional, yes yes our regular if-else and case statements.

Conditionals 🤔

Just like in any programming language, conditionals are used to make decisions based on conditions. We have if-else statements and case in a shell. The basic syntax of if-else statement is as follows:



if condition; then #Code to execute if condition is true; else # Code to execute if condition is true; fi

Wait, if you find it confusing I know it can be, let's write a simple shell command to create a directory named 'aniket'. If it doesn't exist it will print a custom message.



if mkdir "aniket"; then echo "created aniket"; else echo "come on use another name"; fi

Whenever we complete a line we use ;. Run the above command twice, in the first go you will see "created aniket" and check with ls command if it has created the directory. When you run the same command again, it will display:



mkdir: aniket: File exists
come on use another name

So, we understood how to use if-else. If you wanted to use else if just add elif and continue. But what if we wanted to execute checks in the conditionals? Then:



if [ condition ]; then
    # Code to execute if condition is true
else
    # Code to execute if condition is false
fi

If you look carefully we write the condition inside third brackets meaning inside [...]. In shell scripting, the or test command is used to evaluate conditional expressions. It is a fundamental component of the if-else construct. Remember, we could have used (), but it's preferable to use [...]. When we proceed further you will get further clarity. Let's check with a simple odd/even detector.



if [ $((number % 2)) -eq 0 ];then echo "even"; else echo "odd"; fi

Since, our length of code is growing it's better to use to script. Too many semi-colons may make it a bit untidy. If, we wanted the above example to be written in a script, it would have been:



#!/bin/bash

number=7

if [ $((number % 2)) -eq 0 ]; then
    echo "Even"
else
    echo "Odd"
fi

💡 Just above we learned how to take variables as input. As your learning task, modify the above script and try to take the number as a variable and check if it's working.

The [[ ]] double square brackets are an enhanced version of [ ] and provide additional features and improvements in comparison. They are often preferred for complex conditions. Let's see an example:



#!/bin/bash

age=25

if [[ $age -ge 18 && $age -le 30 ]]; then
    echo "Person is between 18 and 30"
else
    echo "Person is not between 18 and 30"
fi

Remember previously, only one command was present in UNIX, i.e. test. So, if you wanted to check instead of using [], You needed to use test. [[...]] is a bash extension that is generally used while performing complex calculations. For example:



if test "$age"="10"; then ...

Which can now be written as,



if ["$age"="10"]; then ....

💡 If you are still here with me, great job. Trust me I am proud of you. Just as a bonus for you.

Shellcheck scriptName.sh

Shellcheck is a utility to detect bugs and mistakes in the script. Possibly it can save you hours of hard work in debugging.

Cases 🥺

Case statements are more efficient than using multiple elif statements. The basic syntax is:



case $variable in
    pattern1)
        # Code to execute for pattern1
        ;;
    pattern2)
        # Code to execute for pattern2
        ;;
    pattern3)
        # Code to execute for pattern3
        ;;
    *)
        # Code to execute if none of the patterns match
        ;;
esac

Start with case and end with esac, just a reminder every block ends with ;;. $variable is the variable whose value is being tested and pattern1, pattern2, pattern3 are the patterns to match against. Let's understand Case Statements further with fruits example:



#!/bin/bash

fruit="apple"

case $fruit in
    "apple")
        echo "It's an apple."
        ;;
    "banana")
        echo "It's a banana."
        ;;
    "orange")
        echo "It's an orange."
        ;;
    *)
        echo "It's something else."
        ;;
esac

*) is a wildcard pattern that matches anything not matched by previous patterns. In this example, the script checks the value of the fruit variable and executes the corresponding block of code based on the matching pattern. If none of the patterns match, the code under *) is executed. The case statement is a powerful tool for handling multiple conditions in a concise and readable manner in shell scripts.

Looping Through Life 🤩

In shell scripting, for, while, and until loops are used for iterative execution of code. Here's an overview of each along with examples:

1️⃣ for loop
The for loop is used to iterate over a sequence of values, such as numbers or elements in an array.



#!/bin/bash

for ((i=0;i<5;i++))
do
  echo $i
done

2️⃣ While Loop
The while loop executes a block of code as long as a specified condition is true



#!/bin/bash
i=10
while [i -gt 0]
do
  echo $i
  ((i--))
done

3️⃣ Until Loop
The do-while loop or until loop is similar to the while loop, but it guarantees that the code block is executed at least once before checking the condition.



#!/bin/bash
i=1
until ((i==0))
do
  echo $i
  i = i-1
done

Just for fun, let's use for loop to see what we can do with it, very specific to shell.



for i in {1..5};do touch file$i.cpp; done

The particular command will let you create files in the current directory, with names file1.cpp, file2.cpp,.. file5.cpp.

💡 Now, as a task try to remove the files that were created using the for loop. Tip: Replace touch with rm.

These loops provide flexibility for repetitive tasks in shell scripts, and the choice between them depends on the specific requirements of the script. If you noticed, all three loops for, while, and until loop had keywords done and done.

When we start the loop we mention do and when we are done looping we end with done. I am pretty sure, you have used multiple loops to date and it would be boring to explain each and everything with further explanation. So, let's dive into functions.

Functions 📚

In shell scripting, functions are used to group a set of commands into a single unit that can be reused and called with a specific name. The basic syntax of the function is as follows,



functionName() {
    # Code to be executed
    # ...
    # Optionally, return a value
    return value
}

The function definition in Shell is quite similar to JavaScript, right? With (),{} defining the start and end of the function and return used to return a value from a function. Let's write our first generic function post which we will learn a shell-specific function.



#!/bin/bash

addNumbers() {
    sum=$(( $1 + $2 ))
    echo "Sum: $sum"
}

addNumbers 5 7

Here, we define a function named addNumbers. As we learned function parameters can be accessed using $1,$2 so we parse the arguments and add them. In the end, we call our function addNumbers with args 5 and 7.



unset -f functionName

Is used to delete the function if you have defined in local scope of your terminal instead of script. If you want to export functions directly from terminal without using scripts:



declare -xf functionName(){...}

With the flag -x we define exporting and -f we define it as a function. I told you the power of declare, remember? If not, scroll a bit and revise.

Final Script 💎

Now, as promised let's check out a bit of a Shell-specific example for further clarity. By writing a script that checks the status of multiple servers using functions and parallel execution. Not only we will learn more about function but revise all the topics that we learned so far.



#!/bin/bash

checkServerStatus() {
    local server=$1
    local status=$(curl -sI "$server" | head -n 1 | cut -d' ' -f2)

    if [ "$status" == "200" ]; then
        echo "$server is up."
    else
        echo "$server is down."
    fi
}


servers=("http://twitter.com" "http://google.com" "http://blahblah.in")

checkAllServersInParallel() {
    for server in "${servers[@]}"; do
        checkServerStatus "$server" &
    done
    wait
}


checkAllServersInParallel

The above function contains quite a few concepts, right? Such as using arrays, for loop, if-else, and understanding the scope of variables such as local. I will demystify the above script, for more clarity.

The checkServerStatus function takes a server URL as an argument and uses curl to check its HTTP status. An array servers contains the URLs of multiple servers to be checked. The checkAllServersInParallel function iterates through the array, calls checkServerStatus for each server in the background (&), and waits for all background jobs to finish (wait). If you have read previously a bit about how the web works, I am sure you understand why we have used wait over here. The script then checks the status of multiple servers in parallel.

Although the script teaches us and reminds us of almost all the concepts. There is a particular line, which might confuse you, right?



local status=$(curl -sI "$server" | head -n 1 | cut -d' ' -f2)

It was meant to confuse you and remind you there is a long way to go. This line of code utilizes three commands to check the HTTP status code of a server's response. The curl command fetches the headers of the specified server, head -n 1 selects the first line containing the HTTP status, and cut -d ' -f2 extracts the actual status code. The resulting status code is stored in a variable named status. This approach allows the script to determine the server's responsiveness based on its HTTP status.

As stated before, the purpose, of the blog is to help beginners get started with Shell Scripting. Before I abruptly end it over here, I will add a bit of concepts that can be handy in your Shelling Journey.

Restricted Shell ✋

When aiming for code portability across various Unix-like shells, incorporating POSIX features becomes essential, ensuring compatibility with older shells beyond Bash or Zsh. However, in organizational settings or secure environments, limiting user capabilities is crucial for security. System administrators achieve this through the use of a restricted shell. If you're on your personal computer and wish to verify whether your shell is restricted, you can check by running:



echo $0

This command will display the name of the current shell. If it's a restricted shell, it might be named something like rsh or rbash. Checking for these restrictions helps maintain a secure computing environment in scenarios where executing arbitrary commands is restricted. If you are on your personal computer and want to check if it is restricted run:



shopt restricted_shell

Where, shopt implies Shell Options. By default Restriction is off, to turn it on run:



rbash

The restricted shell option is read-only and cannot be changed with shopt command. Restriction can often stop from changing directory, accessing particular files and more. It is generally used in large MNCs and always in banks to let users perform as much as required.

With the help of shopt, we can even configure options for our system. Say, when you want to change your directory instead of writing cd dirName just type the dirName, by just running



shopt -s autocd

It's always important to know how to get back, so if you want to unset the particular configuration do:



shopt -u autocd

There is an array of options to try, just type shopt and play around.

Taking Input 🧩

In shell scripting, capturing user input is achieved using the read utility, comparable to std::cin in C++. To take input with a prompt read is the utility we are looking for.



read

Without specifying a variable, the input is stored in the default variable named $REPLY.For a more customized approach, you can use options with read. For instance:



read var

Stores the value in variable var. To secure sensitive information, like passwords, without displaying them, use:



read -s var

Additionally, you can limit the input length with flags like -n10, restricting it to 10 characters. There are various options available to tailor the behavior of read based on specific needs. Experimenting with these options can provide you with versatile ways to handle user input in shell scripts.

AFinally, when prompting users for specific input, you can enhance the user experience by using the -p flag followed by the prompt text. For example:



read -p  "Enter Favorite Dish" dish

This not only captures user input but also provides a clear prompt, improving the overall interaction in your shell script. Experimenting with these features allows you to create more user-friendly and interactive scripts.

Outro 💚

Congratulations🥂 on embarking on your shell scripting adventure! We've covered a lot in this blog, from the basics of variables to the intricacies of case statements, loops, if-else conditions, and even crafting entire scripts. By now, you've probably realized that the world of shell scripting is as vast as the command line itself, and there's always more to learn.

Consider this blog your trusty launchpad, propelling you into the fascinating realm of automation and efficiency. We've just scratched the surface, and there's a galaxy of possibilities waiting for you to explore. Remember, Rome wasn't built in a day, and neither is a master shell scripter!

In our next installment, we'll dive into the mysterious world of scheduling cron tasks and delve deeper into the art of scripting. Get ready to take your scripting skills to the next level!

And for those thinking, "Hey, I need more of this wit and wisdom in my content," consider hiring me as your content freelancer. Let's script some success together!

If you run an organisation and want me to write or create video tutorials please do connect with me 🤝

Build Email Verifier With Go

Aniket Pal — Mon, 15 May 2023 14:36:24 +0000

The first step of cold mailing starts with getting the email address of the technical recruiter. It has been often said over the internet to randomly form email with the combination of first and last name of the recruiter.

Say, if you want to mail me the possible combinations can be aniket.pal@companymail.com, pal.aniket@companymail.com, paniket@companymail.com, aniketpal@comapanymail.com and related. Creating a humongous list of multiple permutation and combinations. To shorten the search space we will build an application to verify if the companymail.com is valid or not.

The purpose of the tutorial is to give you, the sheer understanding of the capabilities go-lang has. We won't be using any third party modules, with just the core modules of Go we will be building.

Why Choose Go? 🤨

The story goes, Google developers developed Golang while waiting for other languages to get compiled. Google developers had to completely rethink system development as a result of their displeasure with their toolset, which drove them to develop a lean, mean, and compiled solution that supports huge multithreading, concurrency, and performance under stress.

Every organisation looking at scale is leveraging Golang to build containerised microservices.

Moreover, Go is a great language for creating simple yet efficient web servers and web services. It provides a built-in HTTP package that contains utilities for quickly creating a web or file server.

Also, if you are interested in the Cloud Native Ecosystem, Go is the language you should start with. Incase, you have never developed a backend server with Go checkout Build Server With Go Under 10 minutes.

What we will build? 👨‍🚒

The small tool will check, if the email domain exists or not. The aim is to understand how to build backend and frontend with just using Go Lang. We will start with the backend server then shift to building the front end, while providing you a space to explore.

Building the Backend 🚪

We would be using the frontend of the application to get the domain we need to look for.

Defining package main and importing the required packages.



package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net"
    "net/http"
    "strings"

    "github.com/gorilla/mux"
)

If you are confused on how to install gorilla/mux and what does the package do, read Building a server with Go under 10 minutes. As of now, We will be building the REST APIs a little later. First, let us assume we get the domain as a string which we need to work upon.

Defining a handler, isValidDomain. The handler just takes one parameter say,domain of type string



func isValidDomain(domain string){
// controller code goes here
}

Let's start working with the controller function. Firstly, defining variables to check if the particular domain has MX Records, SPF Records and DMARC Records. If they have what are the records.



  var hasMX, hasSPF, hasDMARC bool 
  var spfRecord string
  var dmarcRecord  string

Now, it is the time to check on the internet for the data. The intensive in-house Go packages, get the job done with net/http package. Incase, you would have been using say ruby or nodejs you would have required to install further binaries. ```go

mxRecords,err := net.LookupMX(domain)

if err != nil{
    log.Printf("Error: %v\n",err)
}

if len(mxRecords)>0{
    hasMX = true 
}

We are using the net package to get the `mxRecords`, incase we don't get the data we get error. We handle the error in the following lines. To check if we have received mxRecords, we check if the length of the array is more than one, meaning if the array contains more than one element. 

```go


    txtRecords, err := net.LookupTXT(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range txtRecords{
        if strings.HasPrefix(record,"v=spf1"){
            hasSPF = true 
            spfRecord = record
            break
        }
    }

Similar to mxRecords, we handle the data and error using := operator.

Remember, := is a declaration, whereas = is an assignment operator.

Post handling the error, we traverse the txtRecords slice, since we don't require the index we ignore it via _. Since, Golang doesn't permit having unused variables we use the underscore operator. When we loop over txtRecords, we check if version of the following record is spf1. If so, we mark hasSPF positive and store the value of record in spfRecord.



    dmarcRecords, err := net.LookupTXT("_dmarc." + domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range dmarcRecords{
        if strings.HasPrefix(record ,"v=DMARC1"){
            hasDMARC = true
            dmarcRecord = record 
            break
        }
    }

Looking up for dmarcRecords is pretty much similar to txtRecords. We add _dmarc. in the prefix of the domain url, and use the net package to check if the corresponding records for dmarc exists or not. While traversing the dmarcRecords slice, we check if version of the record is DMARC1 we mark hasDMARC true and store the value of record in dmarcRecord.

To check if we are getting a values and our handler isValidDomain is working fine. Let's print the values we have till now.



fmt.Printf("domain=%v\n,hasMX=%v\n,hasSPF=%v\n,spfRecord=%v\n,hasDMARC=%v\n,dmarcRecord=%v\n",domain,hasMX,hasSPF,spfRecord,hasDMARC,dmarcRecord)

Compiling the isValidDomain script, we have.



func isValidDomain(domain string){
    var hasMX, hasSPF, hasDMARC bool 
    var spfRecord string
    var dmarcRecord  string 

    mxRecords,err := net.LookupMX(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    if len(mxRecords)>0{
        hasMX = true 
    }

    txtRecords, err := net.LookupTXT(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range txtRecords{
        if strings.HasPrefix(record,"v=spf1"){
            hasSPF = true 
            spfRecord = record
            break
        }
    }

    dmarcRecords, err := net.LookupTXT("_dmarc." + domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range dmarcRecords{
        if strings.HasPrefix(record ,"v=DMARC1"){
            hasDMARC = true
            dmarcRecord = record 
            break
        }
    }


    fmt.Printf("domain=%v\n,hasMX=%v\n,hasSPF=%v\n,spfRecord=%v\n,hasDMARC=%v\n,dmarcRecord=%v\n",domain,hasMX,hasSPF,spfRecord,hasDMARC,dmarcRecord)
}

Since, our function is working. Let's start working on building our REST APIs. Firstly, we will be defining 2 structs namely, DomainURL and DomainVar. One for decoding input and one for encoding the output.



type DomainURL struct{
    DomainURL string `string:"domainurl"`
}

type DomainVar struct{
    Domain string `json:"domain"`
    HasMX bool `json:"hasmx"`
    HasSPF bool `json:"haspf"`
    SpfRecord string `json:"spfrecord"`
    HasDMARC bool `json:"hasdmarc"`
    DmarcRecord string `json:"dmarcRecord"`
}

If you are a JavaScript developer struct is similar to ES6 class. Now, let us define a slice, which is similar to vectors in C++.



var domainVars []DomainVar

A slice is similar to an array, the difference is that when you want to use arrays in Golang you need to define the length. This is why we use a slice, we also tell it that it will contain posts. Over here domainVars is a slice of type DomainVar. Adding the route for POST request in the main function.

First creating a new request router. The router is the main router for our web application and will later be passed as parameter to the server. It will receive all HTTP connections and pass it on to the request handlers we will register on it. We create the router, in the main function. Post registering the router, let's define the endpoints using HandleFunction. We call HandleFunction by r.HandleFunc(...)



func main(){

    r := mux.NewRouter()
    r.HandleFunc("/form",formHandler).Methods("POST")

        fmt.Print("Starting server at port 8000\n")
        log.Fatal(http.ListenAndServe(":8000",r))
}

In HandleFunc we provide 2 parameters, firstly the route in which we want to see the magic and secondly we write the name of the particular controller function which performs the magic. The corresponding GET and POST has the regular meaning, to build the CRUD operation for the application just add PUT and DELETE according to the route.

The server port can be migrated to any value required.The script for POST Request Handler aka form Handler one step at a time.



func formHandler(w http.ResponseWriter, r *http.Request){
   w.Header().Set("Content-Type","application/json")
}

We define the the Header for the particular controller over here. We’re just setting the header “Content-Type” to “application/json”.



func formHandler(w http.ResponseWriter, r *http.Request){
   w.Header().Set("Content-Type","application/json")

   var domainUrl DomainURL
   json.NewDecoder(r.Body).Decode(&domainUrl)
}

Now, we pass the domain value in the isValidDomain function. We store all the in a domainVar. Post that we append the domainVar into the domainVars slice. Then we use the encoding package to encode all the domainVars data as well as returning it at the same line. Ultimately, sending in the POST request, fetching the data, storing in a struct, appending into the slice and then returning back the slice.



func formHandler(w http.ResponseWriter, r *http.Request){
    w.Header().Set("Content-Type","application/json")

    var domainUrl DomainURL
    json.NewDecoder(r.Body).Decode(&domainUrl)

    domainVar  := isValidDomain(domainUrl.DomainURL)
    domainVars = append(domainVars, domainVar)

    json.NewEncoder(w).Encode(domainVars)
}

Since, our route is configured let us modify our isValidDomain function such that it can we can get the required values for DomainVar struct.



func isValidDomain(domain string) DomainVar{
    var hasMX, hasSPF, hasDMARC bool 
    var spfRecord string
    var dmarcRecord  string 

    mxRecords,err := net.LookupMX(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    if len(mxRecords)>0{
        hasMX = true 
    }

    txtRecords, err := net.LookupTXT(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range txtRecords{
        if strings.HasPrefix(record,"v=spf1"){
            hasSPF = true 
            spfRecord = record
            break
        }
    }

    dmarcRecords, err := net.LookupTXT("_dmarc." + domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range dmarcRecords{
        if strings.HasPrefix(record ,"v=DMARC1"){
            hasDMARC = true
            dmarcRecord = record 
            break
        }
    }

    fmt.Printf("domain=%v\n,hasMX=%v\n,hasSPF=%v\n,spfRecord=%v\n,hasDMARC=%v\n,dmarcRecord=%v\n",domain,hasMX,hasSPF,spfRecord,hasDMARC,dmarcRecord)

    var domainVar DomainVar
    domainVar.Domain = domain
    domainVar.HasMX = hasMX
    domainVar.HasSPF = hasSPF
    domainVar.SpfRecord = spfRecord
    domainVar.HasDMARC = hasDMARC
    domainVar.DmarcRecord = dmarcRecord

    return domainVar 
}

The modified function has a return type DomainVar which ultimately return the object of the same type. We create an instance type DomainVar and then start assigning values for each key. Finally return the object.

*The complete Backend Code as of now is: *



package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net"
    "net/http"
    "strings"

    "github.com/gorilla/mux"
)

type DomainURL struct{
    DomainURL string `string:"domainurl"`
}

type DomainVar struct{
    Domain string `json:"domain"`
    HasMX bool `json:"hasmx"`
    HasSPF bool `json:"haspf"`
    SpfRecord string `json:"spfrecord"`
    HasDMARC bool `json:"hasdmarc"`
    DmarcRecord string `json:"dmarcRecord"`
}

var domainVars []DomainVar


func formHandler(w http.ResponseWriter, r *http.Request){
    w.Header().Set("Content-Type","application/json")

    var domainUrl DomainURL
    json.NewDecoder(r.Body).Decode(&domainUrl)

    domainVar  := isValidDomain(domainUrl.DomainURL)
    domainVars = append(domainVars, domainVar)

    json.NewEncoder(w).Encode(domainVars)
}

func isValidDomain(domain string) DomainVar{
    var hasMX, hasSPF, hasDMARC bool 
    var spfRecord string
    var dmarcRecord  string 

    mxRecords,err := net.LookupMX(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    if len(mxRecords)>0{
        hasMX = true 
    }

    txtRecords, err := net.LookupTXT(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range txtRecords{
        if strings.HasPrefix(record,"v=spf1"){
            hasSPF = true 
            spfRecord = record
            break
        }
    }

    dmarcRecords, err := net.LookupTXT("_dmarc." + domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range dmarcRecords{
        if strings.HasPrefix(record ,"v=DMARC1"){
            hasDMARC = true
            dmarcRecord = record 
            break
        }
    }

    fmt.Printf("domain=%v\n,hasMX=%v\n,hasSPF=%v\n,spfRecord=%v\n,hasDMARC=%v\n,dmarcRecord=%v\n",domain,hasMX,hasSPF,spfRecord,hasDMARC,dmarcRecord)

    var domainVar DomainVar
    domainVar.Domain = domain
    domainVar.HasMX = hasMX
    domainVar.HasSPF = hasSPF
    domainVar.SpfRecord = spfRecord
    domainVar.HasDMARC = hasDMARC
    domainVar.DmarcRecord = dmarcRecord

    return domainVar 
}


func main(){
    r := mux.NewRouter()

    r.HandleFunc("/form",formHandler).Methods("POST")


    fmt.Print("Starting server at port 8000\n")
    log.Fatal(http.ListenAndServe(":8000",r))
}

Building the Frontend 🎨

We won't be actually focusing on beautifying the application rather our focus would be on getting things done. For the frontend, we would require a package go-fiber. Although, the frontend could have been developed without installing any additional package, the reason for using go-fiber is to understand how to install and work with external packages.

Let's start with creating the directory and changing our working directory to it.



mkdir verifier-frontend && cd verifier-frontend

Initialising the main.go file for the frontend



touch main.go

Running the command, creates an empty file in our working directory over here which is verifier-frontend.

We need to define a package for the Go file. Since this will be the main file we have, we add the package main.



package main

The above line is the first line of the program.

Now, let's import all the necessary packages we require to build our application 🛍️



import (
    "log"

    "github.com/gofiber/fiber/v2"
)

If, I would have been you and read all the imports, I would have been pretty much confused. But, trust me once you read the blog I can bet you would get a clear idea why we included the following packages.

Creating the go.mod file which will store all the necessary packages required, it is similar to package.json.



go mod init github.com/Aniket762/namaste-go/verifier-front

To ensure that the go.mod file matches the source code in the module, we run



go mod tidy

All the imported packages except for one is already present with the binary file you executed while installing Go. So, let's install Gorilla Mux.



go get github.com/gofiber/fiber/v2

If you are a JavaScript developer, the following command is pretty much similar to npm install

Now, we are all set to start building our frontend. Let's get the code from Go Fiber's official documentation, run and understand it.



package main

import (
    "log"

    "github.com/gofiber/fiber/v2"
)

func main() {
    app := fiber.New()

    app.Get("/", func (c *fiber.Ctx) error {
        return c.SendString("Hello From Verifier's Frontend 👋")
    })

    log.Fatal(app.Listen(":3000"))
}

Post importing the packages, we write the main function. The main function is the first which gets started when we compile and execute the program. We create an instance, app of type fiber. We shorthand c for the context.

The HTTP request and response are held in the context, which is represented by the Ctx struct. It provides methods for the request's body, HTTP headers, arguments, and query string.

Let's run the server at port 3000. You can change the port according to your convenience. Now, let us run the server and check if everything is in sync. Switch to your terminal and execute go run main.go

You should get this on your terminal which shows the port we are running into, number of handlers, process IDs and number of processes we are running. The following data is of real use when you shift to building more complex applications. Navigate to localhost:3000 or which ever port you have written.

Outro 💚

The application we have developed is a prototype and not at all production ready. The only purpose of the blog was to give you the exposure on how to build a full stack application with native go packages. You can research further and build a production ready application.

Now, just like any other tutorial let us end the tutorial with a task. We have developed the REST APIs using gorilla/mux and frontend starter with go-fiber. Try build the POST route for the frontend. Incase you aren't able to build I would be soon posting an article on building frontend with Go-lang. All the best for building the complete frontend. Incase, you don't miss when I publish how to build frontend using go-fiber follow me on my socials ^-^

Incase you have developed the application or have anything to discuss under the sun feel free to get in touch with me on LinkedIn or Twitter 💖

If you run an organisation and want me to write or create video tutorials please do connect with me 🤝

Build Server With Go Under 10 minutes

Aniket Pal — Mon, 23 Jan 2023 17:03:43 +0000

The introduction of microservice architecture revolutionised the way softwares are developed today. Post microservice architecture replaced monoliths, containers replaced virtual machines. With this transformations, almost every modern software is built in microservice architecture. Golang is widely used among developers building Microservices. Its usage can be seen across industries as a popular option for server-side application development.

Go is a great language for creating simple yet efficient web servers and web services. It provides a built-in HTTP package that contains utilities for quickly creating a web or file server.

Why Build backend Server with Go? 🗽

The Story goes, Go was allegedly created by Google engineers while they waited for other programmes to compile. They had to completely rethink system development as a result of their displeasure with their toolset, which drove them to develop a lean, mean, and compiled solution that supports huge multithreading, concurrency, and performance under stress.

Every organisation looking at scale is leveraging Golang to build containerised microservices

Several programming languages have less memory and cannot communicate with the hardware. Thus, developers mostly prefer Golang that helps build software. Besides, it is the procedural language that is beneficial for implementing reliable, effective, and simple software. Golang supports the environment to accept different patterns that are equivalent to other more beneficial languages. Tech gaints have started using Go, post it's inception.

Graphics: Bacancy Techology

It can be a genuine delight to run Go on contemporary hardware, either inside containers or on virtual machines. Go was created to allow concurrency and scale as more cores are added since it was intended to run on multiple cores. If you are someone who is interested in the Cloud Native Ecosystem, Go is the language you should start with.

What we will build? 👨‍🚒

The following tutorial will include building a basic backend server with just main.go file. I won't be making the application complex by following the directory structure. Connecting DBs would make the tutorial a bit large and might confuse beginners who are just starting out with Go.

In the application we will learn how to add multiple routes, start server, write controllers and perform GET and POST requests.

The schematic diagram above is an attempt to showcase how the server is to be designed to work. The server we will be building will have 2 routes one for GET and the other for POST. Where in the home route will be configured for GET and the form route for POST.

Pre-Requisite ⚙️

If you don't have Go installed in your system follow the official docs to install it. Trust me it won't take more than 3 minutes to do it. To check if you have installed it properly, just run:

go version

When I run the particular command, I get go version go1.17 darwin/amd64 in return. If you get something similar congratulations 🎉 you are all set to begin your backend developer journey with Go.

Understanding: Syntax 👨‍🏫

Go is syntactically similar to C, but with memory safety, garbage collection, structural typing, and CSP-style concurrency. Now, if you have never coded in Golang before and don't have any clue how the syntax works. Refer to Namaste-Go. A learning repository, to get started with Go lang.

Aniket762 / namaste-go

A repository to learn the basics of Go Programming. The language was developed by Google Engineers but currently is widely used in the Cloud Native ecosystem. If you want to start your career in the landscape this is the starting point.

Namaste Go 🙏

View on GitHub

Once, you are done with the basics let's dive into building the backend server with Go lang.

Developing the Backend 👨‍💻

Let's create a directory goServer and change our current directory to it.

mkdir goServer && cd goServer

Post running the command we are in the goServer directory, all you need to do now, create a main.go which you can name according to whatever naming convention you follow, say server.go. As mentioned before, we won't be following the modularised approach to keep things simple. So, let's create the file.

touch main.go

Running the command, creates an empty file in our working directory over here which is goServer.

We need to define a package for the Go file. Since this will be the main file we have, we add the package main.

package main

The above line is the first line of the program.

Importing Packages 🛍️

Now, let's import all the necessary packages we require to build our application.

import (
    "fmt"
    "log"
    "net/http"

    "github.com/gorilla/mux"
)

Now, let us understand why did we include so many packages into our application. Demystifying the packages:

1️⃣ fmt: for printing message
2️⃣ log: for logging errors
3️⃣ net/http: for handling HTTP requesting when creating GO APIs
4️⃣ gorilla/mux: for URL matcher and routing. It helps in implementing request routers and match each incoming request with its matching handler.

Creating the go.mod file which will store all the necessary packages required, it is similar to package.json.

go mod init github.com/Aniket762/namaste-go/goServer

To ensure that the go.mod file matches the source code in the module, we run

go mod tidy

All the imported packages except for one is already present with the binary file you executed while installing Go. So, let's install Gorilla Mux.

go get "github.com/gorilla/mux"

If you are a JavaScript developer, the following command is pretty much similar to npm install <package_name>

Lastly, before we run the program to check if everything is in sync till now. We add the main() function which would get first executed when we compile the program.

func main() {
    fmt.Printf("Hello from GoServer 👋")
}

Summarising the entire codebase till now,

package main

import (
    "fmt"
    "log"
    "net/http"


    "github.com/gorilla/mux"
)

func main(){
    fmt.Printf("Hello from GoServer 👋")
}

To run the following Go Script execute the following command.

go run main.go

If you get Hello from GoServer 👋 in the console. Wohoo! You are all set to proceed further!

Building the server 💪

r := mux.NewRouter()

Post registering the router, let's define the endpoints using HandleFunction. We call HandleFunction by r.HandleFunc(...)

   r.HandleFunc("/",greetingsHandler).Methods("GET")
   r.HandleFunc("/form",formHandler).Methods("POST")

Writing the GET Route

Now, as we have defined the routes, let us write the controllers. Starting out with the greetingsHandler aka GET Controller.

func greetingsHandler(w http.ResponseWriter, r *http.Request){
    fmt.Fprintf(w,"Greeting from Go Server 👋")
}

By default, this function accepts a ResponseWriter to send a response back and a Request object that sends the parameters to the server. The controller simply sends back a message.

To check if our GET function is working, let's write the code to spin up the server in the main function.

        fmt.Printf("Starting server at port 8000\n")
    log.Fatal(http.ListenAndServe(":8000",r))

The Listen and Serve function keep the custom port open, here 8000. Until and unless the program is interrupted the server is up.

Before we move forward with the POST request handler let us, check if everything is functioning as expected. Let's comment out the POST route and go for it. Your code should look something like this 👇

package main

import (
    "fmt"
    "log"
    "net/http"

    "github.com/gorilla/mux"
)

func greetingsHandler(w http.ResponseWriter,r *http.Request){
    fmt.Fprintf(w,"Greeting from Go Server 👋")
}


func main(){
    r := mux.NewRouter()

    r.HandleFunc("/",greetingsHandler).Methods("GET")

    fmt.Println("Hello from GoServer 👋")
    fmt.Printf("Starting server at port 8000\n")
    log.Fatal(http.ListenAndServe(":8000",r))
}

Running the server 🏃‍♂️

Now, we have the basic server ready. Let us create a build and then run it.

Using the particular command we generate the .exe file if we are on windows and .deb if in linux. Ultimately generating a binary file which can be executed.

go build

Post creating the binary file, we execute the file via the following command.

go run main.go

Hit, localhost:8000 in your favourite browser. If it is something like this as shown below. Cheer yourself up, you have completed 80% of the tutorial.

Leveling Up: Post Request 🚀

It's great to see you are still reading. The beauty of Go can be enjoyed when you start with Post Requests. We will be using json encoding so, let's import another module for it,

import(
   "encoding/json"
)

First let's create a simple struct Founder, which will consists name, age, email address and name of the company they have founded.

type Founder struct{
    Name string `json:"name"`
    Age uint32 `json:"age"`
    Email string `json:"email"`
    Company string `json:"company"`
}

If you are a JavaScript developer struct is similar to ES6 class. Now, let us define a slice, which is similar to vectors in C++.

var founders []Founder

Let us insert dummy data into our slice, such that when we return our slice after perform POST request we can visualise more than element in our slice.

founders = append(founders, Founder{Name:"Mehul",Age:23,Email:"random@random.com",Company: "BharatX"})

Let us write the script for POST Request Handler aka form Handler one step at a time.

func formHandler(w http.ResponseWriter, r *http.Request){
     w.Header().Set("Content-Type","application/json")
}

We define the the Header for the particular controller over here. We’re just setting the header “Content-Type” to “application/json”.

func formHandler(w http.ResponseWriter, r *http.Request){
    w.Header().Set("Content-Type","application/json")

    var founder Founder
    json.NewDecoder(r.Body).Decode(&founder)
}

We send parameters while sending the request, as key value pairs. The request object is broken down into string with decoder method. Let's try to play a bit around with the parameter we have received.

func formHandler(w http.ResponseWriter, r *http.Request){
    w.Header().Set("Content-Type","application/json")

    var founder Founder
    json.NewDecoder(r.Body).Decode(&founder)

    founder.Age = founder.Age * 2;
    founders = append(founders,founder)

    json.NewEncoder(w).Encode(founders)
}

We multiple the age of the founder by two, and append it to the slice. Then we use the encoding package to encode all the founders data as well as returning it at the same line. Ultimately, sending in the POST request, modifying the data, appending into the slice and then returning back the slice.

So, your final piece of code should look like:

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net/http"

    "github.com/gorilla/mux"
)

type Founder struct{
    Name string `json:"title"`
    Age uint32 `json:"age"`
    Email string `json:"email"`
    Company string `json:"company"`
}

var founders []Founder

func greetingsHandler(w http.ResponseWriter,r *http.Request){
    fmt.Fprintf(w,"Greeting from Go Server 👋")
}

func formHandler(w http.ResponseWriter, r *http.Request){
    w.Header().Set("Content-Type","application/json")

    var founder Founder
    json.NewDecoder(r.Body).Decode(&founder)

    founder.Age = founder.Age * 2;
    founders = append(founders,founder)

    json.NewEncoder(w).Encode(founders)
}

func main(){
    r := mux.NewRouter()

    founders = append(founders, Founder{Name:"Mehul",Age:23,Email:"random@random.com",Company: "BharatX"})

    r.HandleFunc("/",greetingsHandler).Methods("GET")
    r.HandleFunc("/form",formHandler).Methods("POST")

    fmt.Println("Hello from GoServer 👋")
    fmt.Print("Starting server at port 8000\n")
    log.Fatal(http.ListenAndServe(":8000",r))
}

The process does not end here, let us check on what we have developed so far is working or not. Firstly, let's build and spin up the server

go build && go run main.go

API testing with Postman 📬

Now, let's head over to Postman. You can use anything you want to but I prefer Postman.

Firstly, put the API endpoint in this case it is, http://localhost:8000/form, select POST from the dropdown. Navigate to Body tab and choose raw and then select JSON. Write dummy data, here is the one I wrote:

{
    "name":"Kunal Shah",
    "age":42,
    "email":"kunal@coolguy.com",
    "company":"CRED"
}

Hit the Send button. Check for the status code and the output which you got. Incase it's exactly what you were expecting, then it's time for celebration 🍻🥳

Outro 💚

Firstly, thanks a lot if you are still reading. Remember, Go is like chess or, well, the game of Go: it takes a moment to learn and a lifetime to master. Luckily, unlike chess, Go’s difficulty goes down with experience and soon you’ll be coding fast and furious programs in one of the world’s most modern languages.

The purpose of this blog was to give programmers who are enthusiastic about backend development with Golang a kickstart. The server built consists of basic features such as multiple routes, creation of GET and POST request, building binaries, running server and ultimately giving a basic understanding on how exactly Go works.

I plan to write more backend development with Go tutorials. If you don't want to miss out, follow me at Twitter. Till then checkout the wonderfully curated Go Documentation. Incase, you have some questions regarding the article or want to discuss something under the sun feel free to connect with me on LinkedIn 💖

If you run an organisation and want me to write or create video tutorials please do connect with me 🤝

Bundling YAMLs for Kubernetes with Helm

Aniket Pal — Sat, 07 Jan 2023 05:11:45 +0000

The introduction of microservice architecture revolutionised the way softwares are developed today. Post microservice architecture replaced monoliths, containers replaced virtual machines. With this transformations, building application became complex due to multiple containers. Container orchestration was a new bottleneck which was solved by Kubernetes.

In the process containers and Kubernetes brought in many new challenges. The most difficult being deploying and managing applications on Kubernetes. An array of open source tools are being developed to ease out the process. One such tool is Helm.

History: Start of Helm 🗽

After being in the community for more than a year, Kubernetes finally made its first general availability release with version 1.0 in the summer of 2015. Deis, a business devoted to supporting the Kubernetes Ecosystem, created Helm in its initial iteration.

Introduced at the inaugural KubeCon. In January of 2016, the project merged with a GCS tool called Kubernetes Deployment Manager, and the project was moved under Kubernetes. Since its beginning, Helm has been intended to function as a package management tool. With Helm V4, Helm is currently on its fourth iteration. With Helm V3, its third iteration, it attracted a lot of adoption due to reduction of security concerns.

What exactly is Helm? 🤔

If you are a javascript or a python developer, you can think Helm to be equivalent to npm or PyPI. The concept of package managers are not new, if you are a linux users I am pretty sure you have used apt/yum for installing and uninstalling softwares.

With the growth of complex services, deploying with only one k8s configuration file was not feasible. Where to even store these YAMLs became a problem as the amount of YAMLs grew. Helm is here to help with these issues.

Graphics: Logz.io

Suppose we want to implement Elastic Stack for logging. To do it we need to write configmap, secret, stateful set, permissions and more. Writing all of these at times becomes pretty tedious and often increases the chance of error. So bundling is done. These bundles are known as Helm Charts.

Why is Helm so popular? 🔥

The big benefits to using Helm are pretty similar to CI and CD goals: repeatability and consistency. Imagine, a server used by millions of users having reductant code, the amount of cash burn will be insane. Moreover, bundling reduces the chance of errors. Helm also referred as templating engine helps in defining a common blueprint. Dynamic values are replaced by place holders which helps in achieving code consistency and POCs.

Ultimately using Helm saves the development time. It takes care of the CI/CD pipeline where in we can focus on building what matters. With Helm we no longer need to construct individual YAML files for every application in a stack. Helm Charts are highly customisable, we can just download a Helm Chart and combine it with our YAML file, thus no longer we are required to build everything from scratch.

Helm Developer develops the Helm Charts such that it can be used by everyone. Post creating the Chart, it is pushed into the repository which is then further used by developers. Developers pull the image from the repository using Helm, post modifying according to the required value they use it while defining their own clusters.

Not only software developers or open source initiatives can use Helm packaging. Every individual or organisation installing softwares for day to day purpose can use Helm packaging. Using a package manager lowers complexity and duplication in coordinating the procedures specified in the package manager as Kubernetes investments continue. Helm having the feature of Release Management also power rollback add-on incase anything goes wrong.

What are Helm Charts? 📈

Helm charts are bundles or collections of Kubernetes YAML files that make up an application. The main format that Helm operates is a Chart. Helm Charts can be stored in Chart Repositories with ease because they are file-based and adhere to a convention-based directory structure. Charts can be installed and uninstalled into Kubernetes clusters.

A running instance of a Chart is referred to as a Release, just as the relationship between an image and its container.

A Helm Chart is ultimately an executable template that transforms Chart definitions into Kubernetes manifests.

Helm Charts, when created must have a version number. Helm Charts can reference other Charts as dependencies, a core to any package manager. Moreover, there are private and public repositories where in we can store Helm Charts.

Demystifying Helm Charts 💁🏻‍♂️

Using a single Helm Chart, Helm allows us to deploy many configurations for the same application. Helm accomplishes this using a template engine. In response to input parameters that can be modified in a values.yaml file, the template engine generates manifest files.

Application configurations can be created and shared via Helm. We can search for Helm Charts on Artifact Hub or using cli via helm search [keyword] command.

We can also find Helm Charts on GitHub, GitLab, BitBucket and related platforms. Here is the verified Helm Chart for Nginx. Now, that we have an idea on where to find Helm Charts, let's dive in to Helm Chart's skeleton.

Breaking down Helm Charts 🚧

Helm Charts are stored in a modular fashion. All particular charts have similar directories.

| -- Chart.yaml
| -- Charts
| -- templates
|    | -- NOTES.txt     
|    | -- deployment.yaml 
|    | -- ingress.yaml
|    | -- service.yaml
| --values.yaml

With these pieces out of the way, Helm can execute on our Chart. To understand Helm Charts we need to understand the 4 basic building blocks:

1️⃣ Chart.yaml file : This is where we put all the information about the chart we are packaging. So, for example, the version number, etc. This is where we will put all those details.

2️⃣ Values.yaml file : This is where we define all the values we want to inject into our templates. Values.yaml is similar to variable.tf in terraform.

3️⃣ Charts: This is where we store other charts that our chart depends on. We might be calling another chart that our chart need for functioning properly.

4️⃣ Templates directory : This directory contains the templates files of the chart, and when they combined with values, they generate the valid manifest files.

Walkthrough: Let's see how it works 💥

To install Helm command use supported package manager or simply download the pre-complied binary.

brew install kubernetes-helm

Installing Tiller with the Helm init command, to start deploying applications into pure Kubenetes cluster.

helm init

Now, Helm can be used particularly in two ways. Either modifying post fetching from web or creating from scratch.

Install Charts from Web 💻

For the first way, ie to install Helm chart from web we need to do two things.

Managing remote stable repositories:

helm repo add bitnami https://charts.bitnami.com/bitnami

And then installing the chart:

helm install my-nginx bitnami/nginx

Now, if we want to use with custom yaml file and override some of the values, we simply need to run

helm install my-nginx bitnami/nginx -f example-config.yaml

Creating Charts locally 👨‍🚒

Using Helm packages is really simple. Writing Helm charts is only a little bit more complex. Creating a Helm chart involves creating the chart itself, configuring the image pull policy, and specifying additional details in the values.yaml file.

Initialising with the create chart command

helm create explore-helm

Post running the command and checking out the starter directory. We can find the chart directory which consists of:

| -- Chart.yaml
| -- Charts
| -- templates
| -- values.yaml

Post configuring according to requirement, we can deploy. This way, Helm charts simplify application deployment on a Kubernetes cluster.

Outro 💚

Helm is undoubtedly the future of K8s Package Management. Helm not just only helps developers who want to install some apps on it, but it also helps DevOps engineer, they can still create YAML manifest files and use kubectl commands to do deployments. Using the templating engine engineers can focus on what really matters, Helm takes care of code consistency and redundancy.

The bundling of YAML files has ultimately reduced the bundle size of the server application and enhanced the CI/CD flow. Moreover, Helm simplifies the DevOps learning curve: we do not require a full, detailed understanding of the function of each Kubernetes object in order to start developing and deploying container applications. Helm easily integrates into CI/CD pipelines and allows software engineers to focus on writing code not deploying applications.

The purpose of this blog is to create awareness about Helm and similar package managers. It is further recommended to go through Helm's official Documentation. Incase, you have some questions regarding the article or want to discuss something under the sun feel free to connect with me on LinkedIn 💖

If you run an organisation and want me to write or create video tutorials please do connect with me 💪

Merging Pull Requests Like a Pro With Mergify

Aniket Pal — Wed, 18 May 2022 04:43:29 +0000

If you have been using Git for sometime now, I am sure working with Pull Requests is now a second nature to you. And it is also possible you have started getting irritated with merge conflicts a lot. But what if I tell you, even after merging the conflicts we can't be sure if the pull request will break our production or not? Have you ever thought, how enterprise projects are maintained? How do so many contributors work together in sync without breaking the production?

Engineering teams at Shopify, Uber, and almost every other company have faced the same issue with merging and used the same hammer to break and finish off the problem. Many engineering teams and open source projects are introducing merge queues as part of their workflows.

What exactly is the problem? 🤔

Combining the individual work of an enormous team of engineers into one codebase is tough. And the difficulty lies deeper than our standard merge conflicts, which version control is fairly good at handling. There’s a way more troublesome type of conflict we can run into when simultaneous changes are made to multiple areas of code that depend upon one another.

Let's understand the problem a bit more using GitHub. Say, we create a Pull Request which passes the CI for the main branch.

Before our PR gets merged there is another commit. Does not matter how the commit is done, be it done by merging another PR or committed directly to the main branch.

Even after the commit, the CI checks show green. Now, as amateur maintainers, we may find this to be fine, as the checks pass. So, we merge the PR, and guess what? There is a high chance we broke the production env.

The general CI checks or any version control system fails over here. Thus, the need for Merge Queue arises.

But how can this even occur? 🤯

You might be wondering, even if all the checks are passing, how can merging the PR break the code at production? Imagine we add a function call in some new code, and our teammate is busy refactoring that function at the same time, changing its signature or return values. Now since the file we changed, didn't have any new changes when our friend committed, the CI would still pass for us. Version control and even tests and checks in CI can’t help with this kind of conflict, but merge queues can.

Understanding the scale of the problem 🛒

At Shopify, developers merge and deploy over 400 pull requests each day to Shopify master. More than a million merchants depend on Shopify, one error while merging may fire up billions of dollars. The engineering team at Uber reported there is a 5% chance of an actual conflict between 2 changes, this number grows to 40% with only 16 concurrent and potentially conflicting modifications. So these conflicts can happen almost every day for large organizations.

What is Merge Queue? ↣

A Merge Queue is exactly what it sounds like: it's a queue that you add our branch/Pull-Request to for it to be merged. Merge Queue is a FIFO queue that manages the merge workflow for our Github repository.

It is always easier for developers to directly commit to the master branch. But with the scaling of the project and with the increase in code complexity and contributors the need for branches arises. Merge Queue in simple terms can be thought of as the tool which gives the advantages of directly committing to the master and also the scalability of creating branches. Aside from the obvious convenience of not having to wait for builds to pass to merge, the big reason for using a Merge Queue is that it can help eliminate bad merges and save us both time and money.

Using a merge queue solves that problem by updating any pull request that is not up-to-date with its base branch before it is merged. The update forces the continuous integration system to retest the pull request with the new code from its base branch, catching any potential regression.

Workflow of Merge Queue? 🗒

Merge queues operate on a rather simple premise. Suppose we have 2 pull requests of 2 differents features. Neither Git nor CI were wrong in saying that any of the two branches were, at one point in time passing all the checks. But those checks mean that the PR can be merged with the main branch at only that particular moment of time and might change later. Therefore, any time the main branch changes, we must reevaluate the compatibility and functionality of the PR.

Merge queues make sure every PR is inserted into the queue and is merged in the right order. Merges queues ensure that all branches/PRs are merged using a specific process. That process is:

A pull request is approved by one or more reviewers
The pull request is added to the merge queue
When the pull request reaches the top, the branch/PR is updated to the main i.e. all CI checks are run again
The pull request is removed from the queue and is either merged or closed.

How organisations are solving the issue? 🤝

Github offers a native merge queue solution but currently it is only available in a limited public beta. This is potentially the best solution for GitHub users as it is implemented under the GitHub Actions umbrella. For now, we need to be one of the lucky ones selected for the beta to experiment with that feature.

If you are GitLab's premium user you need not worry about the merge queue because in GitLab it is already publicly available.

Mergify provides a merge queue feature that is easy to set up and configure to our needs. It's free-forever for open-source projects and comes with a 14-day free trial for private projects.

Mergify not only helps solves problems with merge queues but has many fascinating features which can nowhere be found in GitHub.

A technical deep dive 🚀

To understand the technical nuances in dept let's get our hands dirty and witness some of the features of Mergify.

In a big organization, solving merge conflicts is not the only problem we face. We must assign each PR for review to the right person, make sure it is reviewed before merging, set labels, rebase branches, backport PRs to different branches and the list can go on and on. You can use Mergify to solve all these problems. The amount of features Mergify provides, it is hard to fit in this one blog, so let's try out some of them. In this hands-on we will be:

adding checks before merging
merging automatically
using commands to instruct the Mergify bot
creating and using a merge queue

Let's start by creating a Github repository. I am naming it as Mergify, but you are free to do it as you want. Let us hit the particular URL to initialise our project.

https://github.com/new

Fill in the details and then click on the Create a Repository button.

Now, since we have our repository initialised let us get an account on Mergify. We get an option to start with our Github account directly with ease. When you click on the Signup button if you are new to the tool, it would redirect you to the auth page.

Once we are authenticated, we get access to the Dashboard. During the authentication we get an option to install the tool to all repositories or selected ones.

As we can see in the dashboard, the Merge Queues button helps us inspect the Pull Requests in the form of a Queue. The Editor Config option gets us started with a starter template of our configuration which we can customise according to our need. The Usage tab shows the option on who are the users linked with. The sidebar has options such as Application keys, where you can store your Private and Public keys as well.

Let us click on the button Add more repositories to add the repository we created.

Now, we can work on the Mergify repository and implement features according to our needs.

For the purpose of this hands on, let's build a basic React project to showcase the features. Once you have cloned the repository, change the working directory to the Mergify directory and then run the following command to create a new React application.

npx create-react-app .

Spin up the application on your localhost

npm start

If you find the below screen, you are good to go.

Now that everything is set up, let's start with Mergify.

The first step is to create a Mergify configuration for our repository. A simple configuration file could be something like this

pull_request_rules:
  - name: Automatic merge on approval
    conditions:
      - or:
        - "#files=1"
        - files=README.md
    actions:
      merge:
        method: merge

Let's understand this. We added a pull request rule which says Mergify should automatically merge all pull requests that have changes in any one file or in the README.md. This helps small changes to get merged quickly to the main branch.

We can use and and or conditions to make our rules more specific. Now to check if our configuration is correct, click on the blue button saying check my configuration. We can also add a PR to check it against which will give us more details on the rules created.

We created a branch called header and sent a PR to the main branch which adds a header text to the website.

Let's try checking our Mergify configuration against this PR.

It clearly shows on the right, that the rule matches since the PR only changed one file. After testing the rule we can easily add this configuration to our repository by clicking this button

If you check your github repository you will find a new PR from Mergify. Just merge the PR and your rule is added.

If you see carefully, once you merge the PR, the header PR also gets merged automatically. If you open the closed header PR, you will notice, Mergify has merged it for you.

This was a very simple configuration, but with Mergify you can add many more complex rules. The sky is the limit!

You can also instruct the Mergifyio bot to do certain tasks for you like updating the configuration file. Get a complete list of commands here

Lastly, let's look into the merge queue.
A simple usage could be to merge pull requests serially, ensuring all of them pass the CI one by one.

queue_rules:
  - name: default
    conditions: [] 

pull_request_rules:
  - name: merge using the merge queue
    conditions:
      - "#approved-reviews-by>=2"
      - check-success=Travis CI - Pull Request
    actions:
      queue:
        name: default

In the above rule, we define a queue named default which allows any pull request to be merged once it enters it. To enter the queue the PR must have 2 approving reviews and pass the CI. Once a pull request is in first position within the queue, it'll be updated with the newest commit of its base branch.

Similarly we can have multiple queues, for example urgent and default. This feature is although currently available only for premium users. You can add labels tp PRs to take them to different queues as shown below.

Outro 💚

The features of Mergify are endless. The purpose of this blog was to spark an interest in this amazing tool which can save you a lot of time and efforts. Check out all the features in the documentation. Don't wait too long, create your account and get started with Mergify. In case you have some questions regarding the article or want to discuss something under the sun feel free to connect with me on LinkedIn 💕

If you run an organisation and want me to write for you please do connect with me 🙈

Adding voice assistant to React Apps using Alan AI

Aniket Pal — Thu, 07 Apr 2022 06:06:54 +0000

The ultimate luxury is to have a personal assistant who listens to our needs and does them for us. Thanks to Artificial intelligence assistants aka voice assistants, this is actually possible! Voice assistants come as small packages and perform actions after hearing a wake-up word or command.

The speech and voice recognition market is currently worth billions of dollars. By 2025, it's projected to surpass $30 billion! Integrating voice assistant in an application enhances UX, ultimately increasing user retention rate.

If you run a startup or are building personal projects, adding a voice assistant will take your application to the next level. Companies like Alan help you integrate the voice assistant in just a matter of some minutes. In this article, we will integrate our Alan AI voice assistant into a React application. But, Alan AI does support a lot more frameworks and libraries which you should check from here and give it a try.

What are voice assistants? 💬

Voice assistants are devices/apps that use voice recognition technology, tongue processing, and AI to reply to humans. Using this technology, the device synthesises the user’s message, breaks it down, evaluates it, and offers a meaningful response in return.

Chatbot ❗️= Voice Assistant

Chatbots and voice assistants have a couple of things in common. Both are constantly learning more about their respective users, counting on newer data to enhance the standard of interactions. And both use automation to speak instantly and make the user experience (UX) convenient.

But, chatbots don't understand human emotions. Although chatbot answers accurately but they don't understand our moods or feelings and don't know how to respond accordingly. Voice assistants, on the other hand, can evaluate natural language complexities using advanced natural language processing.

What is Alan AI? 🤔

There are two categories of AI voice assistants: one is general purpose voice assistants and the other is bot voice assistants. Voice assistants like Siri, Alexa belongs to the first category while voice assistants which can be completely customized according to our need like Alan AI falls under the second category.

Alan is a complete end-to-end Voice AI Platform that lets you embed a contextual voice assistant into your existing application UI.

Alan AI enhances the experience of the application with an intelligent, engaging, and personalized conversational voice assistant. The best part is that there is no need to create spoken language models, train the speech recognition software, deploy and host voice components — the Alan AI backend does most of the work. With Alan, a voice experience for your app can be built and developed by even a single developer, rather than a team of Machine Learning and DevOps experts.

Alan allows you to transcend the capabilities of touch and sort interfaces and voice-enable any complex workflow or function in your app. Voice scripts are written in JavaScript, which makes them highly customizable and versatile . Alan runs on almost every platform. So without further ado, let's start with React + Alan AI.

In this tutorial we will be creating an web application that will tell the user about data structures. The user will be able to choose the data structure and get an introduction about it through both manually and through voice commands via Alan AI.

Let's setup the project ✨

Set up a new React project using create-react-app

npx create-react-app react-alan
cd react-alan

Let's run the app to check if everything is working.

npm start

If you can see the screen as shown below, you are good to go!

Now, let's add Alan Web SDK to the app. Here, we will be using the NPM package to install the module.

npm install @alan-ai/alan-sdk-web --save

Next we'll import alanBtn to the React application. So add this line to your App.js file.

import alanBtn from "@alan-ai/alan-sdk-web";

Now, let's add the AlanBtn to the React component in App.js.

import React, { useEffect } from 'react';

useEffect(() => {
  alanBtn({
    key: 'YOUR_KEY_FROM_ALAN_STUDIO_HERE',
    onCommand: (commandData) => {
      if (commandData.command === 'go:back') {
        // Call the client code that will react to the received command
      }
    }
  });
}, []);

Next, let's get your private key from Alan Studio to run the application. Sign up for Alan Studio from Studio Page. Once you are authorised you need to create a project by just clicking on the Create Voice Assistant button.

Create an empty project and name it anything (Example: reader).

Within matter of seconds you have your project ready to work with. On the upper right side of your screen you can find Integrations tab. Click on it to get your key and customize the speech by changing voice, time of reaction, microphone time out and so on.

Once, you get your private key, paste it in the key parameter in your App.js. If you see the screen as shown below, you are good to go.

And that's it! You have successfully setup React with Alan AI integration.

The next question you think have in our mind is what can we do with the integration?🤔 Let's build a web application to automate learning Data structures more fun and engaging. Let's build Qooma - A voice based Data Structures buddy. Upon a single click on the voice assistant it would take us through the routes and tell you what we want to know about the data structure. Qooma as a prototype has minimal features which gives you the opportunity to add more and make a useful end to end learning product with Alan AI.

This is how our project structure should look like:

Now let's create all the routes using react-router-dom in App.js. After adding the routes, this is how you App.js should look like:

import React, { useEffect } from "react";
import alanBtn from "@alan-ai/alan-sdk-web";
import Home from "./pages/Home";
import "./App.css";
import { BrowserRouter as Router, Routes, Route } from "react-router-dom";
import Array from "./pages/array";
import LinkedList from "./pages/linkedList";
import Stack from "./pages/stack";
import Queue from "./pages/queue";
import Graph from "./pages/graph";
import Tree from "./pages/tree";

const App = () => {
  useEffect(() => {
    alanBtn({
      key: 'YOUR_KEY_FROM_ALAN_STUDIO_HERE',
      onCommand: ({ command, category }) => {
        console.log(category)
        if (command === "home") {
          window.location.replace('/');
        } else if (command === "goto") {
          window.location.replace("/read/"+category)
        }
      },
    });
  }, []);

  return (
    <Router>
      <Routes>
        <Route exact path="/" element={<Home />} />
        <Route path="/read/array" element={<Array/>}/>
        <Route path="/read/linkedlist" element={<LinkedList/>}/>
        <Route path="/read/stack" element={<Stack/>}/>
        <Route path="/read/queue" element={<Queue/>}/>
        <Route path="/read/tree" element={<Tree/>}/>
        <Route path="/read/graph" element={<Graph/>}/>
      </Routes>
    </Router>
  );
};

export default App;

Now let's create a home page which shows all the data structures on cards, with links to their personal pages. To give the page basic styling, we'll be using Material UI.
Home.js:

import React, { useState, useEffect } from "react";
import { Container, Typography } from "@mui/material";
import Grid from "@mui/material/Grid";
import { Link } from "react-router-dom";
import "../App.css";

const Home = () => {
  return (
    <Container style={{ marginTop: "3%" }}>
      <Typography textAlign="center" variant="h2" component="h3">
        Learn about Data Structures
      </Typography>
      <Typography textAlign="center" variant="h5" component="h3">
        Try saying "Open stack"
      </Typography>
      <Grid container className="grid">
        <Grid item xs={4} className="gridItem">
          <Link to={"/read/array"} className="text">
            <Typography variant="h5" component="h5">
              Array
            </Typography>
          </Link>
        </Grid>
        <Grid item xs={4} className="gridItem">
          <Link to={"/read/linkedlist"} className="text">
            <Typography variant="h5" component="h5">
              Linked List
            </Typography>
          </Link>
        </Grid>
        <Grid item xs={4} className="gridItem">
          <Link to={"/read/stack"} className="text">
            <Typography variant="h5" component="h5">
              Stack
            </Typography>
          </Link>
        </Grid>
        <Grid item xs={4} className="gridItem">
          <Link to={"/read/queue"} className="text">
            <Typography variant="h5" component="h5">
              Queue
            </Typography>
          </Link>
        </Grid>
        <Grid item xs={4} className="gridItem">
          <Link to={"/read/graph"} className="text">
            <Typography variant="h5" component="h5">
              Graph
            </Typography>
          </Link>
        </Grid>
        <Grid item xs={4} className="gridItem">
          <Link to={"/read/tree"} className="text">
            <Typography variant="h5" component="h5">
              Tree
            </Typography>
          </Link>
        </Grid>
      </Grid>
    </Container>
  );
};

export default Home;

Let's add some more custom styling in App.css

body{
  background-color: #cee3e9;
}

.grid{
  margin: 50px 0px 50px 150px;
}

.gridItem{
  margin: 30px !important;
  text-align: center;
  max-width: 200px;
  min-height: 100px;
  padding: 40px;
  background-color: #fff;
}

.text{
  color: black;
  text-decoration: none;
}

.text:hover{
  color: rgb(71, 154, 223);
}

Now that we're done with the home page, let's create the pages for each data structure.
array.js:


import React from "react";
import {
  Typography,
  Card,
  CardContent,
  CardMedia,
  CardActionArea,
  Container,
} from "@mui/material";

const Array = () => {
  return (
    <>
      <Container style={{ marginTop: "3%" }}>
        <Typography textAlign="center" variant="h2" component="h3">
          Learn about Arrays
        </Typography>
        <Typography textAlign="center" variant="h5" component="h3">
          Try saying "Tell me about Array"
        </Typography>
      </Container>
      <Container
        style={{ display: "flex", justifyContent: "center", marginTop: "3%" }}
      >
        <Card sx={{ maxWidth: 500 }}>
          <CardActionArea>
            <CardMedia
              component="img"
              height="250"
              image="https://res.cloudinary.com/dqlh4eyt8/image/upload/v1648017909/james-harrison-vpOeXr5wmR4-unsplash_or1hrm.jpg"
              alt="code"
            />
            <CardContent>
              <Typography gutterBottom variant="h4" component="div">
                Array
              </Typography>
              <Typography variant="body2" color="text.secondary">
                In computer science, an array data structure, or simply an array, is a data structure consisting of a collection of elements, each identified by at least one array index or key. An array is stored such that the position of each element can be computed from its index tuple by a mathematical formula.
              </Typography>
            </CardContent>
          </CardActionArea>
        </Card>
      </Container>
    </>
  );
};

export default Array;

You can create similar pages for all other data structures you want to add like linked list, stacks, queues and so on.

Finally, let's add our voice script to Alan AI, to make our assistant customized. Add the following script to your project in Alan AI Studio.

intent('What does this app do?', 'What can I do here?', 
      reply('This app will tell you about the different types of data structures.'));

const CATEGORIES = ['array', 'linked list', 'stack', 'queue', 'tree', 'graph'];
const CATEGORIES_INTENT = `${CATEGORIES.map((category) => `${category}~${category}`).join('|')}`;
intent(`(go to|show|open) (the|) $(C~ ${CATEGORIES_INTENT})`,(p)=>{
    if(p.C.value.toLowerCase()=='linked list'){
        p.play({command:'goto',category:'linkedlist'})
    } else {
        p.play({command:'goto',category:p.C.value.toLowerCase()})
    }
})

intent(`(show|what is|tell me about|what's|what are|what're|read) (the|) $(C~ ${CATEGORIES_INTENT})`,(p)=>{
    if(p.C.value.toLowerCase()=='array'){
        p.play('In computer science, an array data structure, or simply an array, is a data structure consisting of a collection of elements, each identified by at least one array index or key. An array is stored such that the position of each element can be computed from its index tuple by a mathematical formula. ')
    } else if(p.C.value.toLowerCase()=='linked list'){
        p.play('In computer science, a linked list is a linear collection of data elements whose order is not given by their physical placement in memory. Instead, each element points to the next. It is a data structure consisting of a collection of nodes which together represent a sequence.')
    }else if(p.C.value.toLowerCase()=='stack'){
        p.play({command:'read',category:'stack'})
        p.play('In computer science, a stack is an abstract data type that serves as a collection of elements, with two main principal operations: Push, which adds an element to the collection, and Pop, which removes the most recently added element that was not yet removed.')
    }else if(p.C.value.toLowerCase()=='queue'){
        p.play('In computer science, a queue is a collection of entities that are maintained in a sequence and can be modified by the addition of entities at one end of the sequence and the removal of entities from the other end of the sequence.')
    }else if(p.C.value.toLowerCase()=='tree'){
        p.play('In computer science, a tree is a widely used abstract data type that simulates a hierarchical tree structure, with a root value and subtrees of children with a parent node, represented as a set of linked nodes.')
    }else if(p.C.value.toLowerCase()=='graph'){
        p.play('In computer science, a graph is an abstract data type that is meant to implement the undirected graph and directed graph concepts from the field of graph theory within mathematics.')
    }
})

intent('(go|) back', (p) => {
    p.play('Sure, going back');
    p.play({ command: 'home',category:'none'})
})

In this script, we are giving the assistant four custom features:

Answer questions like what does this app do?
Take you to any data structure page you want by saying 'open'. For example:'Open array'
Tell you about the data structures by saying 'tell me about. For example:'Tell me about array'
Go back to the home page

After writing all the code and adding the voice script, this is how your app should look like:

And you're done! Ofcourse you can write a much better optimized code, but as beginner tutorial, the code is kept as basic as possible. You can try playing around with the Alan assistant, ask questions about the data structures and even customize the app even more. The Alan Button is highly customizable according to your need. Starting from the button layout to the start command everything is documented really well and you can check it out here. Once, you are done playing around and configuring the button as per your need check out Script concepts to make it even more functional.

Outro 💚

The world of voice assistants is a big and fascinating topic. Players like Alan have helped users get on board quickly and with ease. It offers wholesome features with the power to make an impact. There’s no better way to test-drive Alan than by diving in and playing with it.

The purpose of the blog is to create awareness about Voice assistants and build a community of React developers who can integrate voice assistants into their applications. To learn further I would recommend you to go through Alan AI's official Documentation. If you enjoy using Alan AI do join their community on Slack to know more about them and even contribute to it. In case you have some questions regarding the article or want to discuss something under the sun feel free to connect with me on LinkedIn💕.

If you run an organization and want me to write for you please do connect with me 🙈

Getting into software testing with Jest

Aniket Pal — Tue, 01 Mar 2022 19:46:24 +0000

Starting from open-source projects to enterprise software, most softwares have a huge codebase. Don't believe me? Here is a fact! Some of the Google codebases have over 2 billion lines! Understanding and debugging each line manually is something that probably only Superman can do. So when contributing to a project, it is essential to keep in mind that your code doesn't disrupt the existing functionalities.

What is Testing? 🧪

In software, testing is the process of finding any gap, error, or missing requirements and verifying if it matches our needs.

Suppose you give an input in an electrical DC Machine. With your theoretical knowledge, you will have some expected output right? But in real life the output might be a bit different. So in testing, we generally determine the difference between the expected and the actual values and try to fix it as much as possible.

Software testing is divided majorly into 3 categories:

Unit Testing: testing a single function
Integrating Testing: testing a function that calls a function
End to End Testing: validating a DOM (i.e. we check if everything is in sync)

In this article, let's focus on Unit Testing. Why? Because it is easy to implement and very commonly used.

But how do we know what to test? 🤓

When it comes to testing, even a simple block of code could paralyse beginners. The most common question is "How do I know what to test?"

Suppose we're writing a web application, a good starting point would be testing every page of the app and every user interaction. But, web applications are also made of units of code like functions and modules that need to be tested too.

While writing code, there are mostly two scenarios:

You inherit legacy code which comes without tests
You have to implement a new functionality out of thin air

What to do? For both the cases we can think tests to be bits of code that check if a given function produces the expected result or not. Here's how a typical test flow looks like:

import the function to test
give an input to the function
define what to expect as the output
check if the function produces the expected output

Really, that's it! Testing won't be scary anymore if you think in these terms:

input - expected output - assert the result

What is Jest? ✨

Jest is a JavaScript testing framework powered by Meta. It focuses more on simplicity and support for large web applications. It is used for testing applications using Babel, TypeScript, Nodejs, React, Angular, Vuejs, and Svelte. Jest is one of the most popular test runners these days and the default choice for React projects.

Jest ships in the NPM package and you can install it in any JavaScript project by running:

npm install --save-dev jest

Let's see a demo 💥

Setting up the project

mkdir jestDemo
cd jestDemo

Now you are in your directory, so let’s initialise it with NPM.

npm init -y

The flag -y helps you initialise with all default values. Now, let’s install the jest NPM package.

npm install --save-dev jest

The project structure is very important, so let’s make it now.

For testing, it is essential to name the testing file with the name of your JavaScript file you want to test and concatenating the word test in between. In this demo we will be testing a script to subtract 2 elements. The script is written in subtract.js so the corresponding testing file will be subtract.test.js.

Open up package.json and configure a script named test for running Jest:

"scripts": {
   "test": "jest"
},

Now we are good to go😁 Let’s start with the scripting of subtract.js and subtract.test.js

In subtract.js:

function subtract(a,b){
   return a-b
}
module.exports = subtract

In subtract.test.js:

const subtract = require('./subtract')
test("Must subtract properly",() =>{
   expect (subtract(1,2)).toBe(-1)
})

And that's it! Now let’s test it.

npm test

After the test, it gives you an output showing the status of the code and comparing it with the actual result and the specified expected value. You will get an output similar to

To get a more detailed and structured visualisation of you tests run:

jest --coverage

Jest coverage command gives a more detailed analysis where the test fails and code can be improved accordingly.

Outro 💚

Testing is a big and fascinating topic. There are many types of tests and many libraries for testing. In this Jest tutorial, you learnt how to configure Jest for coverage reporting, how to organise and write a simple unit test, and how to test JavaScript code. There’s no better way to test-drive Jest than by diving in and playing with it.

The purpose of the blog is to create awareness about Jest and similar testing tools. To learn further it is recommended to go through the Jest's official Documentation. In case you have some questions regarding the article or want to discuss something under the sun feel free to connect with me on LinkedIn 💕

If you run an organisation and want me to write for you please do connect with me 🙈

Understanding neural search framework: Jina AI

Aniket Pal — Mon, 28 Feb 2022 19:29:26 +0000

Search is a big business and is getting bigger each day. Just a few years down the line, searching meant typing text in a search bar. With advancement it now allows us to search for audio, video, images, and many more. Just before the turn of the millennium, there were just 3.5 million Google searches per day. Today that figure is around 5 billion searches per day.

Trust me or not Google search is dying. If you have ever googled about recipes or even some blogs recently, I don’t need to tell you that Google search results have gone to shit. You would have already noticed that the first few non-ad results are SEO-optimized sites filled with affiliate links and ads. If you don't believe me give a read on what happened in Norway 🇳🇴 Here comes the need for a better search engine.

Just a bit about Symbolic Search ⚜️

Symbolic search works better when you want your search to be fully customized. Google search is a general-purpose search engine, which cannot be used just everywhere. If you have a team of developers who has the skill and time to jot down each parameter Symbolic search can be taken to heights. Say, you want to build a blog site, where an author pays the team to show the blog on top so you might rank according to the need and the money flow goes well.

The hell lot of customization in indexing products, searching users, implementing filters, and sorting along with more comes with a high cost. You Have to Explain Every. Little. Thing and moreover Text is fragile.

Therefore, If you are someone who plans to create the next search engine with more accurate results with fewer ads or affiliate links OR You are someone who wants to add search in your application be it in production or development you need to know this. According to reports, Google started using deep neural networks in the search engine in 2016. If you want to disrupt the industry with your build, I suggest you check out Jina. Let's get started.

What is a neural search? 🤔

In short, a neural search is a new approach to retrieving information. Instead of telling a machine a set of rules to understand what data is what, neural search does the same thing with a pre-trained neural network. This means developers don’t have to write every little rule, saving them time and headaches, and the system trains itself to get better as it goes along.

Where can it be fruitful? 🧐

In the early internet days, it was really amazing for firms to build search engines that use text as the parameter and correspondingly pull and rank URLs from the web. With the recent advancements in Deep Learning, a neural search system can go beyond simple text search. Let me take you through a not so techy situation.

You are sitting in a fancy restaurant and hear some beats which you like now you want to search for beats and songs which are similar to what you have heard neural search comes into play. The very next moment you like the shinny tree upfront and you want to have it in your house, just click a photo and search it, you get the aptest results from it.

Starting from getting most similar furniture, garments, songs, memes it can be used by any business and I can bet it is the future for search!

How I came to know about Jina? 💥

Before starting with what is Jina AI, I thought sharing this would make this more interesting. Remember the time when Money Heist was releasing their final season and some companies gave holidays to employees to bid goodbye to the amazing series? Jina was the first startup to do so, according to my knowledge.

I was amazed by the company culture and went straight to their career portal without knowing about their product, but at a point in time, they were not hiring interns. After visiting their GitHub I thought it to be an alternative to Elastic until I used it in my side project.

I wanted to build a small search engine for one of my software, with having 0 knowledge of AI I used K-means clustering to build a basic AI model. After resolving a hell of lot of bugs with integrations and model training I was waiting with an inefficient text search bar 🙂. Then was the time I went straight to elastic and then Solr but at that point of time I wanted something different, thus went for Jina AI and boom. The quick implementation, cloud-native approach, and accuracy made me a fanboy! This blog is just an appreciation post for the amazing open-source project.

What is Jina? 🚀

Jina AI is a cloud-native neural search framework for any kind of Data

Jina is an approach to neural search. It’s cloud-native, so it can be deployed in containers, and it offers anything-to-anything search. Text-to-text, image-to-image, video-to-video, or whatever else you can feed it. It empowers anyone to build SOTA and scalable neural search applications in minutes. Jina runs on 3 fundamental concepts Document, Executor and Flow.

What makes Jina different from other players

Local & Cloud friendly 🌥: From the very beginning it had distributed architecture, scalable and cloud-native. Ultimately, which gave the same DX on local, Docker Compose, and Kubernetes.
Server, Scale, and Share 🚀: It helps you to scale your applications to meet availability and throughput requirements. Serves a local project with HTTP, WebSockets or gRPC endpoints in just minutes.
Time Saver ⏱: Let's you quickly build solutions for indexing, querying, understanding multi or cross-modal documents. The document is the basic data type that Jina operates with. Video, image, text, audio, source code and PDFs are the types of document.

Own Stack 🍱: Jina empowers to keep end-to-end ownership of your solution. Avoid integration pitfalls you get with fragmented, multi-vendor, generic legacy tools. Move over, Jina has out-of-the-box integrations with cloud-native ecosystems.

Outro 💚

There’s no better way to test-drive Jina than by diving in and playing with it. Jina's team provides pre-trained Docker images and jinabox.js, an easy-to-use front-end for searching text, images, audio, or video. The purpose of the blog is to create awareness about Jina and similar neural search frameworks. To learn further it is recommended to go through the Jina's GitHub and Jina's blogs. In case you have some questions regarding the article or want to discuss something under the sun feel free to connect with me on LinkedIn 💕

If you run an organisation and want me to write for you please do connect with me 🙈

PWA: Learning, Implementing and Loving ft. NextJS under 2 mins

Aniket Pal — Thu, 17 Feb 2022 04:44:46 +0000

When I started my web dev journey, I was always overwhelmed by the concept of transforming a web application to mobile app. Being a self taught developer I was pretty clueless on how to do it. It took me 3 months to realise the upscale advantage and the methodology behind it. I believe it's time to give back to the community. So let's learn it all in just 2 mins starting now!

Understanding PWA ✨

If mobile apps and the web had a baby, that baby would be PWA. At its core, PWA helps businesses build web applications with the look and feel of native mobile apps but using web technologies like JavaScript, CSS, and HTML. PWA offers features such as push notifications, offline support and much more. Progressive Web Apps are secure by default. The technologies powering PWA require apps to be served over HTTPS to work. Companies are adopting PWAs which are reducing time in developing, testing and maintaining applications.

Gartner predicts, up to 20% of companies will abandon their native mobile applications. Instead, they believe that PWA will become a more viable alternative to them.

In fact, several companies, like Alibaba, have shown that by simply focusing their efforts only on a PWA app instead of having separate native apps increased conversions and average purchases. Since, our blog promises on building PWA with NextJS in 2 minutes let's jump directly into it.

Time starts ⏳

Over here, I will be starting from scratch but you can definitely add PWA to your existing project.

yarn create next-app --typescript

We have started with the basic NextJS starter with typescript support. Now, let's add the dependency which does the work for us.

yarn add next-pwa @types/next-pwa

Now, we need manifest.json to specify the meta data about the application.

{
  "name": "PWA Application with Next",
  "short_name": "PWA",
  "icons": [
    {
      "src": "/192x192.png",
      "sizes": "192x192",
      "type": "image/png"
    },
    {
      "src": "/384x384.png",
      "sizes": "384x384",
      "type": "image/png"
    },
    {
      "src": "/512x512.png",
      "sizes": "512x512",
      "type": "image/png"
    }
  ],
  "theme_color": "#000000",
  "background_color": "#000000",
  "start_url": "/",
  "display": "standalone",
  "orientation": "portrait"
}

You can use this to generate manifest for your application or copy the above manifest. You need 3 images of dimensions 512 x 512 , 384 x 384 and 192 x 192 which you can generate from here. Place all these files - images and manifest.json into the public directory.

Let's create a _document.ts page in the pages directory. You can add the code in the file.

import Document, { Html, Head, Main, NextScript } from "next/document";

class NextDocument extends Document {
  render() {
    return (
      <Html>
        <Head>
          <link rel="manifest" href="/manifest.json" />
          <link rel="apple-touch-icon" href="/192x192.png"></link>
          <meta name="theme-color" content="#000000" />
        </Head>
        <body>
          <Main />
          <NextScript />
        </body>
      </Html>
    );
  }
}

export default NextDocument;

We are almost done!🥳 Let us finally add our new configuration in next.config.js.

/** @type {import('next').NextConfig} */
const withPWA = require("next-pwa");

const nextConfig = {
  reactStrictMode: true,
};

module.exports = withPWA({
  pwa: {
    dest: "public",
    register: true,
    skipWaiting: true,
  },
  nextConfig,
});

Congratulations!🥳 You were really quick in building your PWA application. Hopefully you were able to do it within 2 mins and faced no errors in the process.

Let me give you a bonus on how to add offline support to your application. Create an _offline.tsx file in pages directory. And replace with the particular code:

import type { NextPage } from "next";

const Offline:NextPage = () =>{
   return (<h1>You need to check your internet connection 😢 </h1>);

}

export default Offline;

Make sure to add some awesome UI elements and make the offline page more interactive rather than just a statement. Below I have added one of my open source contributions which was to add PWA feature with offline support in Fission which focuses on building Serverless Functions and Workflows for Kubernetes.

Incase you have faced an error and your application is not working or you just want to interact with me feel free to reach me out on LinkedIn 💕

Acing the MLH Fellowship selection rounds

Aniket Pal — Wed, 02 Feb 2022 06:28:42 +0000

Since my selection, I have received a lot of queries regarding my MLH Fellowship journey and tips and tricks to get into the fellowship. Here, is an attempt to answer those queries and express my gratitude to this amazing program.

The start for me 🙈

The first hackathon which I took part in was Hack The Mountains which was powered by MLH. I was in my sophomore year when I first came to know about MLH and then it took me another 6 months to know about the MLH Fellowship. But don't worry, you don't have to wait that long!🙈

How MLH Fellowship started 🧐

Due to the pandemic, many students in the community lost their internships and jobs. So, MLH launched an educational program called MLH fellowship to help student developers get hands-on experience. The MLH Fellowship is a remote internship alternative for aspiring technologists. You can spend 12 weeks for main fellowship or 3 weeks for prep fellowship while building your skills by collaborating on real-world projects. The fellowship is powered by GitHub, Meta, Adobe, AWS, American Express, Indeed and many other industry players.

The Application Process ✨

The first step is to check if you're eligible for the fellowship which you can check from here. Once you are done, you can proceed to the initial application. From this point onwards, I'll be giving a lot of tips and tricks that are solely based on my thoughts and experience.

The initial application tests you both technically and intellectually. You will be asked to submit a code sample, and I would recommend you focus more on the quality of the project because you will be asked questions based on it in the following interviews. You must make sure not to submit someone else's code. The project you submit should be unique in terms of idea or implementation. Submitting one file code or algorithm implementation of a particular DSA problem would not be recommended. It is NOT necessary to submit a project in the application if you have an awesome Pull Request, it works. The admission team is more interested in knowing what you have learned in the process and how it is unique. A lot of people wanted to know the project I had submitted so here you go, to know more about it refer here.

Aniket762 / playground

Play around with code while we make sure you aren't lost!

Playground

In India, more than 95% of schools don't allow smartphones in class. Being a Computer Science teacher becomes very difficult as it is nearly impossible to check the handwritten code of each student line by line. Also talking in India, in most schools, the students don't have the privilege to have one computer each for them so that they can run their code on their own. So it becomes a nightmare for the teacher as they might have to manually type each code of each student and run it or take the risk of missing out on bugs. So we came up with Playground to make this job easier. Another reason for creating Playground was that once we entered college we understood the importance of coding. Coding not only helps you in a Software job but also helps increase logical reasoning and problem-solving ability. But many students don't have…

View on GitHub

The code sample shows your technical expertise and the behavioural questions focus on showing your interest in the program and your enthusiasm for the community.

Break down the essay questions and write things that show your interest in the program, what you are bringing to the table, what makes you different, and similar experiences if you had any. Make sure your answers show a lot of enthusiasm and answers the question why you should be selected out of all the candidates. After 2 to 3 edits of your answers, I am sure you can make it to the interviews. Jumping straight to the phone screen round.

Phone screen round 📱

Make sure you have proper audio, video, and internet connection. The interview starts with basic eligibility questions and then takes you to more generalised questions. Make sure to convey the reason why you want to be in this program and answer every question with confidence. Remember, you know it's a good interview when it feels less like an interview and more like a conversation. Phone screening round is more like an HR round which may also include some layered tech questions such as the project which you are most proud of and why. Preparing HR questions can be at times tricky I would suggest checking out DanLok's YT. Once you pass this interview, you move on to the last and final round, the technical round!

Technical round 🧑‍💻

This is the most important round and decides if you will be a fellow or not. The technical round has no HR question and it will be a code conversation as the team calls it. First, you are asked to share you screen and give a demo of the project you submitted. Then you would be asked to go through the code and explain some parts of it while also answering some questions based on the code. You need NOT write any code while sharing your screen, but you should know every line properly along with the reasons why you are using a particular module or tech stack. For example, you submitted a simple blogging website and used NextJS for the front-end, so you might face questions ranging from the reason for choosing Next over any another framework to complex conceptual questions in the area depending upon the interviewer. You might be asked your future plans with the code sample like how you plan to scale your application or what new features you would like to add. So make sure to know each and every thing about your project.

Conclusion 🤲

I have worked in several organisations starting from leading a campus club to mid-range product-based startups and also got the opportunity to work with one of the finest tech companies in the world. Relating those with my MLH interviews, it felt pretty easy and comfortable, because of a couple of reasons:

MLH has one of the most amazing sets of interviewers who will make you very comfortable the moment the interview starts.
The questions asked in the interviews are not on business logic or to trap you, all they want is to check your enthusiasm for technology and the entire program.
The technical interviews varies depending upon the interviewer and the project. Talking about my experience, the questions I faced were easy for me. Since it was my project and I knew every single line in it, so I was very confident with every question asked. Moreover, I did a couple of mock interviews with Sumana Basu which helped me gain a lot of confidence and is something I would highly recommend.

So that's all the tips from my side. If you still have any queries or need any help with the application, feel free to reach me out on LinkedIn or drop a comment 💕

Lastly, a huge shoutout to the 14 highly enthusiastic coders I met during the fellowship! 🥳

Urge to build CLIs : Beginners Eye🚀

Aniket Pal — Mon, 17 Jan 2022 07:28:13 +0000

I will first take you to the reason and urge I have for building CLI tools and then make your hands dirty by building a CLI-based calculator with Cobra! In case you are well motivated to build CLIs, I would suggest you directly jump to the code valley.

The urge to build CLIs 👨‍💻

I find it very difficult to install or run an application while switching between various GUI components in the OS, be it Windows/ Mac or Unix. The commands always follow a particular logic, be it any OS for example using sudo gives us admin privileges, using -v gives us the version of the application, and so on. Moreover, GUI does not have the same level of functionality and granular control as a command-line interface. So the CLI provides greater flexibility of use. A GUI needs to use additional system resources to load the graphical part thus it is going to be slower than using the command line. With a CLI, users have all the control over the file system and operating system, and the tasks become simple. You can create a script that contains a few lines of command and it will do the work for you. Although learning bash has a steep curve because of its ugly syntax, the only datatype is a string but the automation it provides is worth it.

Relating Web with CLI tools 🕸

Having experience in building backend applications for 2 years, I feel the way CLI works is very similar to that of building full-stack web applications. For instance, a simple CLI tool to generate Temperature at a location requires just passing the location as a prop and then doing API calls to fetch the data and display it on the CLI.

Building Simple CLIs 🚀

Initially, I started creating simple CLI tools such as joke generator, fuzzy calculator, code compiler, reminder CLI tools, and some more. The one CLI tool which particularly made me learn a lot of tools was while developing a network CLI tool for Domain migrations within various hosting providers. In the particular CLI, I used Urfave CLI to get started with the CLI structure which made naming labels easier and generating new commands simpler. Urfave also has various classes within it which make overriding functions easier for example creating CLI flags or creating commands. The network CLI had basic functionalities such as when hostname is passed as a parameter it returns name servers, IP address, and CName for that particular host when passed with a particular hostname. A simple but according to me effective CLI tool. Comparing Urfave and Cobra, Cobra uses more template code which makes execution of the programs slower. Although a novice in building CLI applications, I am highly enthusiastic about learning and exploring more and more in this field.

While going through the web I created a repo to learn Go lang and build CLIs I suggest you look into the Code Valley. If you want me to write a blog on how to build a CLI tool with proper explanation let me know in the comments 🐣

Transform your smart devices to intelligent communicators

Aniket Pal — Wed, 29 Sep 2021 20:20:26 +0000

Ever built a product which you yourself used the next day? Well, I did! I had a Frenectomy and I was restricted to speak for the next 48 hours. That time, Donna came to the rescue!

betaoverflow / donna

Transform your smart devices to intelligent communicators.

Donna

According to a survey of 601 people, 57.9 % of participants experienced communication difficulties, with 23.5 % reporting severe difficulties. Only 75.1 % of participants communicated verbally; more than half found communicating with professionals and non-familiar partners difficult. Living in the privileged galaxy of intellects and witnessing a technical revolution in everyday life it is imperative not to overlook the responsibility to utilize technology to contribute to the progress and development of society at large. Communication is the fundamental basis for any individual to live a normal life. People facing trouble in communicating with others can often be a victim of social isolation. For example, most people who are hearing disabled are also speech disabled. In an extensive research conducted in various domains, it was found that hearing impairment and inability to verbally express oneself leads to lack of equal opportunities as well as leads to problems even in…

View on GitHub

Communication is the fundamental basis for any individual to live a normal life. People facing trouble in communicating with others can often be a victim of social isolation. For example, most people who are hearing disabled are also speech disabled. In an extensive research conducted in various domains, it was found that hearing impairment and inability to verbally express oneself leads to lack of equal opportunities as well as leads to problems even in everyday communication.

We wanted to contribute to help this cause so we came up with Donna. For all the suits fans, you know who Donna is. For those who don't, Donna is a strong female character from a famous T.V. show called Suits. She was one person who everyone could count on and whatever the situation might be she was always there for her dear ones. With all the difficulties specially-abled people feel, we hope to never let them feel lonely and helpless having Donna by their side.

How it works 🤓

Donna is an application that always has your back. Through this hack, we want to make sure specially-abled people never feel helpless. Donna supports people with disabilities by allowing them to communicate. It has special cards for a lot of statements people might normally use and need in day to day life routines. Moreover, we have a facility for storing an identity card, emergency contact and other important information to make them feel even more secure. There is also a provision of typing your own statements and using them to communicate with others. The app uses the flutter_tts package for text to speech conversion. Both the web and app is created on flutter along with firebase for user authentication. Although Donna can help a lot of people, we especially made this to help the deaf people who find it really difficult to communicate with the outside world which leads to them not opening up to new people.

How we built it 🧑‍💻

We used flutter to build the frontend and Firebase as our backend service for authentication and database. We used a flutter package to convert text to speech. We added the feature in which the users can create personalised voice cards (a card on clicking which converts the text to speech) through a form. These custom cards are stored in the database for user convenience and are available upon login.

Donna was built in Ability Hacks powered by Abilify, USA in which we were the winner but there are a lot of features we have planned and some which need the support of community.

What's next for Donna 🔥

Next, we want to add more and more categories and functionalities in the app to try and put all the phrases that might be required in day-to-day activities. We also wish to add hospital support and on-call emergency services in the future. In addition to that, we want to make sure, we take care of their mental health as well, so we would also be adding a mood tracker which tracks a person's happiness index every day and helps them make the graph go up!

Support us to empower everyone 🚀

Taking this prototype to production needs your help. The code is open-sourced and we would love to see your contribution:

Check the Github Repo 😎
Fork the Repository 🍴
Star the Repo ⭐️
Send the PR or create an issue 🤓

Incase you like the project and want to help our community drop a star to increase the visibility. And, if you want to discuss more, would love to have a chat with you on LinkedIn