DEV Community: Muhammad Wasi Naseer

Microservices Communication Pattern

Muhammad Wasi Naseer — Wed, 09 Mar 2022 19:47:54 +0000

Microservice is a single application in a suite of small services. Each microservice communicates via a lightweight protocol e.g HTTP, Message Queues. In accordance with the database per service pattern, each of them has its own database. They need to communicate in order to access others data.

Basically, there are two patterns that microservices can use to talk to each other.

Synchronous Communication

In this communication, a microservice communicates via another microservice's API endpoint and waits for its response. The call will be blocked until the response is received. It's easy to understand and implement but it comes with its own drawbacks e.g high latency, downtime, etc.

Asynchronous Communication

A microservice sends a message to another microservice via a message queue and does not wait for the response. It's a non-blocking call. There are a lot of benefits of implementing the asynchronous pattern. Let's discuss some of them here.

Benefits of using Asynchronous Communication

Reduced Coupling

A sender does not need to know about the receiver interface. They can communicate via standard message queues. It increases decoupling in code.

Multiple Subscribers

In asynchronous communication, microservices communicate via message queues. The message can be consumed by multiple subscribers using the publisher/subscribers model.

Failure Isolation

When a microservice goes down, the other microservices can do their work and write messages onto the queue. The failed microservice can read them whenever it will be up and running. The message queues ensure fault tolerance and message reliability.

Responsiveness

Response time will be reduced much if we do the bare minimum work synchronously in the HTTP call sent by the client and do the rest asynchronously and notify the client about the updates..

Drawbacks of Asynchronous Communication

Coupling with Message Infrastructure

Since we use message queues a lot in asynchronous communication, there is a high chance that our code can be coupled with message infrastructure.

Complexity

Handling asynchronous messages is quite complex. We need to deal with duplicated messages and make the request idempotent. We must identify the cost of consuming multiple messages. There are multiple patterns of handling messages e.g at least one message, at most one message, and only one message.

Transaction Management

Dealing with transactions in asynchronous communication is a huge topic to discuss on its own. What if something fails in the transaction then how we can revert the transaction.

To conclude, there is always a tradeoff when we use any pattern. We must identify systems requirements and choose based on them.

Thank you so much for the reading. Please checkout my blog for more content.

References:

Why do we need threads along with processes?

Muhammad Wasi Naseer — Mon, 20 Dec 2021 19:45:03 +0000

Let’s first define what a process is. A process is an independent isolated program to which an OS gives resources, file handles, or credentials. An OS can have many processes with its own address space, heap, or stack. When processes need to communicate, they need to use coarse-grained communication mechanisms such as a socket, shared memory, files, etc. Nothing is directly shared among processes.

A process can have many threads. A thread is like a lightweight process but with a major difference of sharing state. Threads share process-wide resources like files or memory which enables fine-grained communication. All threads within a process have their own stack, program counter, and local variables except heap. This sharing of state among multiple threads leads to concurrency problems which we’ll discuss in our upcoming posts.

Along with the benefits of multiprocessing(resource utilization, convenience, fairness), threads enable us to use the same process-wide resources without using any communication mechanisms.

REFERENCE

Topic#1.1: https://www.amazon.com/Java-Concurrency-Practice-Brian-Goetz/dp/0321349601

Isolation Levels in Transaction

Muhammad Wasi Naseer — Fri, 17 Dec 2021 21:24:01 +0000

As we have already discussed in the article, isolation specifies that each transaction should be executed independently of other transactions. The level specifies how much isolation can be provided to each transaction.

Read Uncommitted

It is the lowest level of isolation in a transaction. At this level, data changed during a transaction, which is essentially uncommitted data, will be directly visible to other transactions.

It can lead to a Dirty Read problem because the data visible during the transaction can either be committed or rollbacked in case of failure.

Read Committed

In the read-committed transactions, data will only be visible to read after the transaction has been committed.

This level may seem okay at first but it has a problem of Non-repeatable Read for other transactions. During a course of a transaction, a non-repeatable read occurs when a row is fetched twice and gives different results each time. The reason is simple. It is because, at the first read, the other transaction didn’t commit the row updates, and on the second read, the update has been committed. This problem can be resolved at the next level by acquiring locks in each transaction.

Non-Repeatable Read

We know that a non-repeatable read occurs when two transactions are executed at the same time. And, one transaction commits a result which the other transaction has already read and on the second read, it gets the committed result.

The problem can be resolved when any transaction has to acquire a read lock if it’s only reading data, and write lock if it needs to write data. In the article Read Lock vs Write Lock, the locks have already been discussed in detail.

Let’s assume two transactions A and B. Transaction A wants to update a row. It has to acquire a write lock which prevents other transactions to acquire a read/write lock. Then transaction B also wants to update some other row but it first needs to query the row that will be changed by transaction A. In order to do that, transaction B has to acquire a read lock which it cannot acquire as transaction A has already acquired a write lock on it.

In short, the locks prevent the problem of non-repeatable read. Hence, this isolation level is called a Non-repeatable read.

However, at this level, we have a new problem known as Phantom Read. A phantom read occurs when we execute a query two times during a transaction and each time we get a different number of rows. It’s because some rows have either been inserted or deleted. In non-repeatable read, we get the same number of rows but the rows have been updated because of update query.

Serializable

It is the highest level of isolation in transactions. Instead of holding the lock on a row, it holds a lock on multiple rows satisfying a WHERE condition. Here, we are holding the range lock.

When we hold a lock on the range then we can avoid the phantom read problem because the newly inserted rows satisfying the condition have to wait for releasing the lock and the same goes for deletion.

Transaction in Database

Muhammad Wasi Naseer — Thu, 16 Dec 2021 20:33:06 +0000

A transaction is a unit of work that must maintain ACID properties. ACID is an acronym of Atomicity, Consistency, Isolation, and Durability

Let’s look at each of them in detail.

Atomicity

A transaction contains a query or a set of queries. In order to maintain atomicity property, the transaction either has to execute all the queries or none of them. In short, all the queries will be treated as atomic. If any of the queries fails, the transaction has to roll back all the previous executed queries.

Consistency

First, let’s understand what a consistent state means in a database. A database is in a consistent state if it follows all the rules defined using constraints, triggers, or cascades.

A transaction must always lead the database to a consistent state.

For example, if a table user has a unique constraint on the national_id column and a query in my transaction tries to insert the duplicate national_id, the query has to be failed. Otherwise, the database will not be in a consistent state as it will violate the unique constraint on our user table.

Isolation

All transactions must be executed independently of other transactions. Each transaction will be executed as if it’s the only transaction executed in the database.

There are multiple levels of isolation levels that we can define in a database. We have discussed them in detail in our this article.

Durability

If a transaction commits data, it must be able to persist them even if the system fails or restart. In short, committed transactions means the data has been persisted, and if it’s not able to save the data due to system failure, it must persist at the system start.

Read Lock vs Write Lock

Muhammad Wasi Naseer — Tue, 14 Dec 2021 22:42:17 +0000

A lock is a synchronization primitive which limits access to a shared resource among multiple threads. We don’t need locks when only one thread is running or there is no shared resource because lock is always acquired on a shared resource. Let’s look at what locks can be acquired.

Read Lock

Multiple threads can acquire read lock at the same time. However, while holding the lock, they can only read the data. Also, no other thread can update the data while the read lock is acquired.

In other words, during the period of holding the read lock on data, it’s guaranteed that the data haven’t been changed by any thread.

Write Lock

Write lock can be acquired by only one thread at a time. During the lock holding period, the thread can read or write data. If a write lock has been acquired, no other threads can read the data or acquire a read lock on the data.

In short, during the period of holding the write lock on data, it’s guaranteed that the data haven’t been read or changed by any thread.

Write lock is more strict than the read lock and ensures better synchronization. In a multithreaded application, we usually go for the read lock if we’re only reading data and taking decisions based on the read data. That’s why we need to ensure that whatever we have read for taking the decisions must be true until the processing has been done and a response has been sent to the client.

What are SOLID principles?

Muhammad Wasi Naseer — Mon, 13 Dec 2021 20:46:47 +0000

SOLID is an acronym of below five object-oriented design principles.

S – Single Responsibility
O – Open Closed
L – Liskov Substitution
I – Interface Segregation
D – Dependency Inversion

Single Responsibility Principle

Every entity (class, method, or module) should have a single responsibility. In other words, every entity must have a single reason to change.

Open Closed Principle

Every class should be Open for extension and Closed for modification.

For example, if there is a method to read a file and the filename is hardcoded in the function, we need to change the method whenever the file name changes. Instead of hardcoding the filename, we can pass it via a parameter in the method.

However, there is an exception to fixing bugs in the code.

Liskov Substitution

In simple words, it is to replace the supertype object with the subtype object and the behavior is not undesirable then this test has been passed.

If S is a subclass of T, then the object of T can be replaced with the object of S without getting any undesirable behaviour.

For example, if we create a class Rectangle with properties width and height then we create its subclass Square which sets the width and height the same whenever we update any of them. When we added the constraint in the square, it leads to undesirable behaviour when we substitute the square object for the rectangle object. In this example, it did not pass.

class Rectangle {
    Integer width;
    Integer height;
}

class Square extends Rectangle {
    public void setWidth(Integer width){
        this.width = width;
        this.height = width;
    }
    public void setHeight(Integer height){
        this.width = height;
        this.height = height;
    }
}


/** 
Let's create a Rectangle object (supertype)
Rectangle r = new Rectangle();
Now checking Liskov Substitution by replacing Rectangle object with the Square object.
The square objects ensures the constraint that width and height must be same. 
However the reference type is rectangle. Hence, the liskov substitution test has not passed.
**/

Rectangle r = new Square();
r.setWidth(5);

Interface Segregation

An interface should only have those methods which can be implemented by any client. In other words, no client should be forced to implement methods that it does not use. If that’s the case, we should divide the interface into two or more interfaces so that every client implements only those methods that it’s using.

Dependency Inversion

Whenever we have a dependency on objects, they should be passed via the constructor. We should not create the objects and make the class tightly coupled with the objects. High-level modules should not depend on the low-level modules, instead, they should rely on abstractions.

It also helps in unit testing. We can mock the dependencies by providing their mocks.

Why Strings are Immutable in Java?

Muhammad Wasi Naseer — Sun, 12 Dec 2021 21:27:48 +0000

Before understanding why strings are immutable in java, we need to think about why do we make something immutable?

Immutable means once created we cannot change it. The one reason that we can think of making anything immutable is synchronization when it’s shared. It’s the same reason that strings are immutable.

In Java, String objects are shared and cached in String Pool. It’s a designated place in a heap where strings are stored and shared among multiple threads if they have the same value. For example: In String Pool, if there is already a string with the value “test” and the program wants to create another string object with the same value then it will get the same reference instead of creating a new string object. Now we know how strings are stored in heap. Let’s look at the reasons why they are immutable.

The first reason is to make it thread safe. Since strings are shared among multiple threads in string pool, we need to restrict any random thread to change it. Any change in a string can be impacted to other threads which are accessing the same string. If a thread wants to update the value of a string, it needs to create another string and takes it reference.
Usually, we use String as a key in Map. If Strings were mutable, anyone can change the value of the strings, and we would lose the actual key.

Memory Management in Java | Heap vs Stack

Muhammad Wasi Naseer — Sun, 31 Oct 2021 11:41:08 +0000

In Java, a variable can either hold a primitive type or a reference of an object. This holds true for all cases. Primitive types are stored in stack and objects are stored in heap. To store objects, it needs to maintain their structure.

We’ve always heard that a variable can be passed by value or passed by reference. But, essentially, in Java, there is no such thing as pass-by-reference. It always passes variables by value. But, you need to see what the variable holds. If it holds primitive type then we call it to pass by value or in the case of the reference of an object, we call it to pass by reference. However, it always passes the copy of whatever it holds.

Remember that whenever a variable is passed, its copy will be passed. And, if the variable contains the pointer of an object then the copy will also contain the same pointer. So, if we change anything in the referred object, it will also reflect all variables that hold the same pointer.

Final Variables and Const Correctness

Final variables in java are not essentially final. I know we cannot change the value assigned to the variable but if you assign a reference to an object. Then, we can change the referred object without changing the reference which the final variable holds. This is against the concept of const correctness.

Const correctness is basically the concept that if you pass a final variable to a function and the function is able to modify the state of the referred object. Then, the language lacks const correctness.

What is static and non-static in OOP?

Muhammad Wasi Naseer — Sat, 21 Aug 2021 22:34:51 +0000

Object Oriented Programming is all about classes and objects. We first create a class in which we specify two things:

data (variables). We use it for storing data
behaviors (functions). We use it for manipulating/retrieving the variables.

We then create object of a class by using the new keyword.

The variables or behaviors can either be accessed at class level or object level.
The class level access means that we can directly access them using the class name. For example:


 int maxRoomInHouse = House.MAX_ROOMS;

The object level means that we can only access them using the object of the class. The object level access is the default access of variables or functions unless we specify static.


House myHouse = new House();
int floors = myHouse.getNumberOfFloors();

We can make a function or variable to be accessed at class level by specifying static. We don't need to create an object to access static variables or functions.

When to use static and non-static?

We should use static variables or functions wherever we need global common variable or behavior across all the objects of the class. For example, in our house example, the House.MAX_FLOORS is the common field that we will be same for all objects of the House class, and same goes for House.getMaxFloors() function.

We should not specify variable or function static whenever the function or variables belongs to each individual object. For example, in myHouse.getNumberOfFloors() is basically specifying how many floors does myHouse object has?

Difference Between Aggregation and Composition

Muhammad Wasi Naseer — Mon, 09 Aug 2021 20:09:31 +0000

Aggregation and Composition are basically the concepts about how to manage objects.

Aggregation

Aggregation uses loose coupling, which means that it does not hold the actual object in the class. Instead, the object is being passed via getter or setter function. It gives the benefit that when the object of the parent class dies then the child class object remains intact.

class House {
 private Table table;

 public void setTable(Table table){
  this.table = table
 }

 public Table getTable(){
  return this.table;
 }
}

public class main(String[] args){

 Table table = new Table();

 // this house' table is set externally and will not be deleted 
 // upon deleting this House object
 House house = new House();
 house.setTable(table);

}

Composition

Composition uses tight coupling, which means that it holds the object. An instance variable is created and the object is stored in it. Whenever the object of the parent class removed/deleted then the child class object will also be deleted.


class Human {

 // This heart will be deleted and nowhere to access once its 
 // human gets deleted
 private Heart heart = new Heart();

}

public class main(String[] args){

 // this human's heart is set internally and will be deleted 
 // upon deleting this Human object
 Human human = new Human();

}

What are Views in Database

Muhammad Wasi Naseer — Sun, 01 Aug 2021 22:44:33 +0000

Database View is a virtual/logical table that is made up of a SQL query on the physical tables. A simple view itself does not exist physically at all but a query under a name.

Reading data from View

When we read data from view, it's basically a query which gets executed under a name

Updating data from View

Insert query

We can insert data into simple views if Primary key and Not null fields are included in the View. If they are not included, constraint error will be thrown

Update query

We can update data in simple views

Delete query

We can also delete data in simple View. Behind the scenes, it executes delete query on the physical table

Types of Views

Simple Views
They are made up of single table.
Complex Views
They are made up of more than one table.

Why we use it?

To simplify complex queries
To hide the complexity of underlying tables to the end user or external apps
To enable backward compatibility. If we make changes in the schema, we can provide views to the existing apps so that they won't break.
To provide extra security layer using read-only views
To provide computed columns as database should not save computed columns

What are Materialized Views?

In order to increase performance in reading data, the data is actually stored on disk. The next time user queries the data from the view, the result wouldn't be calculated on the fly from multiple tables but it's a query on the view.

It is used when the view is made up of multiple tables using JOIN. As we know that joining multiple table decreases performance.

Automating to copy a set of commits from one branch to another

Muhammad Wasi Naseer — Mon, 26 Oct 2020 17:42:10 +0000

This article will describe how to copy a set of contiguous commits from one branch to another branch in a repository

Photo by Yancy Min on Unsplash

Final Script

Code Link

The final version of our bash script copy_commitswould be like this

copy_commits.sh -p <project_path> -s <source-branch> -t <target-branch> -f <commit-start> -e <commit-end>

Let’s assume that we have two branches dev1 and dev2, and we want to move a set of contiguous commits from dev1 to dev2

dev1 branch has 5 commits and the dev2 branch has 1 commit. And, we want to copy the 4 extra commits of dev1 to the dev2 branch by specifying the start commit and end commit.

Steps

We will do the following steps to automate the task.

First, we’ll parse the parameters passed at the time of the script execution. The values will be store in the following five variables:

$PROJECT_PATH
$SOURCE_BRANCH
$TARGET_BRANCH
$FROM_COMMIT
$TO_COMMIT

We will first check out to dev1 branch (source branch) and get all the commits in between the start and end commit (inclusive) in an array.
We’ll then check out to dev2 branch, and we’ll take each of the commits from the array and cherry-pick the commit on dev2 (target branch)

Step 1: Parsing parameters and storing parameters into variables

while getopts ":p:s:t:f:e:" o; do
    case "${o}" in
    p)
            PROJECT_PATH="${OPTARG}"
            ;;
        s)
            SOURCE_BRANCH="${OPTARG}"
            ;;
        t)
            TARGET_BRANCH="${OPTARG}"
            ;;
        f)
            FROM_COMMIT="${OPTARG}"
            ;;
        e)
            TO_COMMIT="${OPTARG}"
            ;;
        *)
            abnormal_exit
            ;;
    esac
done

Following is the code for the abnormal_exit() function

# Prints usage
usage() {                                      
  echo "Usage: $0 -p <project_path> -s <source-branch> -t <target-branch> -f <commit-start> -e <commit-target>  " 1>&2 
}

# Prints usage and exit
abnormal_exit() {                              
  usage
  exit 1
}

Step 2: Getting commits’ hash from the source branch and storing it into an array

commits=($(git rev-list "$FROM_COMMIT^..$TO_COMMIT"))

git rev-list is used to list the git commits in reverse chronological order. Therefore, the latest commit will be at the start of the array.
<start_commit_hash>..<end_commit_hash> is a notation to specify that start from the start_commit_hash (not included) to the end_commit_hash (included). If we want to get start_commit_hash in the result then we have to specify the parent of the start commit. The parent of the start commit can be specified using caret ^ at the end of the commit hash. That is why a caret has been used at the end of the start commit.
$FROM_COMMIT^ : It means that start from the parent of the $FROM_COMMIT. The caret (^) is used to specify the parent of the commit otherwise the $FROM_COMMIT will not be included in the final result.
$(some_command_here) : This notation is used when we want to run some command but want to store the output of the command into a variable. Hence, we can assign its output to a variable on the left.
($(some_command_here)) : If we encapsulate some text into parentheses, then bash will convert them into an array based on the IFS(Internal Field Separator). It is by default whitespace. Hence the output of the $(some_command_here) will be converted into an array.

Step 3: Applying commits on the target branch by picking them from the commits array

We have got all the commits into the commits array. Now, we have to check out the target branch.

git checkout $TARGET_BRANCH

if ! [[ $? -eq 0 ]]
then
    echo $'\u274c' "ERROR: Failed to checkout to $TARGET_BRANCH"
    exit 2
fi

We have to pick each commit one by one and run the cherry-pick command. But, remember that we have to pick the commits in reverse order because the latest commit is at the start of the array. That is why we will start picking commits from length-1 index of the commits array.

# For each commit
for ((i=$((${#commits[@]}-1));i>=0;i--))
do
    echo "Cherry Picking ${commits[i]} ..."
    git cherry-pick "${commits[i]}"
    # If the above command fails, then exit with proper message
    if ! [[ $? -eq 0 ]]
    then
        echo "ERROR: Cherry pick commit with ${commits[i]} failed, exiting..."
        exit 2
    fi
    # Show success message for the copied commit
    echo "DONE: Commit ${commits[i]} copied"
done

# is used to find the length of the array.
The loop starts at length-1, and we will decrementing i until it is less than 0.
$? is used to get the exit status code of the most recent command executed.

Final Script Screenshot

Git Project Commit Log

Screenshot of the script when executed