DEV Community: Azmain Adel

Replication in Distributed Database Systems

Azmain Adel — Thu, 26 May 2022 15:40:26 +0000

Brewer's CAP Theorem

In theoretical computer science, the CAP theorem, also named Brewer's theorem after computer scientist Eric Brewer, states that any distributed data store can only provide two of the following three guarantees:

Consistency: Every read receives the most recent write or an error.
Availability: Every request receives a (non-error) response, without the guarantee that it contains the most recent write.
Partition tolerance: The system continues to operate despite an arbitrary number of messages being dropped (or delayed) by the network between nodes.

In a distributed database system, we have network partitions (P in CAP) by default.
So we get to pick only one of

Availability
Consistency

In this post, I am focusing on the AP combination and I will briefly discuss availability is ensured using database replication.

Availability using Replication

Availability in a distributed system ensures that the system remains operational 100% of the time. Every request gets a (non-error) response regardless of the individual state of a node.

There are two types of database replications:

Active replication - Push
Passive replication - Pull

Active Replication

In active replication each client request is processed by all the servers. So all of the distributed data storage gets updated actively.

Passive Replication

In passive replication there is only one server (called primary) that processes client requests. After processing a request, the primary data storage updates the others passively.

How do the data stores do this?

Passive Replication Methods

Master-Slave replication
Tree replication
Master-Master replication
Buddy replication

1. Master-Slave Replication

A generic replication method.

Master (primary storage) gets the latest data.
Slaves (secondary storages) replicate the data internally.

2. Tree Replication

A structure similar to a tree is maintained. In this case, all the parent nodes are masters and the child nodes are slaves.

Master (tree root) gets the latest data.
Child masters and their slaves replicate the data internally.

3. Master-Master Replication

Every storage is maintained as primary or master.

Every master storage can get latest data.
They replicate each other to stay updated.

4. Buddy Replication

A ring topology of data storage servers.

Each node stores the backup of the previous node on the chain.
The backup is performed using replication.

But what if a node goes offline?

The solution is easy.

Store extra backup.
Update the replication chain.

Cover image taken from unDraw

The 6 R's: Strategies for Cloud Application Migration

Azmain Adel — Thu, 26 May 2022 15:37:32 +0000

“How emigration is actually lived — well, this depends on many factors: education, economic station, language, where one lands, and what support network is in place at the site of arrival.” -Daniel Alarcón

1. Re-host

Lift and shift

This is moving your application from one host to another without making any changes. This migration is usually done by large-scale, business-oriented organizations who want to migrate their application quickly to meet business objectives. These migrations are mostly done by automated migration tools.

Similar to Ctrl+X and then Ctrl+V

2. Re-platform

Lift, modify and shift

This is moving your application from one host to another and making some changes. Let's say you are moving your large-scale application to another cloud platform and want to change the DB while you are at it. This will be considered re-platforming.

3. Refactor/Re-architect

Change the existing environment

Redesign the architecture of your application driven by a strong business need to add new features or upgrade performance. This is performed when the requirements are difficult to achieve in the current architectural states.

An example can be moving from monolithic architecture to microservices.

4. Re-purchase

Move to a new product

This migration commonly refers to moving to the SaaS (software-as-a-service) model. This is changing perpetual licenses to services that will make the application more robust.

For example, using salesforce.com instead of manual CRM.

5. Retire

Get rid of

This means removing or turning off applications that are no longer functional. These savings can boost your business case, direct your team’s attention to the applications people use.

6. Retain/Revisit

Keep that matters the most

This means keeping the applications that are vital to the system or not migration priority to and migrating the others through re-platforming or refactoring. One should only migrate what makes sense for the business.

Image taken from AWS Migration Whitepaper

AWS Fundamentals Course

Azmain Adel — Thu, 26 May 2022 15:35:07 +0000

I recently completed the four-course AWS Fundamentals specialization on Coursera conducted by the AWS team. Beginners can start this one but some basic backend and system design knowledge are necessary.

Link to the course:
AWS Fundamentals Specialization

Brief Introductions to the four courses:

Going Cloud Native:

This course will introduce you to AWS core services and infrastructure.

Topics: EC2, S3, VPC, Gateways, ELB, EFS, EBS, Lightsail, RDS, DynamoDB, Cloudwatch.

Addressing Security Risks:

This course will give you an overview of security best practices when developing and managing applications on AWS.

Topics: AWS Shared Responsibility Model, Security Hub GuardDuty, Secrets Manager, IAM.

Migrating to the Cloud:

This course focuses on analyzing your current environment, planning your migration, AWS services that are commonly used during a migration, and the actual migration steps.
There are optional lab works but access to that is limited to paid enrolled students. Thanks to BUET I got a paid Coursera access.

Topics: DMS, Migration policies, Direct Connect, etc.

Building Serverless Applications:

This course will introduce you to the AWS serverless architecture. Through demonstrations and hands-on exercises you'll learn skills by building and deploying a serverless website & a chat bot.

Topics: Lambda, API Gateway, Cloudfront, DynamoDB, Lex, Poly.

GitHub Repos You Should Star

Azmain Adel — Thu, 26 May 2022 15:33:18 +0000

GitHub is the best place for frameworks, libraries, projects, documentation, and collections of many things.

Here I've listed some GitHub repos that might interest you. I've found these from reading articles, study purposes, suggestions from other people, random repo surfing adventure, and from the curiosity of learning new tools & frameworks.

You can learn and build new things, study for interviews and make yourself a better developer from these. This is not only for software developers, CS students can also gain something from this list. Hope this helps.

Awesome

What is awesome? This is the unofficial wiki of the most awesome GitHub repositories. The categorized and detailed lists include tutorials, libraries, documentation, etc.
Well, this repo is truly awesome indeed.

Free Programming Books

Alongside a huge collection of programming-related books, this repo has free online courses, interactive programming tutorials, programming podcasts, etc.

Moreover, there is support for many languages. A must-have repo.

Coding Interview University

From the repo creator's own words who got hired by Amazon after following this study plan,

"This is my multi-month study plan for going from a web developer (self-taught, no CS degree) to software engineer for a large company. This is meant for new software engineers or those switching from software/web development to software engineering (where computer science knowledge is required)."

That sums it up really well. Follow this repo and study hard.

Tech Interview Handbook

GitHub: Tech Interview GitHub

This is a wonderfully curated repo that will help you with not just the technical studies, but also in applying for jobs, resume writing, interview processes, behavior tips, etc. This has practical content that covers all phases of a technical interview, from applying for a job to passing the interviews to offer negotiation. A must-have.

System Design Primer

The absolute best resource for learning how to design scalable systems. System design is a broad topic. There is a vast amount of resources scattered throughout the web on system design principles.

This repo is an organized collection of resources to help you learn how to build systems at scale.

Build Your Own X

The first thing on the repo,

"What I cannot create, I do not understand" - Richard Feynman

Prefer learning a new language, framework, game engine by practical usage first? Then this repo if for you. This is a great collection of video tutorials, articles grouped by the 'X' technology you love.

Free For Dev

This is a list of software (SaaS, PaaS, IaaS, etc.) and other offerings that have free tiers for developers. Need some free services or APIs or dev tools for your pet project? Search for it here and most of the time you will be lucky.

Developer Roadmap

The roadmap to becoming a (web) developer in 2020. You will find a set of charts demonstrating the paths that you can take and the technologies that you would want to adopt in order to become a frontend, backend, or DevOps engineer.

Do check out another great repo by the same author: Design Pattern for Humans

The Art of Command Line

Whatever your environment is, whether you are developing or doing some research-work, the command line is something you face every day. Fluency on the command line is a skill often neglected. This is a selection of notes and tips on using the command line in subtle ways.

Public APIs

A great list of free APIs for use in software and web development. Build demo apps, test your systems, play with data by calling these free APIs. They're here for you.

Awesome Interview Questions

Feeling confident after studying so much? This repo will help you in understanding what types of questions are asked during tech interviews. The grouped list and the huge collection will help you practice and enrich your knowledge.

Papers We Love

This repository serves as a directory of some of the best papers scattered across the web on academic computer science. The contributors from around the world have created a beautifully curated list on many fundamental and modern topics.

The Algorithms

This is actually not a single repo. This is a collection of repos that includes almost every data structure and algorithm implemented in multiple languages like Java, C++, Python, etc.

Pure Bash Bible

This repo includes commonly-known and lesser-known methods of doing various tasks using only built-in bash features. Using the snippets from this repo can help remove unnecessary dependencies from scripts and in most cases make them faster.

Thanks for reading.

Please do leave a comment or knock me if you have any queries regarding this post.

Getting the SFC™ Certification

Azmain Adel — Thu, 26 May 2022 15:29:53 +0000

Scrum

Most of you have already heard of the term Scrum.

Scrum is one of the most popular Agile methodologies. It is an adaptive, iterative, fast, flexible, and effective methodology designed to deliver significant value quickly and throughout a project. Scrum ensures transparency in communication and creates an environment of collective accountability and continuous progress.

To get the basics, you can watch this short video.

SFC?

SFC™ stands for Scrum Fundamentals Certified. This certification is provided by SCRUMstudy which is owned by VMEdu. They have a broad coverage of Scrum certification:

Scrum Fundamentals Certified (SFC)
Scrum Developer Certified (SDC)
Scrum Master Certified (SMC)
SCRUMstudy Agile Master Certified (SAMC)
Scrum Product Owner Certified (SPOC)
Expert Scrum Master Certified (ESMC)
SCRUMstudy Certified Trainer (SCT)
SCRUMstudy Certified Agile Coach (SCAC)
Certified Implementation Expert (CIE)
Scaled Scrum Master Certified (SSMC)

SFC is tailored to help anyone interested to know more about Scrum; learn about key concepts in Scrum, and to get a basic understanding of how the Scrum framework works in delivering successful projects.

Although this is a free certificate, this provides the necessary knowledge on Scrum fundamentals.

What I learned

I always believed that to become a better software engineer, even if you are going to code most of the time during the initial years, knowledge of team & project management is a MUST. This can be gained through experience and what I have understood, that is absolutely the best way to go. But extra knowledge always helps and I wanted to explore an organized methodology of managing projects.

If you are new to the software industry or currently in your final year of graduation, this course will definitely help you.

This certification helped me to learn many topics of such a methodology, Scrum.

If you are to take the exam, you will know these topics anyway. Also, the SBOK guide has described these perfectly. So I am just pointing out the topics I got to learn from studying for the exam.

The origin story of Scrum
Benefits of Scrum
The six Scrum principles
Scrum Flow
Scrum Aspects
Scrum Phases and Processes
Scrum Scaling
The 19 fundamental Scrum processes, along with inputs and outputs
Scrum Roles

Getting the certification

Registration

To get access to the course contents and exam portal, you need to register on the SCRUMstudy registration page.

You will get access to a student portal where you will find the course contents, downloadable study materials, exam portal and other necessary stuff.

How and what to study?

Now this is a tricky part.

First, let's see the official guideline to prepare for the exam:

Attend the instructor-led training webinar
Watch the course videos and take test exams on SCRUMstudy student portal
Study the SBOK guide

For only the SFC this is way too much. But, to know Scrum and become a master, do follow all of the above methods.

I personally found it hard to keep concentration during the 4-hour webinar. Also, the timezone of Bangladesh was a problem as I had to sit through boring lectures from 9 PM to 1:30 AM including breaks. As the training presentation slides will be provided beforehand, reading that should cover the webinar in my opinion.

So, the primary study material will be the course videos and the SBOK guide. Watch the videos and read the relevant topics from the guide book. As the SFC covers the key concepts and principles of Scrum, you do not need to finish the whole book for this certification. I finished the videos on a weekend and took another couple of days to study the SBOK guide to get the confidence for attending the exam.

The exam

The format of the exam is as follows:

Multiple choice
40 questions
One mark awarded for every right answer
No negative marks for wrong answers
30 questions need to be answered correctly to pass
60 minutes duration
Online unproctored exam

If I had to rate how tough the exam was, I would give it 4/10. In other words, easy!

But, please do not attempt the exam without studying. Actually this study beforehand will benefit you the most as knowing and understanding Scrum will help in the long run.

Certificate

After finishing the exam successfully you will get your SFC certificate. Keep it clean and tidy, forget about it, show it off or do whatever you seem is sensible.

Remember, the knowledge you acquire is your biggest achievement.

By the way, here is the one I got!

Thanks for reading.

Please do leave a comment or knock me if you have any queries regarding this post.