DEV Community: jainnehaa

Microservices : From Principles to Patterns

jainnehaa — Thu, 18 Aug 2022 10:58:00 +0000

Microservice Architecture is a variant of the Service Oriented Architecture structural style, which decompose an application into a collection of loosely-coupled services, interacting with each other using communication mechanisms like API.
Microservices aims to provide Scalability, Decentralization, Availability, Isolation from Failures, Real-time Load Balancing etc.

IDEALS Principles incorporates key Principles of Microservice-based Architecture. It includes some of the SOLID principles. IDEALS stand for Interface Segregation : Deployability : Event-Driven : Availability Over Consistency : Loose-Coupling : Single Responsibility
Adhering to these principles poses implementation challenges which are solved with the help of different design patterns.
Design Patterns are reusable solutions to commonly occurring problems within a given context in software design. Context here is microservices architecture. These patterns complement each other catering to different layers and problems in an application's architecture, as illustrated in image below :

Image Details can be referred at : docs.microsoft.com/microservices-design-patterns

Once we decide to Refactor our application from Monolithic to Microservices, first question in our minds is : "So, what's the plan, how we gonna do it ?", as we have an existing function business application in Prod along with multiple new requirements lined up.
One way to do so is via Strangler Fig Pattern. Strangler Fig, also called Strangler is named after any of numerous species of tropical figs (genus Ficus, family Moraceae) named for their pattern of growth upon host trees, which often results in the host's death.
Strangler Fig supports incremental refactoring of a legacy application into a new (strangler) microservice application, by gradually replacing specific pieces of functionality with new services via steady and frequent releases.
It consists of two types of services. One, those which implement functionality that previously resided in the monolith, which shrink over time. Second, services that implement new features.

Another pattern for refactoring is Anti-Corruption Layer which implements a façade between new and legacy applications, to ensure that the design of a new application is not limited by dependencies on legacy systems.

I

Interface Segregation Principle suggests each microservice interface to attend to a specific need of a client's program. In other words, clients should not be forced to depend on an interface they don't use.
One way to implement the principle is through an API gateway.
API Gateway address the problem of how an individual service is accessed in Microservices-based application by helping understand various communication protocols, translate them as needed, modify the request and response to suit the service and the client programs, etc. API gateway acts as the single entry point for all clients and handles requests by proxying or routing to the appropriate service or by fanning out to multiple services.

D

Deployability Principle acknowledges critical design decisions and technology choices developers need to make regarding packaging, deploying and running microservices.
Deployability involves:

Configuring the runtime infrastructure including containers, pods, clusters, persistence, security and networking
Scaling microservices in and out or runtime environment migration
Expediting the commit+build+test+deploy process
Minimizing downtime
Synchronizing version changes
Monitoring microservices health to identify and remedy faults

Each service is deployed as a set of service instances for throughput and availability.
The Single Service per Host and Multiple Services per Host patterns are two different deployment strategies, addressing ways to package and deploy them.

Single Service per Host deploys each single service instance on its own host.
This pattern provides service instance isolation avoiding resource conflicts and making monitoring and management straightforward, but stands comparatively less efficient to Multiple Services per Host in resource utilization.
Multiple Services per Host run multiple instances of different services on a shared host. This could be done by deploying each service instance as a JVM process or by deploying multiple service instances in the same JVM.
There are further refinements to this Single Service per Host pattern like Service Instance per VM Pattern and Service Instance per Container Pattern.
Serverless deployment Pattern is an alternative solution to those mentioned above.

E

Event Driven Principle
One of the critical design decisions to make microservices fault-tolerant is to avoid cascading failures caused by downtimes of dependent services.
Event-driven Principle helps in this aspect as the producer and consumers of the event are not tightly coupled. The messaging channel used for communication keeps both ends of it independent and hence keeps them loosely coupled. This principle improves reliability, scalability and throughput of the entire system.

Event Sourcing is a design pattern implementing this principle, which helps reliably, atomically update the database and send messages/events. For example, a service that participates in a saga (explained below) needs to atomically update the database and sends messages/events. Similarly, a service that publishes a domain event must atomically update an aggregate and publish an event, which can be done implementing Event Sourcing.

A

Availability over Consistency
The CAP theorem essentially gives two options: Availability OR Consistency, along with partition tolerance.
For microservices, the main strategy that enables the availability choice is data replication, which can be achieved via Service Data Replication pattern, Command Query Responsibility Segregation (CQRS) pattern etc. CQRS separates the design and implementation of operations that change data (commands) from the ones that only read data (queries).
Message replication through Event Sourcing Pattern (event-driven architecture) generally works on Eventual Consistency, keeping Availability over Consistency.

L

Loose Coupling is a coupling technique used to describe the degree and intent of interconnected but non-dependent components within an information system. Loosely coupled with other services - enables a team to work independently the majority of time on their service(s) without being impacted by changes to other services and without affecting other services.

Several strategies can be used and combined to promote afferent (incoming) and efferent (outgoing) loose coupling. Afferent coupling is a measure of how many other services use a specific service as Incoming and Efferent coupling is a measure of how many different services used by the specific service as Outgoing.
Single responsibility is the idea that enables modeling microservices that are not too large or too slim because they contain the right amount of cohesive functionality.

Few Design Patterns promoting loose coupling are Database per Microservice, Saga, API Gateway, BFF, Saga etc.

Backends for Frontends (BFF) creates separate backend services for different types of clients, such as desktop and mobile. That way, a single backend service doesn't need to handle the conflicting requirements of various client types. This pattern can help keep each microservice simple, by separating client-specific concerns.

Database per Microservice suggests each microservice’s persistent data private to that service and accessible only via its API. A service’s transactions only involve its database.
Here, tight coupling is being handled at database layer.
Separate data store doesn't imply only a database for each service, rather Private-tables-per-service, Schema-per-service or Database-server-per-service, as per requirement.
When we implement Database per Microservice, some business transactions span multiple services, hence need to implement transactions that span services arises, which is addressed yet another patter known as Saga Pattern.
Saga is a long lived transaction that can be broken up into transactions, but still executed as a unit, ensuring eventual consistency.
Few other database related patters are API Composition and Command Query Responsibility Segregation (CQRS) pattern which provide ways to implement queries.

S

Single Responsibility Principle (SRP) is about having cohesive functionality in an OO class. The notion of single responsibility can be extended to the cohesiveness of services within a microservice. The microservice architecture style dictates that the deployment unit should contain only one service or just a few cohesive services. If a microservice is packed with responsibilities, that is, too many not quite cohesive services, then it might bear the pains of a monolith.

Here, we have seen only a bunch of design patterns with only few microservices principles. It is wider topic which cater to variety of domains, use-cases and challenges.

References :

microservices.io
martinfowler.com
docs.microsoft.com
wikipedia.org
cs.cornell.edu
lokajittikayatray.com
infoq.com
wikipedia.org
britannica.com
techopedia.com
stackoverflow.com

Image References :
docs.microsoft.com

Network Authentication Protocols

jainnehaa — Tue, 16 Aug 2022 18:23:00 +0000

Authentication keeps invalid users out of databases, networks, and other resources.
Authentication Protocols are well defined, industry standard ways to validate the identity of remote clients/users (Authentication) before granting them access to server data (Authorization) and in monitoring network resources and information needed for billing of services (Accounting).

Different Authentication Protocols used to validate users are PAP, CHAP, Oauth 2, TACACS+, RADIUS, Kerberos, SAML, LDAP/Active Directory etc. These can be used simultaneously for different purpose.

Password authentication protocol (PAP):
It is one of the oldest authentication protocols deprecated now, yet used sometimes.
In PAP, system compares user credentials (username and password) to a database to validate. Since there is no encryption, it is least secure.

Challenge-Handshake Authentication Protocol (CHAP):
CHAP periodically reauthenticates users identity using a 3-way handshake. Each challenge is different from last version and verification is done even within the same session.

Extensible authentication protocol (EAP):
Wireless networks and point-to-point connections uses EAP.

Kerberos:
Kerberos uses symmetric keys between client-server (secret-key cryptography), pulled from a centralized key distribution center known as Key Distribution Center (KDC).
There are few drawbacks viz, the process is time-sensitive, so devices using the Kerberos must have relatively well-synced clocks. Also, it not advised to use this protocol for networks heavy on virtual hosting because every host requires its own set of Kerberos keys.
Further, in 2020, Kerberos stopped working after a system update.

Lightweight Directory Access Protocol (LDAP):
The Lightweight Directory Access Protocol is an open, vendor-neutral, industry standard application protocol for accessing and maintaining distributed directory information services over an Internet Protocol (IP) network.

Organizations using LDAP stores information like user credentials (usernames, passwords), email addresses, printer connections and other static data within directories.
LDAP works by binding an LDAP user to an LDAP server.
The client sends an operation request that asks for a particular set of information, such as user login credentials or other organizational data.
The LDAP server then processes the query based on its internal language, communicates with directory services if needed, and responds.
When the client receives the response, it unbinds from the server and processes the data accordingly.

LDAP is the underlying protocol for directory services and is leveraged in Directory-as-a-Service (DaaS), a cloud-based software-as-a-service (SaaS) user directory.

OAuth 2.0:
OAuth 2 is a way for users to allow websites and mobile apps to access their credentials without giving them the passwords, implemented by various sites viz Google, Facebook, Twitter etc. OAuth 2 works with HTTP. It provides access tokens to be applied to a third-party with the permission of the resource owner.
One of the possible security risk with OAuth 2 is that it uses one account for multiple applications, so, if the main account is ever compromised, users risk compromising many more instances. Such an issue was faced by GitHub in 2020.

Remote authentication dial-in user service (RADIUS): RADIUS system verifies credentials (username-password) provided by client by comparing it to data in a database. RADIUS provides centralized authentication, authorization, and accounting (AAA or Triple A) management.
In RADIUS,
Authentication data flows from NAS (Network Access Server) to RADIUS server in Access-Request packets.
Authorization data flows from the RADIUS server to NAS in Access-Accept, Access-Reject, CoA-Request, and Disconnect-Request packets.
Accounting data flows from the NAS to the RADIUS server in Accounting-Requests.

Diameter is derived from RADIUS and provides a framework for authentication and accounting messages.
It is a Peer-to-Peer (P2P) protocol. Peers can be statically configured or dynamically discovered.
Extensibility is an important trait of the Diameter protocol. It supports a modular architecture with the base protocol and application-specific extensions.

There are many other efficient and widely used authentication protocols like SAML (XML based protocol), TACACS (IP-based authentication), it's enhanced versions and various others.

Reference :

datatracker.ietf.org
frontegg.com
auvik.com
sailpoint.com
okta.com
getkisi.com/blog
jumpcloud.com
stackoverflow.com
Wikipedia - Authentication Protocol
Wikipedia - PPP
Wikipedia - LDAP

Web Accessibility Primer

jainnehaa — Sun, 14 Aug 2022 18:50:00 +0000

Web Accessibility is designing and developing websites, tools and technologies such that people with disabilities can also perceive, understand, navigate, interact and contribute on them.

Web accessibility encompasses auditory, cognitive, neurological, physical, speech and visual disabilities that could affect access to the Web.

Web accessibility also benefits people without physical disabilities but with situational disabilities, socio-economic restrictions on bandwidth or speed etc, viz:

people using devices with small screens, different input modes, etc
people with temporary disabilities such as a broken arm or lost glasses
people with situational limitations such as in bright sunlight or noise around hindering listening to audio
people using a slow internet connection or having limited or expensive bandwidth

Accessibility is the practice of making websites usable by as many people as possible and is the right thing to do. Providing accessible sites is part of the law in some countries, which can open up some significant markets that otherwise would not be able to use your services or buy your products.

Accessibility, Usability, and Inclusion are closely related aspects.
Accessibility addresses discriminatory aspects in user experience for disabilities.
Usability is about designing products to be effective, efficient, and satisfying. Usability practice and research often does not consider the needs of people with disabilities.
Inclusion/Universal Design/Design For All is about diversity and ensuring involvement of everyone to the greatest extent possible, addressing broad range of issues including:

accessibility for people with disabilities
access to and quality of hardware, software, and Internet connectivity
computer literacy and skills
economic situation
education
geographic location
culture
age, including older and younger people
language Accessibility focuses on disability and does not try to address broader issues as Inclusion.

Implementing Accessibility is a topic of wide range. So, listing down only few strategies and tips to give a big picture on how to to implement it (based, on documentations and approaches adopted by various organizations)

Ensure readability when using color by highlighting content changing fonts to make to noticeable and readable by all including those with color blindness.
Avoid usage of color alone to convey information, instead provide additional identification that does not rely on color perception. For example, asterisk along with color to indicate required form fields, labels along with color in graphs

Descriptive hyperlink text usage to help recipients understand who use a screen reader or for anyone who would like to know exactly where the link will take them.
Provide sufficient contrast between foreground and background
Create designs for different viewport sizes

Add descriptive text to images as with 'Alt Text' to help when an image won’t load or if a recipient can’t see the visual.
Ensure Forms are Keyboard Accessible : Users who cannot use a mouse, navigate web pages with the keyboard. Forms that use JavaScript to manipulate form data, set focus, change form elements or submit forms, can create interactions that only work with a mouse. Ensure that your web site forms can be understood and operated with the keyboard alone.
Screen reader users navigate using the Tab key. Although labels are announced when form inputs received keyboard focus, other text between the form controls is usually skipped. Be sure to include any instructions at the beginning of the form, or associate them with specific fields using aria-describedby.
Text to Speech : Text-to-speech (TTS) technologies read written text aloud, providing essential accommodations for users with certain impairments and is capable of synthesizing natural, human-like audio from virtually any piece of text, with applications ranging from in-home assistants (like Alexa and Google Home) to language-learning tools and screen readers.
Video Captions are essential for people with disabilities and useful for all.
Work using everyday language
Speech to Text : Speech recognition can be used for dictating text in a form field as well as navigating to and activating links, buttons, and other controls.
Speech recognition is about recognizing words for speech-to-text (STT) transcription, virtual assistants and other speech user interfaces. This technology identifies who the speaker is, not the words they’re saying.

Reference :
w3.org/WAI
developer.mozilla.org
wikipedia.org
microsoft.com
query.prod.cms.rt.microsoft.com
webaim.org
boia.org
microsoft.com

Image Resources :
Contrast Sufficiency
Viewport sizes

Significance of Computer Specs : Processor & Memory

jainnehaa — Sun, 14 Aug 2022 13:30:00 +0000

Considering configurations mentioned in technical specifications for a typical laptop, this write-up would intend to explain meaning and significance for 2 parameters : Processor and RAM.

# Processor :
A processor or CPU is the brain of the computer, giving it the processing power.

Example of a processor from AMD :
AMD Ryzen™ 7 5825U (up to 4.5 GHz max boost clock, 16 MB L3 cache, 8 cores, 16 threads)
Here,

AMD Ryzen™ 7 5825U is of series - Cezanne (Zen 3, Ryzen 5000) Barcelo (Zen 3 architecture). AMD is a brand, Ryzen 7 one of the series amongst 3,6,7,9 etc.
4.5 GHz max boost clock :
Clock Speed is the speed of a computer CPU, measured in hertz. One billion hertz per second = one gigahertz (GHz). It is a measure of frequency and is used to describe bus speeds and CPU clock speeds.
16 MB L3 cache :
Cache memory is a chip-based computer component that makes retrieving data from the computer's memory more efficient. It acts as a temporary storage area that the computer's processor can retrieve data from easily. Cache memory is sometimes called CPU memory or Hardware Cache.
Cache is graded into different levels in increasing size order as :
Level 1 (L1) < Level 2 (L2) < Level 3 (L3) < Level 4 (L4 - not very common)
The more L2 and L3 memory available, the faster a computer can run. However, not a lot of physical space is allocated for cache. There is more space for RAM, which is usually larger and less expensive.
Storage Memory Hierarchy :

8 cores :
Core is a processing unit within a CPU. CPUs can have multiple cores. More cores a processor has, the faster it can process information.
Each CPU core has its own L1 cache, but may share L2 and L3 caches.
16 threads :
Threads are the virtual components or codes, which divides the physical core of a CPU into virtual multiple cores. A single CPU core can have up-to 2 threads per core.
Hence, if a CPU is dual core (i.e., 2 cores) it will have 4 threads.
And if a CPU is Octal core (i.e., 8 core) it will have 16 threads and vice-versa.

Another example of a processor from Intel has configuration against similar parameters :
Intel® Core™ i5-1240P (up to 4.4 GHz with Intel® Turbo Boost Technology, 12 MB L3 cache, 12 cores, 16 threads)

Image shows the naming convention adopted by Intel for it's processors

# RAM :
Random Access Memory (RAM) is the computer's short-term, working memory. The computer places data in the RAM when a program is currently using it to work on a task. Once you finish, the data is deleted from the RAM so the memory is available for other tasks.
The more RAM you have, the more programs you can have running at once and the faster the computer will respond.
When shopping for RAM, check to see what the maximum amount is that your motherboard and processor can support—RAM can only work to those limits.

Example :
16 GB DDR4-3200 MHz RAM (2 x 8 GB)

3200 MHz :
RAM is measured by how many cycles per second it can perform. For example, if RAM is rated at 3200 MHz, it performs 3.2 billion cycles per second.
The more cycles your RAM can perform per second translates to how much data can be stored and read – making for smoother user experiences.
There’s usually a direct correlation between higher DDR (Double Data Rate) RAM and the clock cycles those modules accommodate.
16 GB DDR4 :
RAM has evolved from DRAM to SDRAM and now to DDR.
Dynamic Random-Access Memory (DRAM), introduced in the 1970s, is an asynchronous memory and not regulated by a clock. This posed a problem in organizing data as it comes in so it can be queued for the process it’s associated with. Also, DDR being asynchronous could work as fast with processors that were just getting faster.
Synchronous Dynamic Random Access Memory (SDRAM) is synchronous and relies on a clock to synchronize signals, creating predictable orderly cycles of data fetches and writes. However, SDRAM transfers data on one edge of the clock.
DDR SDRAM means that this type of SDRAM fetches data on both the leading edge and the falling edge of the clock signal that regulates it, thus the name DDR (Double Data Rate). Prior to DDR, RAM would fetch data only once per clock cycle.
DDR is not flash memory like the kind that is used for Solid State Drives (SSDs), Secure Digital (SD) cards, or Universal Serial Bus (USB) drives. DDR memory is volatile, which means that it loses everything once power is removed.
DDR has faster transfer rates and high capacity.
DDR has progressed from DDR to DDR2, DDR3, and now DDR4 SDRAM (as in Table below).
DDR2 – DDR4 evolved to require lower supply voltages saving power and some changes to increase the speed.
DDR2 SDRAM reduced operating voltage to 1.8 volts and a clock multiplier was added to the memory module to again double data transfer speeds while operating at the same bus speed.
DDR3 RAM integrated a 4x clock multiplier, again doubling the memory transfer rate for the same bus speed.

Comparison of SDRAM in order of release. Clock and transfer rates are approximate and vary by manufacturer :

2 x 8 GB : 2 slots of 8GB RAM each

Reference :
amd-ryzen-7-5825u
amd.com
androidauthority.com
bbc.co.uk
intel.com/processor-numbers
intel.com/resources
intel.com/products
12th-generation-intel-core-i9-processors
techtarget.com
geeksforgeeks.org
wepc.com
microcontrollertips.com

Image/Table Resource :
Storage Memory Heirarchy
Intel Naming Convention
DDR and SDRAM Comparision

FaaS : Function as a Service

jainnehaa — Sat, 13 Aug 2022 09:00:00 +0000

Function as a service (FaaS) is a cloud computing services providing a platform to customers to develop, run, and manage application functionalities, without the complexity of building and maintaining the infrastructure associated with developing and launching an app.

Building an application following this model is one way of achieving a serverless architecture and is typically used when building microservices applications. Use cases for FaaS are associated with "on-demand" functionality that enables the supporting infrastructure to be powered down and not incur charges when not in use.

AWS Lambda was the first FaaS offering by a large public cloud vendor, followed by Google Cloud Functions, Microsoft Azure Functions, IBM/Apache's OpenWhisk (open source) in 2016 and Oracle Cloud Fn (open source) in 2017.

Let's understand how a function work with a serverless framework, taking Azure Functions into consideration :

Serverless Functions execute code—written in the language of one's choice with Azure Functions, with an event-driven compute experience.
Azure Functions scales on demand and one pays only for the time your code is executed.
Available as a managed service in Azure and Azure Stack, the open source Functions runtime also works on multiple destinations, including Kubernetes, Azure IoT Edge, on-premises, and even in other clouds.

A Function should be written to perform a single job such as:
Saving a user to the database
Processing a file in a database
Performing a scheduled task
Separation of concerns should be taken into consideration

Few sample scenarios to implement Azure Functions :

Anything that triggers an Azure Function to execute is regarded as an Event or Trigger. A trigger defines how a function is invoked and a function must have exactly one trigger. Triggers have associated data, which is often provided as the payload of the function.
Few Triggers on Azure Functions are :

An HTTP Trigger (e.g., for a REST API)
A scheduled timer (e.g., run every 5 minutes)
A Service Bus Queue trigger (e.g. a workitem from another Function)
An IoT/Event Hub message (e.g., a message from a device or service)
A Webhook fires (e.g., Github project update) And many more..

Binding to a function is a way of declaratively connecting another resource to the function; bindings may be connected as input bindings, output bindings, or both. Data from bindings is provided to the function as parameters.
You can mix and match different bindings to suit your needs. Bindings are optional and a function might have one or multiple input and/or output bindings.
Like, Blob Storage, Queue, CosmosDB etc

Triggers and bindings let you avoid hardcoding access to other services.
Each function has its own code file and binding configuration file as function.json where triggers are triggers are configured.
(used along different languages like Java, Javascript, Python, C#).

{ "scriptFile": "azure-functions-example.jar", "entryPoint": "com.example.Function.echo", "bindings": [ { "type": "httpTrigger", "name": "req", "direction": "in", "authLevel": "anonymous", "methods": [ "GET","POST" ] }, { "type": "http", "name": "$return", "direction": "out" } ] }
AWS Lambda has similar use case and can cater to file processing, stream processing, web applications, IoT backends and mobile backends.
One such scenario is illustrated as below - File Processing :

Amazon Simple Storage Service (Amazon S3) to trigger AWS Lambda data processing in real time after an upload, or connect to an existing Amazon EFS file system to enable massively parallel shared access for large-scale file processing.

References :
docs.microsoft.com/azure-functions
aws.amazon.com/lambda/
serverless.com
azure.microsoft.com/serverless/

Wikipedia - Serverless Computing
Wikipedia - FaaS
Wikipedia - Microsoft Azure

Image Resources :
AWS Lambda - File Processing

Trade-offs of Distributed Data Stores : Theories and Theorems

jainnehaa — Sat, 13 Aug 2022 05:30:00 +0000

Distributed Data Store refers to either a distributed database storing on a number of nodes (often in a replicated fashion)
OR
a computer network storing information on a number of peer network nodes.

Certain theorems describes potential limitations and trade-offs (say, in consistency) for distributed databases, CAP Theorem is one of them.
The CAP Theorem states that any distributed data store can provide only two of the following three guarantees :
Consistency-Availability-Tolerance towards Network Partition
This theorem is also named as Brewer's theorem after computer scientist Eric Brewer who gave it's hypothesis and the theorem was proved by Gilbert and Lynch.

Considering a distributed system with more than one node,
In a Consistent system, once a client writes a value to any of the nodes and gets a response, it expects to get the latest value next time it read back from any of the nodes.
Being Available means all non-failing nodes continue to serve requests if the system is partitioned.
System should function correctly despite arbitrary Network Partitions.

It is important to note that Consistency in CAP Theorem [Consistent-Available-Partition-tolerant] isn't the same as in ACID [Atomic-Consistent-Isolated-Durable] database transactional properties.
CAP comes from the distributed systems theory, while ACID belongs to database systems one.
“C” in the CAP Theorem refers to each node in the system having the same data, the “C” in ACID refers to a single node enforcing the same rules on every potential commit, such as the type of a field or a field not being null.

Database systems designed with traditional ACID guarantees in mind such as RDBMS choose consistency over availability,
whereas systems designed around the BASE philosophy, common in the NoSQL movement for example, choose availability over consistency.
Eventual consistency is a consistency model used in distributed computing to achieve high availability, if no new updates are made to a given data item, eventually all accesses to that item will return the last updated value. Eventually-consistent services are often classified as providing BASE semantics (Basically-available, Soft-state, Eventual consistency).

Basically available: reading and writing operations are available as much as possible (using all nodes of a database cluster), but might not be consistent (the write might not persist after conflicts are reconciled, and the read might not get the latest write)
Soft-state: without consistency guarantees, after some amount of time, we only have some probability of knowing the state, since it might not yet have converged
Eventually consistent: If we execute some writes and then the system functions long enough, we can know the state of the data; any further reads of that data item will return the same value

Consistency Model specifies a contract between the programmer and a system, wherein the system guarantees that if the programmer follows the rules for operations on memory, memory will be consistent and the results of reading, writing, or updating memory will be predictable. Consistency models are used in distributed systems like distributed shared memory systems or distributed data stores (such as filesystems, databases, optimistic replication systems or web caching).

PACELC Theorem is an extension to the CAP theorem. It states that an additional trade-off exists: between latency and consistency, even in absence of partitions, thus providing a more complete portrayal of the potential consistency trade-offs for distributed systems.

PACELC Theorem states that in case of network partitioning (P) in a distributed computer system, one has to choose between availability (A) and consistency (C), but else (E), even when the system is running normally in the absence of partitions, one has to choose between latency (L) and consistency (C). PACELC reorders CAP into PAC and adds a statement about what happens when the network is present. That is, you may choose between increased latency and consistency (Else Latency or Consistency).

Database PACELC ratings :
The default versions of DynamoDB, Cassandra, Riak and Cosmos DB are PA/EL systems: if a partition occurs, they give up consistency for availability, and under normal operation they give up consistency for lower latency.
Fully ACID systems such as VoltDB/H-Store, Megastore, MySQL Cluster and PostgreSQL are PC/EC: they refuse to give up consistency, and will pay the availability and latency costs to achieve it. BigTable and related systems such as HBase are also PC/EC.
Cosmos DB supports five tunable consistency levels that allow for tradeoffs between C/A during P, and L/C during E. Cosmos DB never violates the specified consistency level, so it's formally CP.
Cosmos DB offers five well-defined levels. From strongest to weakest, the levels are:
Strong, Bounded staleness, Session, Consistent prefix, Eventual
MongoDB can be classified as a PA/EC system. In the baseline case, the system guarantees reads and writes to be consistent.

A high availability requirement implies that the system must replicate data. As soon as a distributed system replicates data, a trade-off between consistency and latency arises.
Thus, the choice of the system characteristics depends on the application and requirements.

References :
Wikipedia - Distributed Data Stores
Wikipedia - Consistency Model
Wikipedia - PACELC
mwhittaker.github.io
edisciplinas.usp.br
medium.com/@skeller88
blog.thislongrun.com
Eventual_consistency
ardalis.com
Azure CosmosDB

Image Resources :
CAP-theorem-with-databases-that-choose-CA-CP-and-AP
Summary Table

Quick Guide to Structural and Creational Design Patterns

jainnehaa — Wed, 10 Aug 2022 14:35:00 +0000

This post is in continuation to Quick Guide to Behavioral Design Patterns

USE-CASES Structural Patterns :

Adapter 'adapts' one interface for a class into one that a client expects, both being incompatible.
Client -> Target Interface -> Adapter Class (for compatibility) -> Adoptee Interfaces

Composite has a tree structure of objects where every object has the same interface. It lets clients treat individual objects and compositions of objects uniformly.
1-to-many "has a" up the "is a" hierarchy

Decorator add responsibilities to objects dynamically, without changing its interface.
(Vocab Hint : 'Decorate' means 'Wrapping a gift, putting it in a box')

Facade defines a new single unified interface to client to interact with a complex subsystem for existing objects

Proxy creates an object representing another object. It uses an extra level of indirection as a wrapper or delegation to support distributed, controlled, or intelligent access.

Bridge decouples an abstraction/interface from its implementation so that the two can vary independently

Private Class Data pattern seeks to reduce exposure of attributes by limiting their visibility.
It restricts accessor or mutator access. It allows the class designer to remove write privilege of attributes that are intended to be set only during construction, even from methods of the target class.

Flyweight a fine-grained instance with common properties used for efficient sharing. Flyweight explains when and how State objects can be shared.

USE-CASES Creational Patterns :

Singleton lets you ensure that a class has only one instance, while providing a global access point to this instance.
Encapsulated "just-in-time initialization" or "initialization on first use".

There should be only one Singleton instance, whereas a Flyweight class can have multiple instances with different intrinsic states.
The Singleton object can be mutable. Flyweight objects are immutable.

To avoid tight coupling between the creator and the concrete products, Factory Method lets define an interface for creating an object but let subclasses decide which class to instantiate.
Factory Method is based on inheritance but doesn’t require an initialization step.
Single Responsibility Principle and Open/Closed Principle is achieved.

In order to get rid of repeated initialization code, Prototype lets you copy existing objects without making your code dependent on their classes.
Prototype requires an initialization of the cloned object, which is without coupling to their concrete classes.
This provides an alternative to inheritance when dealing with configuration presets for complex objects.

Abstract Factory defines a Factory Method per product.
It produces families of related objects without specifying their concrete classes. A class delegates object creation to a factory object instead of instantiating concrete classes directly.
Abstract Factory classes are often based on a set of Factory Methods, but you can also use Prototype to compose the methods on these classes.

Builder separates construction of a complex object from its representation, so that the same construction process can create different representations.
The Builder pattern suggests that you extract the object construction code out of its own class and move it to separate objects called builders.

Abstract Factories, Builders and Prototypes can all be implemented as Singletons.
Many designs start by using Factory Method (less complicated and more customizable via subclasses) and evolve toward Abstract Factory, Prototype, or Builder (more flexible, but more complicated).

References :

sourcemaking
refactoring.guru
w3sdesign
wiki.c2
wikipedia
wikipedia
wikipedia
wikipedia
wikipedia

Quick Guide to Behavioral Design Patterns

jainnehaa — Wed, 10 Aug 2022 14:10:00 +0000

Design patterns are typical solutions to common problems in software design, which are tried and tested.
Each pattern is like a template that one can customize to solve a particular design problem in their code.
However, we need to make a check before applying if it is truly required or simpler existing code would do better.

Based on purpose, there are 3 categories of patterns :
Behavioral : dealing with communication between objects
Structural : define relationships among various entities (objects, classes etc), via composition, inheritance etc
Creational : related to object creation mechanisms

USE-CASES :

When one needs to decouple senders and receivers : Chain of responsibility, Command, Mediator and Observer Patterns can be used.

Chain of Responsibility handle or forward a request along a chain of potential receivers, based on run-time conditions.

Command normally specifies a sender-receiver connection with a subclass.
Polymorphism is important to Command. In this pattern, we can encapsulate a request into a stand-alone object that contains all information about the request and pass is as method arguments, log, delay or queue it's execution and support undoable operations.

Mediator reduces chaotic dependencies between objects. The pattern restricts direct communications between the objects and forces them to collaborate only via a mediator object.

Observer distributes communication by introducing observer and subject objects, with One-To-Many Relationship.
It is similar to the publisher-subscriber pattern, but with the difference that there is no message broker or event bus as third party in Observer Pattern.
Also, the observers and subject communicate directly, mostly synchronously and subject maintains a record of the observers.

Memento represents the internal state of an object at a particular time which can be captured and restored later using encapsulation.
(Vocab Hint : 'Memento' means 'an object kept as a reminder of a person or event')

Chain of responsibility and Command Pattern represent requests as objects.
When we need to pass requests as token and need to invoke at a later time, Command and Memento Patterns can be used.

State Pattern alters an object's behavior when its state changes. The State pattern is closely related to the concept of a Finite-State Machine.

Strategy encapsulates an algorithm inside a class.

Template method describes the skeleton of a program in parent class. It then defer the exact steps of an algorithm to a subclass.

To select algorithm on run-time, Template method uses inheritance, while Strategy uses composition and delegation.

Interpreter implement a specialized computer language to rapidly solve a specific set of problems.

Null Object is designed to act as a default value of an object.

Iterators are used to access the elements of a collection sequentially without exposing its underlying representation.

Visitor defines a new operation to a class without change. It is a way to separate an algorithm from an object.

Iterator can traverse a Composite. Visitor can apply an operation over a Composite.

References :

sourcemaking
refactoring.guru
w3sdesign
wiki.c2
wikipedia
wikipedia
wikipedia
wikipedia
Blogs
wikipedia

Quick Guide to YAML

jainnehaa — Fri, 05 Aug 2022 12:40:46 +0000

YAML is a data serialization format and processing model, used extensively for log files, Internet messaging, filtering and can be used in conjunction with other programming languages.
In computing, serialization is the process of translating a data structure or object state into a format that can be stored or transmitted and reconstructed later possibly in a different computer environment. XML, JSON, BSON, YAML, MessagePack, protobuf are some commonly used data serialization formats.
YAML is a superset of JSON, so JSON files are valid in YAML.
YAML stands for 'Yet Another Markup Language' or 'YAML Ain’t Markup Language', which emphasizes that YAML is for data and not documents.
YAML files can be added to source control, such as Github, so that changes can be tracked and audited.

YAML uses indentation to indicate nesting. Tab characters are not allowed, so whitespaces are used instead. There are no usual format symbols, such as braces, square brackets, closing tags, or quotation marks. YAML files use a .yml or .yaml extension.
The structure of a YAML file is a map or a list. It also contains scalars, which are arbitrary data encoded in Unicode, that can be used as values such as strings, integers, dates, numbers, or booleans.

YAML parser is used to read YAML documents and provide access to their content and structure.
YAML emitter is used to write YAML documents, serializing their content and structure.
YAML processor is a module that provides parser or emitter functionality or both.

YAML Syntax Example :

---
# An employee record
name: Martin D'vloper
job: Developer
skill: Elite
employed: True
foods:
  - Apple
  - Orange
  - Strawberry
  - Mango
languages:
  perl: Elite
  python: Elite
  pascal: Lame
education: |
  4 GCSEs
  3 A-Levels
  BSc in the Internet of Things

Three dashes indicate start of a new YAML document. YAML supports multiple documents and compliant parsers will recognize each set of dashes as the beginning of a new one. The construct that makes up most of a typical YAML document is a key-value pair.
YAML supports nesting of key-values, and mixing types.

YAML has been criticized for its significant whitespace, confusing features, insecure defaults, and its complex and ambiguous specification.
Configuration files can execute commands or load contents without the users realizing it. Editing large YAML files is difficult, as indentation errors can go unnoticed. Truncated files are often interpreted as valid YAML due to the absence of terminators.

The perceived flaws and complexity of YAML has led to the emergence of stricter alternatives such as StrictYAML and NestedText.

References :
Redhat
yaml.org
yaml.org
wiki
blogs

Heaps Usage : Heapify Vs Heap-Sort

jainnehaa — Mon, 21 Feb 2022 07:50:47 +0000

Definition of Heap : Heap is a Complete Binary Tree, which satisfy the property that parent node value is larger than child nodes' (in case of MAX-HEAP) or smaller (in case of MIN-HEAP). This property of Heap has to be followed by each node with it's child nodes at each level of tree.

Storage in memory : Binary Tree is being stored in memory in form of array as data structure, following level traversal to order elements such that, for each index i as parent, children would be 2i+1 and 2i+2

Heapify(Construct Heap) is an approach (either top-down or bottom-up), which arranges elements in any array such that, when checked for any index i as parent, its child nodes values at index 2i+1 and 2i+2 would follow either MAX-HEAP or MIN-HEAP property. It doesn't sort array in ascending or descending order.

Heap-Sort : Sort array in ascending or descending order. As name suggests, we are using Heap BT property here to rearrange array elements. Heap Sort is an application of Heap BT.

Steps to sort an input array via Heap-Sort:

Arrange array elements in order as of MAX-HEAP or MIN-HEAP, using Heapify(Construct Heap) logic. (Logically it will follow Heap BT property, but storage wise it is still an array)
Put a loop over array, start from root node, compare it with last leaf node, swap if required and delete leaf node. Insert deleted element in a different array. Follow this until all nodes are deleted. Now, when we delete any node in Heap, we need to do Heapify residual array as part of delete operation on Heap BT. And the deleted elements while insertion in array follows order as ascending (MIN-HEAP) or descending (MAX-HEAP).

This is similar to selection sort, which on comparison in each pass, places max/min element to final position and gets excluded in comparison in next pass.