DEV Community: Prasanna

What I did to become AWS certified 😎

Prasanna — Sat, 04 Dec 2021 06:49:28 +0000

I recently passed the AWS SAA certification 🎉, penning my experience here.

I have been working with AWS for the past ~3 years on and off based on the projects and client needs. One of the challenges for me is that I was always reactive. Usually the needs are EC2, ECS, S3, Cloudfront, Cloudwatch and RDS and that's it. I haven't tried lots of their services, and I wasn't even aware of the optimisations that can be done on both cost as well as in the architecture front.

Decided to pick up this certification AWS SAA to proactively design and build secured, resilient and cost optimised architectures upfront and not to wait for the issues to pop up and then optimise. This is my first certification and certainly re-lived my college exam days while preparing for this exam.

Preparation

Started this around ~3 months back, but was very much overwhelmed with the amount of materials (courses/ notes) that's on the internet. Eventually settled on the following things,

Udemy course by Stephane Maarek 🎥 This is the longest course (~27hours) I have ever signed up for. This is an excellent start if you are looking to get this certification. This gives an intro and decent depth on all (almost) the AWS services. If you follow the hands-on approach along with Stephen, you will be equally confident in managing these services.
This Github notes 📝 Though the videos are exhaustive, it's impossible to remember everything that's in there. That's where these Github notes helped me. Whenever I had some 15 mins, i goto a section in here and read through that.
Anki Cards: 📇 Using flashcards to revise and memorise is my habit for a long time. Thanks to this reddit user, there is an excellent summary of all the services. You can import them to the anki app and you are good to go. 20 cards a day and your certification isn't far away 😁
Took the practice exams in Udemy. ✍🏼 This is an eye opener for me. I can understand and relate to most of the questions (thanks to the course and notes) but this is where we need to apply the knowledge we learnt and it wasn't easy. I decided to take all the tests in the course as open book types. After a couple of tests, I realised an open book takes a lot of time and decided to make some guesses and mark those guess questions for later review.

On the day experience

I took the exam through Pearson Vue online. Make sure to keep the work desk (wherever the laptop/computer is placed) clean. They do check that 😅
You can check in 30 mins before the scheduled time, and likely the exam will also start early.
One day before the exam, go through the system check and keep things ready.
You will have your video and mic turned on throughout the exam, you can't mute or turn off the video.
My guess on the evaluation part. There are lots of questions where they offer partial marks. So the score isn't binary (right or wrong). so don't give up halfway, if you have done the preparations well you are likely to pass the exam.

Some tips if i want to re-do this

Book the exam dates upfront. There isn't enough motivation to run through the gazillion notes, so have a milestone. It isn't a big problem, since you can amend the dates twice.
Pair with a like minded person. Again not to drop the ball on the certification and have someone to talk to.
You can't master concepts along with the details in one go. accept that. so don't spend too much time on one thing.
Spend more time on model exams/tests. From a time perspective, split it 50-50 roughly. Spend 50% time to go over the video courses, understanding the concepts and the remaining 50 to go over the model tests.
For non native english speakers, there is a provision to get 30 mins extra time for the exam. https://aws.amazon.com/certification/policies/before-testing/ (ESL+30). It's good to take this time, i cut close to the finishing 140 mins.

Happy preparing, and all the best !!

Micro-services Patterns: Saga - to music or to dance?

Prasanna — Thu, 10 Jun 2021 06:37:16 +0000

Cross posted from Prasanna's blog

What is a Saga pattern

Saga design pattern is a way to manage a single business transaction that spans across various micro services. Saga pattern breaks a single business transaction into a sequence of local transactions that updates each service and publishes a message or event to trigger the next local transaction step. If any of these local transaction fails, saga will execute the subsequent flows to rollback and cleanup the transaction.

Why do we need this

In a micro-services architecture, it's hard to trace where the transaction has failed and what needs to be rolled back etc. Saga brings in a structure to deal with this problem and also allows the micro-services to act within the specified boundaries.

Implementing Saga

Saga is implemented by adding a transactionId to all the local transactions. A transactionId represent a single business transactions, so with that as an identifier it is possible to trace all the corresponding service interactions.

This managing of local transactions can be done in 2 styles.

Orchestration style
Choreography style

Orchestration Style Sagas

Orchestration pattern mimics an Orchestra, where each person (system in this case) waits for the conductor (another system) to give instructions on what needs to be done.

I don't prefer this design usually for the following reasons,

Tight coupling between the conductor and the other systems in the ecosystem.
Conductor is a single point of failure.

Choreography Style Sagas

Choreography pattern mimics a dance performance where each dancer knows their role and can perform it independently. Hence, there is no need of centralised conductor role.

Some benefits of this pattern are,

Faster development. Teams can build independently, with defined contracts.
Loose coupling, easier to change systems.
Better fault tolerance. There is no single point of failure here

A Sample Scenario

Let us walk through a sample use case and see how we can go about solving it using choreography style sagas.

Since we are talking about music and dance, let me take a use case of booking a movie ticket.

A customer can purchase a movie ticket by paying online.
Once the payment is successful, the movie ticket is confirmed to that customer.
If there is a payment failure, no ticket will be issued, and the order stands cancelled.
If the show is cancelled, the customer will be fully refunded.
If the customer choses to cancel a ticket, then they will be refunded a partial amount only.

Identifying Events and Commands

Now, to implement them independently by various services, we need to identify the boundaries of various services and also their resposibilities.

Event storming is one activity that the team can do to identify these events aka boundaries of responsibilities.

Results of the event storming looks like this:

I will be using the following notion to illustrate various commands and events involved in the above use-case.

Once these events and commands are identified, teams independently can go ahead and start implementing them. Rollbacks are just another events mapped to a different command.

This gives a power to the team to act independently and reduces the bottleneck on a single service.

Slicing Microservices

Prasanna — Sun, 04 Apr 2021 08:36:34 +0000

Cross posted in prasanna.dev

Building applications using micro-services are becoming a default go-to architecture these days. I have been part of few teams that build and deploy micro-services in a large scale.

One pertinent question that often asked is "Did we slice it right?"

What is slicing a service mean

Slicing a micro-service refers to defining the boundaries of the micro-service.

What should the service be responsible for
What kind of data should it hold
When it should delegate it's responsibility to another micro-service.

Below are some of my experiences, that i have seen working.

When to do it

One of the common behaviour we did in the teams I worked so far, is to let micro-services evolve organically. We add feature/capabilities to the existing service and later trim down the service by spawning a new one.

Some of the benefits,

No need of upfront discussion. Most likely we will have lesser information about the feature, then likely that our design might reflect the incompetencies.
Spawning a new service will have it's own cost. How much ever you automate, it still adds up the infrastructure cost, maintenance cost. If we are wrong about the slicing then we had to spend some more time and effort to unify it with some other service.
Once we have built a feature, most of the people in the team will understand the use-case and will appreciate the need of a separate service. It doesn't become a single person's decision or a group of architect's decision. But a decision that comes from ground up. There is a better chance for the service the retain it's shape when it grew this way.

This approach does require a good discipline in having a constant check on the growth of a service. A highly coupled service is very hard to break down later. And if we are too late to cut down a service, it might become an expensive operation too.

How to do it

Here are some of the themes i have come across. I will try to explain my thoughts using a bare minimum add-to-cart like domain problem.

Entity based slicing

Very common and obvious start for a new service.

Example: UserService dealing with the CRUD of a User entity in the system.

It's easy to conclude entities (domain models) as boundaries, since it's intuitive for people to see the separation. But whether it's right? is highly questionable and depends a lot, on the use case.

It's simple, easy and often end up in chaos

One of the significant problems, i have seen is that these services will be very much in demand by other service. They entire network becomes very chatty. One can assume the User entity will be needed at each and every step of the application and will have lots of interaction. Worst, is when the services decided to retain their copy of the data to enhance the performance of the application.

Orchestrator will become a monolith. Since there will be lots of entity services to do mundane operation, orchestrator will become the one service to hold the business logic.

Journey based slicing

Slicing services based on user journeys and behaviour. This is quite popular, especially among the product people. Mostly, the product evolution happens on a feature/journey based. So whenever a new journey is identified it's time to build a new service.

Ex: RatingService a service that allows you to rate an entity. It can be products, people or article etc. Behaviours can include to make sure you don't rate same article twice, compute average ratings etc.

One of the advantage of this technique is that usually the teams i have worked in the past, they own the journey and hence it's clear for them what needs to be the part of this service

Huge drawback I have seen with this approach, is that it forces the data being duplicated across services. In order to maintain the true flavour of Microservice, we will end up having independent databases and eventually having duplicate data

Best of both worlds

It makes a lot of sense to combine the above 2 approaches. Identify the core entities (domain models) of the system, and have them as either independent or logically grouped service. Apply journey based slicing on top of this entity services. So, the teams will own the journey service and the entity services can be maintained by group of teams.

Some of the conventions that i have seen/worked while slicing Micro-services

Entity services should be thin and lenient.

Entity services should be merely act as APIs to the database operations. Do not try bringing in business logic here.

For example, making the email-address non updatable in the entity service level. It makes sense to not to allow the end-user to allow update the email addresses, but often that might be a need from a back office personnel (admin user). So restricting such operation in the entity level might not be worth it. Journeys should take care of validations.

It's hard to predict the future requirements so keep EntityServices simple and open for extension.

Avoid journey service calling other journey services

Journey services, should be independent of others. Store data that are necessary for the journey and use entity services to collaborate with common data.

Build composite entities wherever needed

Now, to answer the immediate question that will raise due to the above constraint. How to manage the duplicate business logic. To be honest, DRY principle is overrated in my opinion. But in case if you are looking for such thing, then try adding another layer of composite entities.
These composite entities, will encapsulate multiple entities and some business logic around this. One classic example from the app we built is a tax computation service. It involves the product, and the location of the buyer to calculate the tax.

All these things are from my past experience, I'm sure I'm going to learn more and course correct myself in this journey. But one thing that i feel will always help in evolving micro-services is to constantly question the slicing decision to get it at a right state. And also a good knowledge on the Domain driven design helps a lot to take these decisions.

Scaling Applications Using Micro-Frontends

Prasanna — Mon, 25 Jan 2021 18:58:06 +0000

Cross posted from Prasanna's blog

This blog post is a summary of a presentation that I made at the Archimydes Mastermind Event that happened on 17th Dec 2020.

Modern web applications tend to have complex and feature-heavy Frontends when compared to backends.

With so many choices for frameworks and programming paradigms, building a consistent Frontend to scale is a challenging problem to solve. We cover ways in which you can scale your Frontend application and your development teams by using a Micro-Frontends design pattern.

I'll start by introducing the pattern of Micro-frontends first. Then we'll be looking into some of the key decisions that need to be taken while starting a Micro-frontend project. Finally, we will see the circumstances where this pattern will be effective.

1. Scaling Applications

In general, scaling applications implies scaling backend applications to serve an increasing number of users. Usually, it's about how to:

Increase Performance
Reduce Latency
Sustain load
Manage compute costs

All these parameters are typically applicable for the backend applications.

For frontend applications, we typically stop with a good CDN to deliver static assets efficiently. However,

Scaling frontend apps is also about scaling development teams, both by size (growing a size of a single team) or count (multiple teams)

Additionally, applications are getting more frontend heavy because:

backends are getting easier to deploy and get off the ground
end-user compute is getting cheaper and more powerful everyday
more functionality is being pushed to end-user interfaces and devices

As a result of this, product teams need to figure out an efficient way to build and deliver frontend applications with multiple development teams working at scale. Product teams need to execute this while reducing bottlenecks in the development process.

2. Monoliths, Microservices and Micro-frontends

Monoliths are not a bad design choice

It's always best to start any application as a monolith. Upfront slicing of module boundaries is very hard and tends to go wrong. As the application grows, it's better to identify module boundaries and split them up.

Microservices

From monoliths, the best choice to evolve the backend services as microservices. We can then guarantee:

Strong module boundaries
Independent deployment
Polyglot development and tech diversity

However, most of the microservices I have seen are as follows

Independent deployments ! == Independent releases

Teams are able to develop and deploy backends independently. However, they need to wait for the frontend to be developed and deployed.

Enter Micro-frontends

Micro-frontends are nothing but taking the concept of micro-services to the frontend. Slice the frontend of the application to respect the module boundaries of the backend, and create an end-end independent release path.

All of Microservices' promises + Independent releases

Gains with Micro-frontends

Independent teams
Independent releases
Simple, decoupled codebases
Incremental upgrades

Problems that need solving

T*o 'share, or not to share'?* - Code reusability is one of the most overrated principles in software development. The problems of reusability are often ignored or not shared. In going the micro-frontend way, this needs to be discussed among the teams. Out of the gate, a duplicate first strategy works since it allows teams to execute faster initially.
Application loading performance - Micro-frontends can cause an impact on the loading performance of the application. There are ways to mitigate it, but the effort it takes has to be taken into consideration.
Design consistency across the application - Having a larger number of people working on an application will lead to inconsistencies. Again, there are ways to mitigate this, however, the effort involved in mitigation needs to be considered.

3. Key decisions while doing Micro-frontends

Let's go over some of the major decisions that we need to take during the early stages of a micro-frontend application. I will try to cover the solution(s) that we took while building an application with distributed teams across 3 regions for 2 years. The decisions can vary based on the project context but nevertheless these problems need to be solved.

In order to explain the challenges and decision, I will take up the following use-case:

Building an application to allow user to configure and buy a laptop. Similar to that of Apple's.

A user can configure a laptop with various components, accessories, protection plans, etc. The user should be able to search for accessories, or maybe built-in models, and then finally should be able to order the product and get it fulfilled.

Apart from the 3 services - configure, search, and order, I will have another service called "Frame" merely to hold the application together.

Frame: A business logic agnostic orchestrator service that knows how to download the rest of the services' frontend

A) Composing multiple front-ends into a single application

End users don't care about the tech used. Their experience should not be affected due to tech.

Composing multiple frontends into a single application is one of the first problems that needs solving when choosing micro-frontends.

Composing front-ends

We can achieve this composition in 2 ways, let's go over the pros and cons of these approaches.

Build-time Composition vs Run-time Composition

Build-time composition is where multiple frontend applications are built as a single big application and served. This can be accomplished using npm or yarn packages.

Pros:

Good dependency management, resulting in smaller bundles
Independent cross team development

Cons:

A monolith built by different teams
Non atomic deployments

A Run-time composition is where the frontends get integrated into the browser directly when the user requests a page. This may be done on the "Server-Side" or in the "Client-Side"

Pros:

Independent teams with independent deployments
Atomic deployments, so no versioning issues

Cons:

Too many API requests from Client(?), with increased bundle size

Toolkit options for Runtime composition

Server side:

SSI (Server Side Includes)
Tailor (from Zalando)

Client Side:

JSPM
SystemJS
FrintJS
Single-Spa

We chose Run-time composition for the project we worked on. Since our app was rendered on the client-side, it was simpler for us to achieve this.

B) Communication between the frontends

Multiple frontends need to share data with each other. Though this needs to be minimal, it's unavoidable. A couple of options to achieve this is by:

State management tools

A global store in the application and all frontends using the same library to access the store.

Window events

Another approach could be to utilize the window (DOMs) eventing capability. Below is a sample event.

We used to communicate through common redux store and redux events as all the apps in our micro-frontends were using Redux.

C) Design Consistency

One of the hardest problem to solve for is design consistency.

In our team, we addressed this challenge by forming guilds. Consider that there are three teams, and each team has a designer assigned to it.

We formed a guild comprising of all designers and some interested developers. They encompass a virtual team. They take all the design decisions and make sure their respective teams are abiding by the central design tenets.

Initially, the guild created a style guide for the application. Mainly CSS and the application teams copy-pasted it from the style guide to build components.

As we developed more features, we started pulling out Higher-order JS components and made them sharable. This is more of an evolution and works well once you have a stable design system in place.

And also, since the teams were using the same frontend framework (React) it was easier for us to build this component library.

D) Testing Strategy

Deciding on "How to test" is important. Since it's a relatively newer paradigm and there are lots of moving parts in the application.

Primarily we will be discussing the "Integration tests" and "Functional tests" from the testing strategy, as there won't be much difference in the way the "Unit tests" are done.

Integration tests

Having a lightweight "Consumer Driven Contracts" (CDC) helped us a lot.

A CDC is where the consumer services' give some tests to the provider service. A provider has to run all of its consumer services before publishing an artifact for deployment.

This doesn't need to be very complex and can be done quickly using some lightweight options without using any big frameworks. But then, it's all case by case.

In our scenario, Frame was the consumer of all the services and it shared a simple JSON contract and a small JS test with all of its providers. This ensured that the frame wasn't broken when a service deployed automatically.

Functional tests

This is one of my least favorite testing methods, however, like everything else in tech, it does have some staunch supporters and followers. In our case, we only had a few critical and successful user journeys automated using Selenium for functional testing.

These journeys cut across multiple services and hence are harder to develop and maintain. Some of the FAQs I usually get on these tests are

FAQs

Who owns functional tests?

Ans. The product team and business analysts. They define the scenarios for automation.

Who writes functional tests?

Ans. Guild containing QAs from all teams and a few developers.

Who fixes functional tests?

Ans. Team which breaks it.

When should you opt for Micro-frontends?

Micro frontends are not for everyone. It adds significant overhead with development and maintenance.

A. Distributed self-contained teams, with a need for parallelization

If your development teams aren't co-located, and there is a decent amount of parallelization that needs to be done, this could be a reason to implement micro-frontends.

B. Collaborate with different frameworks in the frontend

Imagine you are inheriting a legacy application but want to build a new feature with modern design elements, then micro-frontends gives you a good head start.

C. Teams that have experience building Microservices application, and are willing to take it to the next step

Most of the points mentioned here are forward-thinking practices. Micro-frontends needs a good solid understanding of the domain and good discipline to contain things within one's boundary.

Finally, it's worth remembering that:

It's not a sprint. It's a marathon.

Micro-frontends adds significant overhead to the overall application. This isn't desired for smaller applications or for the application that will be built and managed by a single team. The above mentioned challenges are worth solving, only if you are up for a longer run with multiple teams.