DEV Community: Innocontents Vitcord S.L.

Web API architecture for millenials

Miguel Coleto — Fri, 26 Jul 2019 06:29:39 +0000

At Vitcord we always try to offer the best experience for our users. Recently, we discovered that push notifications management was starting to cause a bottleneck in our monolith so we decided to face this problem before it would become worse.

At the moment we started to study the solution for this challenge, push notifications weren’t a big part of our monolith, so we realized that extracting it to an isolated microservice could be a good idea. Besides, push notifications become a key of success in terms of retention once you have an interesting user base. Extracting this responsibility from our main server means we can scale this part separately and give it a better maintenance too.

Once we decided what we were building we started to think about how we were going to do it. From the beginning, we knew that our database should be Cassandra or DynamoDB, as notifications and messaging are the kind of feature this databases are built for. Choosing the language and framework did take us a little bit longer, but in the end we went for Kotlin and Spring Boot. These two make a perfect couple and some of our tech guys had some experience with Kotlin, besides, the Android team has their app completely in this language.

Choosing the right architecture

When we started designing the system we knew that it wouldn’t have a lot of actions (one for each type of push notification basically), but they could be more in the future. We knew that a vertical oriented approach suited perfectly as it allows to distribute the work easily, plug in or out actions and maintain each vertical separately without touching more code than needed.

After some research we discovered an architectural pattern that fitted our design really well: Action Domain Responder. This pattern was born as an evolution for the classic Model View Controller.

MVC is an architectural pattern for apps with user interfaces born in the 90s. The Controller maintains a state of the program which tells the View how it should be showed to the user. With each change to the state the UI should render automatically. It became very popular in backend thanks to frameworks like Ruby on Rails, which follow this pattern for web applications.

A better MVC for the Web

Action Domain Responder was created in 2014 as a refinement for MVC in the context of web REST APIs. One key principle of REST is that is stateless and each HTTP request is treated independently, so there is no state to keep. ADR proposes a much more natural flow for backend applications:

The Action receives an HTTP request. It collects any required input and calls the Domain with them.
The Domain manipulates the information and applies any business logic. It returns a result to the Action.
The Action calls the Responder with the result of the Domain.
The Responder builds an HTTP response and sends it.

ADR suits our vertical oriented approach because we can fulfill SOLID’s Single Responsibility Principle to the max:

In ADR each Action is a separate class with only one method instead of a Controller with one method for each endpoint. Actions only connect Domain with Responders.
We can model the Domain following a Command pattern which helps us to isolate the business logic and reuse it.
Responders only job is to build HTTP responses based on an input. They can be pure functions.

Every vertical is a set of an Action, a Responder and a Command. This allows us to add or remove them without affecting more components than these. Testing also becomes easy peasy as every piece has a limited responsibility and they usually have one or two methods. Structure is also another plus as it is really easy to find every file by their corresponding Action, and you can tell every feature of the app from a top level look of the Actions.

Theory is good but... show me the code!

Actions define the endpoint they are listening to and only have one execute() method that will be invoked with each request. They call the Domain with the request input, then they call the Responder with the result and lastly they return the response that the Responder built.

Domain is divided in classes following the Command or Use Case pattern. They are in charge of manipulating the data, storing them in database, publishing domain events, etc. As the Actions, they always have one invoke() method which returns an Either datatype from Arrow, the library for functional programming in Kotlin.

Either represents some operation which can fail or succeed. It works like a black box in which we operate, but we won’t know if everything went okay or not until we open it. This is by far better than throwing exceptions because we are telling explicitly that our code can fail and we force ourselves to handle every path.

In this specific case we are telling that this command can return a Notification if everything went right or it can return an Error from the Algebraic Data Type (ADT) we created (Errors), this means the error must be one of the objects under the sealed hierarchy of the parent class.

Responders just need to open the Either box and create a HTTP response based on its content. For this purpose we have the fold() method which accepts two lambdas and executes one or the other, depending if the content is an error or a success.

Modelling the errors in an ADT style forces us to handle every case thanks to Kotlin’s when. If I added another error to the hierarchy, the Responder wouldn’t compile until I handled it in the when block.

In conclusion

We think ADR is a better architectural pattern for REST APIs than MVC and it fits our development workflow really well. It has allowed us to deploy this project in little time and it will be easy to maintain thanks to the responsibility separation, testability and good structure the architecture is giving us.

Thank you for reading this post. Please, give us feedback by commenting or tweet us if you liked the ADR pattern. Stay tuned and follow Vitcord Tech for more posts about our experiences and learnings 🙌

Tearing Vitcord Backend

Samuel Villaescusa Vinader — Thu, 18 Jul 2019 12:46:53 +0000

What is Vitcord? What do we do?

Vitcord is a startup based in Valencia, Spain born 4 years ago.

We are trying to make video stories more valuable. People share videos because they want their friends to partake in their experience: to share the mood that they're in, to see the special place they're at or the event that they're enjoying. With Vitcord, you engage with a video on a whole new level, not only with your friends, but also with people who love the same things that you love. It's an open space for video creators, where you can start telling stories with videos, and people around the world can join your stories.

Our Monolitic API

When we started developing Vitcord’s backend API, we couldn’t imagine ourselves at the point when we would be considering to move into a Micro-service approach. Right now, our API manages 4 main services written in Ruby (Ruby on Rails):

Social Network service (create a new account, follows other users, manage our profile etc)
Upload and process files (upload a new video, store videos, manage thumbnails etc)
Manage Notifications (Push notifications, notifications list etc)
Video threads (create a new thread of videos, management of threads etc)

We think it was a good approach to have all the services into one API and use Rails as our main language, as we were a couple of developers and in that moment we needed to build an MVP as fast as we could to start validating if our project made sense.

Now that we have validated that, and as the user base is growing, our Monolithic API is getting to its limits, so we decided to start the process of moving to Micro-service architecture like 6 months ago, with a first “service” to manage Notifications in the APP.

Extracting Notifications to a Service, why?

First of all, we are a Social Network based on interactions between users, so you might guess that we need a tool to keep our users awake and aware about events taking place in our APP, so we have a lot of actions that end up with a Notification stored in DB and a Push Notification(or a pool of Push Notifications) being sent to other devices. As you might imagine, our biggest collection of items in DB is Notifications, and more than that, it is like 10 times bigger than our second biggest collection. The same for code execution, almost 70% of the time we are managing some kind of logic that will be responsible of creating a new Notification in DB + telling Firebase to send a new Push Notification.

So, don’t you think it is time to keep these tasks separated? We thought the same, so we started maintaining two big modules:

One that manages social network + video management (file uploading, thread management etc)
Another one that manages Notifications + Push Notifications

This means each time we make a new change, normally we have to change something related to the first module and something related to the second module. This is something weird, as we normally want to add features or just change code attached to the first module, but as our Notifications module keeps growing, it is polluting our full code base. In the end, we encouraged ourselves to start building our first Microservice.

DynamoDB over Cassandra and Kotlin over Elixir

When we started to read about technologies to build a Notification service, we had two things clear:

It would need to manage lot of requests per second (as in Vitcord we have lots of transactional notifications)
Store millions of records per second (as each transactional notification has to be saved for a user to be able to show his Notifications history)

Taking this in mind, we got to the point of taking into account a couple of languages to build it (Kotlin and Elixir) and a couple of DB technologies (Cassandra and DynamoDB).

It is clear that Elixir is probably the perfect candidate to build a Notifications service, as it leverages to Erlang VM, meaning that it is great for Scalability as it manages lightweight threads on execution and it is said that Elixir is the natural evolution of Rails (from a syntactic point of view, and also due to Phoenix likelihood with Rails). So, you might be asking Why did you choose Kotlin over Elixir? The answer lies in the fact that no one in the team knows Elixir and we wanted to build the service fast, so we picked one guy from Backend team and one from Android (they build the APP in Kotlin) and we started the development with Kotlin + Spring. In the end, Kotlin makes sense to us as it has coroutines (lightweight threads) that covers our need of great Scalability to manage millions of requests.

The same decision was taken for the DB. As you might know, Cassandra was built thinking on scalability and high availability (with great performance) so it fits our needs as well as Elixir. When we started the development we were thinking on how to manage Cassandra nodes with a Kubernetes cluster. We read a lot about how to manage Kubernetes (we do not have a DevOps-specific person in the team) and at the end we were talking about how much time it would take us to manage and maintain a Kubernetes cluster by ourselves, so we decided to use DynamoDB (which is quite similar to Cassandra, but with the advantage of not needing to manage instances or clusters as it is managed by Amazon) with the promise of returning to Cassandra some day.

Next steps

We are happy with our decisions, as right now for example DynamoDB gives exactly what we need for our Notifications service, and we left management up to Amazon (we know that eventually we will face big charges because of Dynamo, but we will be ready to move DB to Apache Cassandra) and Kotlin is working like a charm.

Right now we are talking about how to fully separate Notifications from our Monolithic API, as for now it is working as Authentication entrance for the Notifications Service, but of course we want to keep each Microservice fully separated from each other.

So those are our next steps:

Build an API Gateway that will work as an entrance point for our Microservices by managing routing and Authentication through an Authentication service
Add an Event Queue to our infraestructure (we are thinking on RabbitMQ or Apache Kafka) to communicate microservice actions
Fully separate Notifications from the Monolithic API
Add new Microservices (maybe one to manage video and files)

We will cover more about our advances in next posts.

Stay tuned!!