Tearing Vitcord Backend

What is Vitcord? What do we do?

Vitcord is a startup based in Valencia, Spain born 4 years ago.

We are trying to make video stories more valuable. People share videos because they want their friends to partake in their experience: to share the mood that they're in, to see the special place they're at or the event that they're enjoying. With Vitcord, you engage with a video on a whole new level, not only with your friends, but also with people who love the same things that you love. It's an open space for video creators, where you can start telling stories with videos, and people around the world can join your stories.

Our Monolitic API

When we started developing Vitcord’s backend API, we couldn’t imagine ourselves at the point when we would be considering to move into a Micro-service approach. Right now, our API manages 4 main services written in Ruby (Ruby on Rails):

• Social Network service (create a new account, follows other users, manage our profile etc)
• Upload and process files (upload a new video, store videos, manage thumbnails etc)
• Manage Notifications (Push notifications, notifications list etc)
• Video threads (create a new thread of videos, management of threads etc)

We think it was a good approach to have all the services into one API and use Rails as our main language, as we were a couple of developers and in that moment we needed to build an MVP as fast as we could to start validating if our project made sense.

Now that we have validated that, and as the user base is growing, our Monolithic API is getting to its limits, so we decided to start the process of moving to Micro-service architecture like 6 months ago, with a first “service” to manage Notifications in the APP.

Extracting Notifications to a Service, why?

First of all, we are a Social Network based on interactions between users, so you might guess that we need a tool to keep our users awake and aware about events taking place in our APP, so we have a lot of actions that end up with a Notification stored in DB and a Push Notification(or a pool of Push Notifications) being sent to other devices. As you might imagine, our biggest collection of items in DB is Notifications, and more than that, it is like 10 times bigger than our second biggest collection. The same for code execution, almost 70% of the time we are managing some kind of logic that will be responsible of creating a new Notification in DB + telling Firebase to send a new Push Notification.

So, don’t you think it is time to keep these tasks separated? We thought the same, so we started maintaining two big modules:

• One that manages social network + video management (file uploading, thread management etc)
• Another one that manages Notifications + Push Notifications

This means each time we make a new change, normally we have to change something related to the first module and something related to the second module. This is something weird, as we normally want to add features or just change code attached to the first module, but as our Notifications module keeps growing, it is polluting our full code base. In the end, we encouraged ourselves to start building our first Microservice.

DynamoDB over Cassandra and Kotlin over Elixir

When we started to read about technologies to build a Notification service, we had two things clear:

• It would need to manage lot of requests per second (as in Vitcord we have lots of transactional notifications)
• Store millions of records per second (as each transactional notification has to be saved for a user to be able to show his Notifications history)

Taking this in mind, we got to the point of taking into account a couple of languages to build it (Kotlin and Elixir) and a couple of DB technologies (Cassandra and DynamoDB).

It is clear that Elixir is probably the perfect candidate to build a Notifications service, as it leverages to Erlang VM, meaning that it is great for Scalability as it manages lightweight threads on execution and it is said that Elixir is the natural evolution of Rails (from a syntactic point of view, and also due to Phoenix likelihood with Rails). So, you might be asking Why did you choose Kotlin over Elixir? The answer lies in the fact that no one in the team knows Elixir and we wanted to build the service fast, so we picked one guy from Backend team and one from Android (they build the APP in Kotlin) and we started the development with Kotlin + Spring. In the end, Kotlin makes sense to us as it has coroutines (lightweight threads) that covers our need of great Scalability to manage millions of requests.

The same decision was taken for the DB. As you might know, Cassandra was built thinking on scalability and high availability (with great performance) so it fits our needs as well as Elixir. When we started the development we were thinking on how to manage Cassandra nodes with a Kubernetes cluster. We read a lot about how to manage Kubernetes (we do not have a DevOps-specific person in the team) and at the end we were talking about how much time it would take us to manage and maintain a Kubernetes cluster by ourselves, so we decided to use DynamoDB (which is quite similar to Cassandra, but with the advantage of not needing to manage instances or clusters as it is managed by Amazon) with the promise of returning to Cassandra some day.

Next steps

We are happy with our decisions, as right now for example DynamoDB gives exactly what we need for our Notifications service, and we left management up to Amazon (we know that eventually we will face big charges because of Dynamo, but we will be ready to move DB to Apache Cassandra) and Kotlin is working like a charm.

Right now we are talking about how to fully separate Notifications from our Monolithic API, as for now it is working as Authentication entrance for the Notifications Service, but of course we want to keep each Microservice fully separated from each other.

So those are our next steps:

• Build an API Gateway that will work as an entrance point for our Microservices by managing routing and Authentication through an Authentication service
• Add an Event Queue to our infraestructure (we are thinking on RabbitMQ or Apache Kafka) to communicate microservice actions
• Fully separate Notifications from the Monolithic API
• Add new Microservices (maybe one to manage video and files)

We will cover more about our advances in next posts.

Stay tuned!!