In this post, I will share my experience of making use of two different cloud platforms for the deployment of the front and back end applications our startup has developed in the past few months, which I will most definitely introduce in my next posts.
But today I would like to discuss the main decisions we had to make when architecting the two core parts of our App:
- the client - the interface between your service and your customers
- the backend - whose resources are consumed by the client. The core of your service.
You might be going through a similar process, so hopefully this post will give you some leads and help you picture the options you have.
The Crazy Programmer@thecrazyprogrmr
12:16 PM - 29 May 2020
43
98
Cloud Provider's customer obsession
In the past years, the big cloud service providers such as AWS and Google came up with their own solutions to facilitate development and deployment of applications. There's literally a service for anything available now. Here's a few of many:
- AWS Amplify - AWS-tailored library to develop your applications. Proposes solutions for authentication, messaging, analytics, AR/VR, ML and many more
- Google Firebase - Google's equivalent to Amplify
- AWS Elastic Beanstalk - a service for deploying and scaling web applications and services.
- Google App Engine - equivalent
- AWS Lambda - allows to run serverless code, for ex. an API without servers
- Google Cloud Functions - equivalent
- AWS DynamoDB - frees you from running your own DB server
- Google Cloud Bigtable - equivalent
Given the vast availability of "tailored" cloud services, it might seem surprising that they are yet not that popular among devs. There's only a hand full of startups which decided to go full serverless with AWS Lambda, relying entirely on the execution of Lambda functions. And why would someone outsource essential services like chat, authentication or DBs to a third party if they're feasible to implement on your own?
It is understandable that cloud providers offer those deeply integrated services mainly because they want to bind their customers into their cloud eco system on a long-term basis by promoting these services with reduced price tag. Unfortunately there is no net reduction in complexity for developers working at the application level.
It's simply swapping one set of hardware based computing primitives for an on-demand, cheaper (in terms of TCO), unfamiliar, proprietary set of software-defined computing primitives.
It is naive to think that cloud vendors fail to understand that startups in first place value flexibility and independence. They want to be able to run their Apps regardless of the infrastructure, environment and scale. And it's no secret what vendors are focused on: customer obsession is in fact a well thought through business strategy. And I am not convinced that its primary target are startups.
Dilemma of choice
Choice itself is a challenge. Making any kind of tradeoffs is difficult. Developers hate that and they most certainly they don't want to waste time figuring out which service is best suited for a particular feature of their application, which of the major cloud providers offers the best/most competitive solution and which is more developer friendly. Furthermore, figuring out the interaction between different services is an additional pain. While doing my research on this topic, I came across Cloudpegboard. Without making any advertisement, this is just an example of how the complexity of cloud services can be used as an edge to make business from.
A good AWS-GCP comparison I found recently.
AWS (or GCS) is just hard to use, and it's not your fault. Finding your way around all the services and figuring out how they interact with each other is a time consuming process. When I first started with AWS they had less than 100 services, now it's 283!
Even company employees only know a fraction of those, and I'm always surprised how HRs expect one to have "expertise"/"experience" with AWS (read GCS) without actually specifying any of the services, whose numbers are growing constantly exponentially.
Fortunately, AWS are aware of this and actively help their clients to gain practical knowledge for using their product: They partner with local digital skills training companies which help them organise seminars and summer schools around the world and introduce AWS on a practical basis. I've been to some AWS events in my city and can highly recommend! Especially if you're a beginner/student! It's a great chance to learn and network at the same time. Not only is it an easy way to get promotional credits, but it can also be an opportunity to get a certificate and formally prove your knowledge. That's how I got my first AWS Summer School certificate a few years ago for completing:
- Architecting on AWS (3 Day training)
- Hands-on cloud computing workshop: CI/CD, BigData, AI/ML, IoT (2 Day training)
It is so much easier to learn when you have an expert guiding you through the process. I would have probably never touched Lambda, Cognito or CodeCommit/CodeDeploy on my own, without need.
Setting requirements
Sometimes this helps eliminating the dilemma of choice. If you have clear requirements from the start in terms of architecture and cost, the choice may be obvious, like in our case for example:
Architecture constraints:
During the architecting phase of our application, we have set the following constraints:
- Stay as platform agnostic as possible
- Keep full control over the API and DB
- Use the cloud for its original purpose - deployment and scaling
- Most important: keep it lazy and simple
Resulting design choices:
The evaluation of the above constraints was quite straight-forward:
- Docker. Learn it once, and use it everywhere. Docker is here to stay.
- Use a robust, proven web framework. Our choice fell on Django, mainly because it's written in Python (yes, ML/DL enthusiast here), has a solid development history and major players like Instagram and Bitbucket chose it.
- As a startup, it's relatively easy to cover your hosting expenses for a few years through special startup programs. Most major cloud providers offer these. We happened to have a few k worth of AWS credits, so this was an easy choice. Right now were just running a simple EC2 with above average specs (t2.xlarge - probably an overkill), but we're considering running swarms on ECS if necessary.
- Since we wanted to reach our users on all three main platforms (Web, Apple & Android) - while having one codebase for the client app - the obvious choice fell on react (mobile-friendly PWA/SPA) and react-native.
Confession
Truth is, sometimes it is not practical to refrain from using some cloud services because there is simply no good (enough) alternative:
- Google Maps: of course we could go ahead and spin up our own OSM server with map tiles, geocoding and everything (I have already gone through this process once before), but the high maintenance costs, limited integration and less profesh look (if you're not using additional services like Carto or Mapbox) kept us away from this path. Furthermore, we have received enough GCS credits after enrolling into the Google Cloud for Startups program, and naturally, we wanted to make use of those.
- Google Firebase: It happens that real time chat functionality requires a real time database, and the easiest and most cost-efficient way of implementing this was to use Google Firebase (which has very generous limits for free usage). Unfortunately, at the time of development, Firebase Cloud Messaging was not yet available for our development framework. We will soon shift to this solution as it is free (!) and allows for push notifications.
Conclusion
At first, we were slightly overwhelmed by the decisions we were faced with. But then we just established our base requirements and stepped through a minimal production setup to meet these goals. The essential features we check-off are:
[x] Our setup is cheap and we still have plenty of cloud credits to use.
[x] The Apps run well even under load.
[x] We can scale at any time or run everything from my colleagues basement computer if necessary.
[x] Deployment is easy: we run a command line script which reliably works. Even though it’s rather basic, our system manages package dependencies, versioning and configuration files.
[x] We have all the monitoring we need: external status checks and log files.
[x] We use third party services where its convenient, but could also do without.
Final remarks
As you can see, in the cloud it might be more comfortable to sit on multiple chairs, especially for startups. Unless you're planning really long-term, it may be dangerous to fully commit to one provider. Therefore it is necessary to design your Apps to be as platform-agnostic as possible and not get trapped in a provider's proprietary framework.
Top comments (0)