DEV Community: Semion Rozov

How to deliver your App to the masses (without becoming hostage to your cloud provider)

Semion Rozov — Sat, 30 May 2020 16:04:44 +0000

In this post, I will share my experience of making use of two different cloud platforms for the deployment of the front and back end applications our startup has developed in the past few months, which I will most definitely introduce in my next posts.

But today I would like to discuss the main decisions we had to make when architecting the two core parts of our App:

the client - the interface between your service and your customers
the backend - whose resources are consumed by the client. The core of your service.

You might be going through a similar process, so hopefully this post will give you some leads and help you picture the options you have.

The Crazy Programmer

@thecrazyprogrmr

Coding in 2000 vs 2020....

credit: @chrisoncode

#Webdesign #webdeveloper #HTML #javascript

12:16 PM - 29 May 2020

43 98

Cloud Provider's customer obsession

In the past years, the big cloud service providers such as AWS and Google came up with their own solutions to facilitate development and deployment of applications. There's literally a service for anything available now. Here's a few of many:

AWS Amplify - AWS-tailored library to develop your applications. Proposes solutions for authentication, messaging, analytics, AR/VR, ML and many more
Google Firebase - Google's equivalent to Amplify
AWS Elastic Beanstalk - a service for deploying and scaling web applications and services.
Google App Engine - equivalent
AWS Lambda - allows to run serverless code, for ex. an API without servers
Google Cloud Functions - equivalent
AWS DynamoDB - frees you from running your own DB server
Google Cloud Bigtable - equivalent

Given the vast availability of "tailored" cloud services, it might seem surprising that they are yet not that popular among devs. There's only a hand full of startups which decided to go full serverless with AWS Lambda, relying entirely on the execution of Lambda functions. And why would someone outsource essential services like chat, authentication or DBs to a third party if they're feasible to implement on your own?
It is understandable that cloud providers offer those deeply integrated services mainly because they want to bind their customers into their cloud eco system on a long-term basis by promoting these services with reduced price tag. Unfortunately there is no net reduction in complexity for developers working at the application level.

It's simply swapping one set of hardware based computing primitives for an on-demand, cheaper (in terms of TCO), unfamiliar, proprietary set of software-defined computing primitives.

It is naive to think that cloud vendors fail to understand that startups in first place value flexibility and independence. They want to be able to run their Apps regardless of the infrastructure, environment and scale. And it's no secret what vendors are focused on: customer obsession is in fact a well thought through business strategy. And I am not convinced that its primary target are startups.

Dilemma of choice

Choice itself is a challenge. Making any kind of tradeoffs is difficult. Developers hate that and they most certainly they don't want to waste time figuring out which service is best suited for a particular feature of their application, which of the major cloud providers offers the best/most competitive solution and which is more developer friendly. Furthermore, figuring out the interaction between different services is an additional pain. While doing my research on this topic, I came across Cloudpegboard. Without making any advertisement, this is just an example of how the complexity of cloud services can be used as an edge to make business from.

A good AWS-GCP comparison I found recently.

AWS (or GCS) is just hard to use, and it's not your fault. Finding your way around all the services and figuring out how they interact with each other is a time consuming process. When I first started with AWS they had less than 100 services, now it's 283!
Even company employees only know a fraction of those, and I'm always surprised how HRs expect one to have "expertise"/"experience" with AWS (read GCS) without actually specifying any of the services, whose numbers are growing ~~constantly~~ exponentially.

Fortunately, AWS are aware of this and actively help their clients to gain practical knowledge for using their product: They partner with local digital skills training companies which help them organise seminars and summer schools around the world and introduce AWS on a practical basis. I've been to some AWS events in my city and can highly recommend! Especially if you're a beginner/student! It's a great chance to learn and network at the same time. Not only is it an easy way to get promotional credits, but it can also be an opportunity to get a certificate and formally prove your knowledge. That's how I got my first AWS Summer School certificate a few years ago for completing:

Architecting on AWS (3 Day training)
Hands-on cloud computing workshop: CI/CD, BigData, AI/ML, IoT (2 Day training)

It is so much easier to learn when you have an expert guiding you through the process. I would have probably never touched Lambda, Cognito or CodeCommit/CodeDeploy on my own, without need.

Setting requirements

Sometimes this helps eliminating the dilemma of choice. If you have clear requirements from the start in terms of architecture and cost, the choice may be obvious, like in our case for example:

Architecture constraints:

During the architecting phase of our application, we have set the following constraints:

Stay as platform agnostic as possible
Keep full control over the API and DB
Use the cloud for its original purpose - deployment and scaling
Most important: keep it lazy and simple

Resulting design choices:

The evaluation of the above constraints was quite straight-forward:

Docker. Learn it once, and use it everywhere. Docker is here to stay.
Use a robust, proven web framework. Our choice fell on Django, mainly because it's written in Python (yes, ML/DL enthusiast here), has a solid development history and major players like Instagram and Bitbucket chose it.
As a startup, it's relatively easy to cover your hosting expenses for a few years through special startup programs. Most major cloud providers offer these. We happened to have a few k worth of AWS credits, so this was an easy choice. Right now were just running a simple EC2 with above average specs (t2.xlarge - probably an overkill), but we're considering running swarms on ECS if necessary.
Since we wanted to reach our users on all three main platforms (Web, Apple & Android) - while having one codebase for the client app - the obvious choice fell on react (mobile-friendly PWA/SPA) and react-native.

Confession

Truth is, sometimes it is not practical to refrain from using some cloud services because there is simply no good (enough) alternative:

Google Maps: of course we could go ahead and spin up our own OSM server with map tiles, geocoding and everything (I have already gone through this process once before), but the high maintenance costs, limited integration and less profesh look (if you're not using additional services like Carto or Mapbox) kept us away from this path. Furthermore, we have received enough GCS credits after enrolling into the Google Cloud for Startups program, and naturally, we wanted to make use of those.
Google Firebase: It happens that real time chat functionality requires a real time database, and the easiest and most cost-efficient way of implementing this was to use Google Firebase (which has very generous limits for free usage). Unfortunately, at the time of development, Firebase Cloud Messaging was not yet available for our development framework. We will soon shift to this solution as it is free (!) and allows for push notifications.

Conclusion

At first, we were slightly overwhelmed by the decisions we were faced with. But then we just established our base requirements and stepped through a minimal production setup to meet these goals. The essential features we check-off are:

[x] Our setup is cheap and we still have plenty of cloud credits to use.
[x] The Apps run well even under load.
[x] We can scale at any time or run everything from my colleagues basement computer if necessary.
[x] Deployment is easy: we run a command line script which reliably works. Even though it’s rather basic, our system manages package dependencies, versioning and configuration files.
[x] We have all the monitoring we need: external status checks and log files.
[x] We use third party services where its convenient, but could also do without.

Final remarks

As you can see, in the cloud it might be more comfortable to sit on multiple chairs, especially for startups. Unless you're planning really long-term, it may be dangerous to fully commit to one provider. Therefore it is necessary to design your Apps to be as platform-agnostic as possible and not get trapped in a provider's proprietary framework.

Petizio - a blockchain-based petition platform

Semion Rozov — Thu, 21 May 2020 12:40:49 +0000

Last year I had the chance to win one of the largest blockchain hacks in Europe - SBHACK19. In a 42-hour intensive hacking marathon my team managed to produce a working MVP for the eGovernment vertical, originally proposed by the Canton of Zürich, the b2Lab of ETH, as well as the blockchain startups procivis and KILT.

Our dAPP - Petizio - allows users to vote pseudo-anonymously after verifying themselves through government identification. Citizens can then create new petitions and vote through the app without the need of exposing their identity in a way that each vote gets count once.

Official Problem Statement:

While the past years have seen significant efforts in the area of eGovernment, Public Services yet lack powerful solutions for many of the new challenges such as Digital ID, E-Voting and the Management and Registration of Documents. This challenge deals with the questions of how a government body can ensure that information can be updated flexibly while also being accurately and truthfully replicable and how it can prove that the information has not been manipulated.

The key areas here are:

How can centralized public registries be protected, making official documents and credentials tamper-proof and easily verifiable, and easing the burden of the individual to aggregate and share his information between governmental bodies as well as other stakeholders, such as employers, banks or insurances?
How can we ensure trusted and secure standards, to solve the current issues around e-Voting and in order to digitize the fundamental democratic process of voting.
Finally, how can institutions give their own representatives the temporary right to make legally binding signatures?

Our team decided to narrow down the challenge to the problem of petitions.

Old Solutions:

how petitions were signed in the old days

Until recently, there were three main approaches to gathering signatures for petitions:

Physical: Petitions where signatures are gathered on the street through physical copies. This approach has multiple limitations ranging from signature consolidation to information dissemination.
Private Entities: Websites such as change.org digitizes the process of voting and allows users to gather votes and make their issues heard. But the lack of identification creates the problem of a single person signing the petition multiple times.
State Owned Websites: There are solutions provided by the state such as in Germany overcomes the problem of identification of citizens, however they are still vulnerable to attacks and they do not provide full anonymity.

None of these solutions have identity verification nor provide anonymity for citizens.

Our Solution:

If you prefer audio-visual info more, here's a video presentation of our solution:

Otherwise, here's the text version: :D

Our solution is to use governmental facilities to verify the citizens and then register the verified citizens on a private blockchain. The citizens are given pseudo-anonymous IDs ~~that can be identified by the government~~. Based on this premise:

The list of verified pseudo-anonyms are publicly displayed on the blockchain,
Verified citizens can create and vote on petitions on the blockchain.
When a citizen votes they use their anonymised pseudo ID.
Petitioner can then compare the pseudo id published by the government and then count the votes.
The counting of the signatures, creation of signature, and petitions are handled by a smart contract.

The entry process to the blockchain is depicted below:

The development of the project was done as follows:

A GitHub repository was used to coordinate.
The repository was then moved into AWS CodeCommit
A Rest API and Composer was used in an EC2 instance
This EC2 instance was used between AWS Managed BlockChain and Angular.Js using REST:API

Why is this a good solution?

Petizio's main strengths:

One vote is one vote: The confirmation method prevents a single agent to vote multiple times.
Immutability: Nobody can tamper with votes once they have been written to the blockchain.
Anonymity: When the issues are being voted for, it is not known who supports the ideas, people can vote without revealing their identity.

Why was this a great project?

For my team, this was a serious topic, dealing with current issues around trust, security and transparency.

Our main goal was to contribute to resolve these issues and perhaps make a step in the right direction and help establishing a new era in the relationship between governments and individuals.

For me personally, this hack was one of the most fulfilling and rewarding experiences ever. I managed to pick up Angular in less than 4h, did an all-nighter to get the front-end done in time and also helped with deployment on AWS. In the end, we won our vertical challenge and received 18.000 CHF in cash, crypto and sponsor services.

The Epic Aftermovie: