DEV Community: Amr Hassan

System design template

Amr Hassan — Mon, 19 Dec 2022 09:12:56 +0000

Most of the time I forget things in my system design interview or waste some time listing things, so I created the below template to help me with that and fasten the process a bit, feel free to use it and please let me know if you have some suggestions

MongoDB vs DynamoDB for your (side) project

Amr Hassan — Sun, 04 Dec 2022 06:45:25 +0000

So here I'm wanting to create an MVP for a side project of mine and look for the cheapest DB aka cheapest data provider to use and let my baby see the daylight with the possibility of scaling and paying less money of course, but before boring you with the comparison I need to tell you more about my use case.

Data access layer (Functional)

I need to list my data accessing pattern first which were:

Insert schemaless data (obviously)
Update the last seen of a record
Query the last 5 elements

In the future queries:

How many updated records per week, month, year

Going to ignore the non-functional, come on it's a side project don't judge me

Data ETA

Assuming that most of the data types going to be text/string with a maximum size of 100KB per record.

How many records going to store per week or month?

10 in the day, 70 in the week, 300 records per month per user

Assuming that we want to support 100 users as a start

100 users * 300 records => 30K per month with the size of 30k * 100KB => 3GB per month

DynamoDB vs MongoDB

	DynamoDB	MongoDB
storage free tier	Free 25 GB of storage AWS Free Tier	512MB to 5GB of storage Shared RAM Upgrade to dedicated clusters for full functionality No credit card required to start
vendor lock-in	amazon specific svc	could use it on your side or Atlas's side
monitoring	provisioning, monitoring, and keeping servers running (not to mention making backups).	none unless you are using Atlas and would have limited feat compared to a dynamo
security	better	meh
record size	400kb	16 MB.
query	basic key-value lookup without the elastic map reduce	better querying framework
data types	number, string, and binary	int, long, date, timestamp, geospatial, floating-point, and decimal128
consistency	eventual data consistency No data validation - must be handled in application ACID transactions apply to table data only, not to indexes or backups Maximum of 25 writes per transaction	strong data consistency Native schema governance and data validation ACID transactions apply to documents, indexes, and backups 1,000 operations per transaction (executing within 60 seconds by default)
connections	HTTPS - kinda unlimited	TCP - M0/M2 cluster tier is 500 per node
indexing	Limited / Complex to manage Indexes are sized, billed & provisioned separately from data	Robust / Easy to develop against No extra cost to index your data

For my use case dynamo fail on the querying side which is important for me compared to the storage and scale for now at least.

Someday will have a use case to abuse dynamodb with but for now, my rational mind says no, for now, how about you would you do the same?

Did you know how discord/zoom works?

Amr Hassan — Mon, 03 Oct 2022 19:41:33 +0000

The secret ingredient is WebRTC.

Here I think that the communication protocols I need to pick when developing a new solution are GRPC, REST, Websockets, and GraphQL.

Thanks for reading Amrs Newsletter! Subscribe for free to receive new posts and support my work.

Since we are talking about real-time communication by default we will drop others beside WebSockets.

thats when I saw WebRTC when reading this discord article.

What is WebRTC?

WebRTC is a fully peer-to-peer technology for the real-time exchange of audio, video, and data, with one central caveat. A form of discovery and media format negotiation must take place.

in order for two devices on different networks to locate one another. This process is called signaling and involves both devices connecting to a third, mutually agreed-upon server. Through this third server, the two devices can locate one another, and exchange negotiation messages.

So here is my short reflection:

Good side

Fast as long as your use case doesnt require all packets delivery in order

Uses UDP which explains why its faster than WebSocket which uses TCP.

peer-to-peer technology and once the connection is up and running, you do not need to pass the communication via a server (unless using TURN).

Secure

With WebSockets, the data has to go via a central webserver which typically sees all the traffic and can access it.

With WebRTC the data is end-to-end encrypted and does not pass through a server (except sometimes TURN servers are needed, but they have no access to the body of the messages they forward).

I was curious to see its compatibility with all browsers; it seems solid

source

Bad side

Require a singling server

complex API compared to WebSockets, but many packages work toward solving that.

When should you use WebRTC instead of a WebSocket?

Rarely. That's the truth.

If you're contemplating between the two and you don't know a lot about WebRTC, then you're probably in need of WebSockets, or will be better off using WebSockets.

I think of data channels either when there are things you want to pass directly across browsers without any server intervention in the message itself (and these use cases are quite scarce), or you need a low latency messaging solution across browsers where a relay via a WebSocket will be too time-consuming.

Resources time:

How to Get Slack Notifications on Your GitHub

Amr Hassan — Fri, 30 Sep 2022 10:48:35 +0000

You probably know that GitHub has an application that you can add to your slack workspace.

While this app is nice and dandy if you want to get notified about *all * actions that happen on GitHub and keep subscribing and unsubscribing to the repositories you are interested in and could even make it publish reminders on a specific channel but it is very noisy.

After digging more into Github, you can find an exciting feature (scheduled reminders) that could enable you only to get notified for the PR you are requested to review, replies to your comments, or even commits...etc.
In real-time or a timed reminder which is something essential to have if you don't want to miss a single PR or be late on reviewing, so buckle up because I'm gonna walk you step by step on how to do it.

But before all of that make sure that you have admin permission on your GitHub organization or know someone who can allow the Github application to read your repositories info.

Open your profile, then click on settings
Scroll down till you find scheduled reminders
Select the organization that you want to get reminders for
Authorize the app,

you can allow the application to read specific repositories instead of all by clicking on only select repositories.
Configure the reminder to your liking

Here is the ideal setup for me

Bouns tip

You can test your reminder by clicking on the below icon inside the scheduled reminders tap

Message queues: SQS vs Kafka

Amr Hassan — Fri, 23 Sep 2022 23:13:31 +0000

Message queue battle royal SQS vs Kafka

Kafka and SQS are very popular message queues, for Kafka, it's a huge scale and operational pain, SQS with plug and play and far far less operational pain.

It becomes harder to choose which one to use as a developer, as a developer we will always choose the laziest approach which is SQS due to the soft process of just clicking here, and their boof you have a message queue.

When actually picking one on top of another I mean we all know that Kafka is a bit complex compared to SQS, in terms of having many brokers and partitions, replicas, and topics while in SQS you only need to specify your queue name and just publish and consume from it.

But there is more to it than the operational and development cost, what I noticed throughout my work with both is that Kafka is more oriented to handle the notion of many consumers & events is that the publisher needs to be aware of the consumer which starts having an unhealthy pattern of smart publisher that adjust the messages for each consumer, which is kinda bad as it makes the relation between publisher and consumer more coupled and strict.

To elaborate that more let's lead with a story,
So let's assume that we have a delivery system where we have many domains let's only focus on the order domain, at first we need to notify the user whenever the order is on its way, and we define that by a field status over the order's domain.

Now we can choose whether the order domain operation is a single queue/topic or each status is a different queue/topic.
Let's go with a single queue/topic for all order operations called orders

So far there is no difference between Kafka & SQS.

Time passes and we needed to start measuring how long it takes for the order to be delivered compared to our initial estimation which is again with the status thing but different consumer this time.

SQS vs Kafka

With Kafka, it's very easy to add a new consumer over our orders topic using the consumer group notion, but for SQS it would be a new queue.
This also makes the order service know who are the consumers and what they need so every time we need to do something different with the orders we will need to modify the orders service to publish to a different topic as well as integrate a new different queue config.

This is the reason why I don't like the usage of SQS with a domain that requires many events.

Let me know what you think of that in the comments below

When To Go Serverless?

Amr Hassan — Thu, 27 Jan 2022 16:19:05 +0000

I'm sure now most of you, if not all, have heard about serverless and how trendy & sexy it's. But what bothered me the most was when to use one?

But before answering that, let’s answer the what & why first and by knowing that the answer will reveal itself.

What is Serverless?

So what is serverless? Does it mean that there is no server? Nope, that's not the case. It means that you are not responsible for managing your infrastructure. All you have to do is write some logic to be executed when a certain something happens. Certain something could be an event, for example, a file uploaded or an HTTP call.

Why Serverless?

Scaling/unpredictable workload.

While the scaling point could be achieved by many solutions already, for example, Kubernetes with KDA could do the same effect, but again most of the cloud providers will provide this functionality out of the box and will be more efficient as you are only paying for the resources you used.

Cost

Theoretically speaking, cost should be very low as you are only paying for what you use, meaning the function would only work when some sort of event happens, and you are only paying for the time the function took to finish the work.

Simplicity/Rapid development

You only care about your business logic. Everything else is not your responsibility, no server, no number of replicas, nothing you want to do 1 2 3 just write 1 2 3.

Security

It's easy to apply a principle such as least privilege where you define the permissions for each function so that there is no such master function (you shouldn't have) that does it all and have permission for everything besides that the cloud provider should provide the latest updated versions of OS that runs your function so you shouldn't worry about that.

The bad side?

What are the limitations?

Cold Start

The time the cloud provider takes to lunch your function, and it depends on how frequent the function was called sometimes, it takes 1 - 2s.

Can't store things in memory/files.

When your function gets triggered, it runs on an ephemeral disk which is something that gets deleted after the function completes. The same thing applies to the used memory.

Execution time

Some cloud providers limit your function running time to a certain number of minutes, so if you want to process something that takes a long time, it’s a big no.

Vendor lockin

This happens when your implementation & logic rely on the infrastructure of a specific cloud provider. For example, if you are using AWS, you will end up relying on many services & permission related only to AWS, which means if you want to switch to another cloud provider, you will suffer a little.

Hard to test

While frameworks such as serverless & SAM could allow you to run your function offline, your function won’t run without storage or a database. So you will end up using some resources in the process and maybe cost you a bit compared to the normal way.

When to go serverless?

Now, this is a hard question to answer. For me, it depends on the problem you are trying to resolve, so if you are doing a time-consuming task, most likely using functions won’t be good enough for you the same thing for handling real-time apps.

On the other hand, if you want something rapid to develop and you are short on the money side, this is definitely something to consider.

Localizing your android app

Amr Hassan — Wed, 03 Nov 2021 15:23:34 +0000

Intro

Following up on what we did last time here.

Our application started to have different audiences with different languages, so we want to localize our app.

Steps

The good thing is that we have a few texts to change and we already using a reference to the strings.xml

/*
file located in::  src/main/res/layout
*/
android:text="@string/my_button_text"

which have this value

/*
file located in:: src/main/res/values/strings.xml
*/
<resources>
    ...
    <string name="my_button_text">Click!</string>
</resources>

Update all the text inside your app to use `strings.xml`

We also have the text we use for the toast inside our MainActivity.java

Toast.makeText(getApplicationContext(), "Clicked!!", Toast.LENGTH_SHORT).show();

So first thing first let's use the strings.xml instead of having the text inside the main file by doing the following update the strings.xml to have a text for the toast

/*
file located in:: src/main/res/values/strings.xml
*/
<resources>
 ...
    <string name="toast_text">Clicked!!</string>
</resources>

and update the MainActivity.java file to use the new text

Toast.makeText(getApplicationContext(), R.string.toast_text,Toast.LENGTH_SHORT).show();

Now we updated all the texts around our App to use the strings.xml values.

It's time to translate our `strings.xml`

Right-click on the strings.xml, select the Open Translations Editor

After that, you would need to click on the globe icon, to choose the language you want to support, I'll choose Arabic

The last thing is to fill in the values for each text

And the android studio will generate a new strings.xml file for the language you just selected.

Here is the result

Congrats!! now you can support any language you want and have more audience around the globe

See you in the next part

Photo by Dollar Gill on Unsplash

Introduction to android - project structure

Amr Hassan — Thu, 21 Oct 2021 21:13:15 +0000

Intro

The point of this article is to give you an idea about the android project structure.

Are you ready, It's time for a story, we want to build an application with a single page that has a single button, and whenever you click on that button it gives the user a feedback msg that the button was clicked.

So you start by creating an android project with empty activity over android studio but a wild project hierarchy shows up

Android project hierarchy

Here are the most used folders

manifests (you can specify the permissions here, your package name is defined here, every activity, service, everything needs to be mentioned here )
java/your-package (contains your logic, staff like what you want to do when the user opens the app, click on the button...etc)
res
- drawable (Contains logo, images)
- layout (Your pages design and content)
- values (any description, the title basically any text on the UI should be written here & referenced in the layouts)
- build.gradle (Module) (The place where you set your dependencies, add different build types, versioning your builds)

Of course, there are other folders & files but these are the ones that you are going to deal with the most.

The run flow

Image source

When you run the app, the first thing execute is the MainActivity, and any activity has the above lifecycle where it calls onCreate() at the beginning, so there where we will find the main layout rendering.

     protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
    }

Which in return start rendering the mentioned layout inside the /res/layout folder where it contains something similar to this

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

     <Button
        android:id="@+id/my_button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="@string/my_button_text"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent"
         />

</androidx.constraintlayout.widget.ConstraintLayout>

Notice this line

<androidx.constraintlayout.widget.ConstraintLayout  .....>

It says contraintLayout can you guess what it might imply? it implies that there are different layouts.
So what are they? and which one you should use?
There are many layouts for example
StackLayout, RelativeLayout, GridLayout...etc
each one has its own use cases.
But if you have a large and complex design with some sort of relationship between sibling views and parent layout then contraintLayout is the one for you.

for example, if you want to position a text at the middle of the view you can just add to the element definition

        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent"

Notice this line

  android:text="@string/my_button_text"

This will use the value defined in values/strings.xml as long as it has the same name and type

 <string name="my_button_text">Click!</string>

This means when the view renders it will show a button with the text Click!

As you can see the layout has the button defined, but we need to add a click listener over the button so that when the user click it we can do something and that's when this android:id="@+id/my_button" attribute over the button come in handy so in the MainActivity inside the onCreate Method we can say

Button bt = findViewById(r.id.my_button);
bt .setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                Toast.makeText(getApplicationContext(), "Clicked!!", Toast.LENGTH_SHORT).show();
            }
        });

This way whenever the button with key android:id="@+id/my_button" is clicked the func onClick will be executed

Congrats!! you finished the story and you have a button that whenever clicked will show a message.

See you in the next part

Cover photo by Mark Boss on Unsplash

Kubernetes namespaces you should never miss with.

Amr Hassan — Tue, 19 Oct 2021 21:17:15 +0000

Kubernetes is shipped with many namespaces some of them are critical for Kubernetes in order to function correctly.
Messing around in one of these namespaces can damage the Kubernetes system.

And these are:

default the home of the homeless
kube-system The namespace for objects created by the Kubernetes system
kube-public This namespace is created automatically and is readable by all users (including those not authenticated). This namespace is mostly reserved for cluster usage, in case that some resources should be visible and readable publicly throughout the whole cluster. This is useful for exposing any cluster information necessary to bootstrap components. It is primarily managed by Kubernetes itself.
kube-node-lease This namespace holds Lease objects associated with each node. Node leases allow the kubelet to send heartbeats so that the control plane can detect node failure.

All Kubernetes system's namespaces will be regenerated again even when you delete them accidentally that's what Kubernetes components will try to do.

But sometimes if you are unfortunate enough deleting namespaces might be stuck at the terminating stage giving no option to regenerate the namespace again.

So the below cases mention what is the importance of each namespace in order to know what the symptoms might look like.

What is the `default`?

default namespace is used as the default place of any objects you create with no namespace specified.

What is the `kube-system`?

Kube-system is the namespace for objects and service accounts with high-level privileges within Kubernetes. Utilization of the Kubernetes controller stems from this namespace, in other words we will have some issues with the controllers and maybe in deploying new pods/deployment.

not only that this namespace also contains other important objects such as kube-dns,kube-proxy,

kube-dns is the authoritative name server for the cluster domain (cluster.local) and it resolves external names recursively. Short names that are not fully qualified, such as myservice, are completed first with local search paths. more details here & here

kube-proxy manages the forwarding of traffic addressed to the virtual IP addresses (VIPs) of the cluster’s Kubernetes Service objects to the appropriate backend pods more details here & here

This means you will have trouble with resolving external/internal communications.

How about the `kube-public`?

kube-public contains a single ConfigMap object, cluster-info, that aids discovery and security bootstrap.

All the above namespaces if you tried to delete them the server will respond with

Error from server (Forbidden): namespaces "kube-public" is forbidden: this namespace may not be deleted

Expect the lucky kube-node-release which was added in Kubernetes v1.14, would get deleted the same as any normal namespace.

What is `kube-node-release`?

kube-node-lease This namespace holds Lease objects associated with each node. Node leases allow the kubelet to send heartbeats so that the control plane (Node controller) can detect node failure.

So what would happen if we deleted kube-node-lease? usually, Kubernetes will create another one with Lease object for each node, but sometimes the namespace removal gets stuck at the terminating status.
Which by then we will have a node Lease with an outdated heartbeat which might tell the Node controller that this node is not reachable and impact the overall communication between the nodes.

How to fix namespace removal stuck at `terminating`?

Of course, you can try to figure out why the namespace is stuck at terminating but sometimes you can't so here we go with the force delete

1- create a temp json file

 kubectl get namespace <terminating-namespace> -o json >tmp.json

2- execute the below

$ kubectl proxy
Starting to serve on 127.0.0.1:8001

3- Edit your tmp.json file. Remove the Kubernetes value from the finalizers field and save the file.

4- Make the below command to update the namespace

 curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8001/api/v1/namespaces/<terminating-namespace>/finalize

5- Your output will look something like the one below

{
   "kind": "Namespace",
   "apiVersion": "v1",
   "metadata": {
     "name": "<terminating-namespace>",
     "selfLink": "/api/v1/namespaces/<terminating-namespace>/finalize",
     "uid": "b50c9ea4-ec2b-11e8-a0be-fa163eeb47a5",
     "resourceVersion": "1602981",
     "creationTimestamp": "2021-10-18T18:48:30Z",
     "deletionTimestamp": "2021-10-18T18:59:36Z"
   },
   "spec": {

   },
   "status": {
     "phase": "Terminating"
   }
}

DEV Community: Amr Hassan

System design template

MongoDB vs DynamoDB for your (side) project

Data access layer (Functional)

Data ETA

DynamoDB vs MongoDB

Did you know how discord/zoom works?

What is WebRTC?

So here is my short reflection:

Good side

Bad side

When should you use WebRTC instead of a WebSocket?

Resources time:

How to Get Slack Notifications on Your GitHub

Bouns tip

Message queues: SQS vs Kafka

Message queue battle royal SQS vs Kafka

When To Go Serverless?

What is Serverless?

Why Serverless?

The bad side?

When to go serverless?

Localizing your android app

Intro

Steps

Update all the text inside your app to use strings.xml

It's time to translate our strings.xml

Introduction to android - project structure

Intro

Android project hierarchy

The run flow

Kubernetes namespaces you should never miss with.

What is the default?

What is the kube-system?

How about the kube-public?

What is kube-node-release?

How to fix namespace removal stuck at terminating?

Update all the text inside your app to use `strings.xml`

It's time to translate our `strings.xml`

What is the `default`?

What is the `kube-system`?

How about the `kube-public`?

What is `kube-node-release`?

How to fix namespace removal stuck at `terminating`?