DEV Community: Arpit Dubey

My 2 cents on Deno 🦕

Arpit Dubey — Fri, 15 May 2020 22:46:22 +0000

With the emergence of new tech, framework, programming language, and patterns every day the web industry is more confusing than ever, making it very hard for a newbie get their feet wet and get the taste of how it feels to get going or start development in the web dev industry. Every time a new tech is introduced everyone goes on to throwing benchmark results on how the new stuff is better than the old one and how it can replace a huge chunk and make the development better.

I as a developer often get overwhelmed and fatigued by this continuous punch of information and knowledge thrown at my face. We, instead of diving deep in the use cases of a particular tech start comparing it with all the other tech that is around it. Deno is one of the same "runtime" that falls into the category.

"What is Deno?", you may ask. So "Deno is a simple, modern, and secure runtime for JavaScript and TypeScript that uses V8 and is built in Rust". It has been under development for almost 2 years and I have been following for less than 6 months and this is what I have found out.

Deno draws a lot of its resemblance from "Node". Just try to rearrange the letters in "N O D E" and you will get a gist of what I am talking about. Deno is the brainchild of Ryan Dahl the creator of NodeJS who left the NodeJS Foundation in January 2012 after creating and contributing to the open-source Javascript runtime for almost 4 years or more. He made JS run on a server possible and he is the sole reason why starting up as a web developer in today's times is a lot easier because a lot of people pick up NodeJS first to learn web development as it is a lot easier to get up and running with it. Node did good and is doing extremely well in the current times but there were a lot of problems with it. Since the development around Node was so fast that with the passing years a lot of code was added to it and some very bad decisions were made around it. As people were adopting the runtime faster than ever before, it became a lot harder for the devs to manage it or fix a bug as devs were relying on them, so they had to patch it instead of rewriting the whole feature which resulted in Node become a Big Ball of Mud.

The devs were under great technical debt and it was impossible for them to undo the changes and the decisions that were made in haste, so everyone decided to keep going forward. Ryan Dahl, on the other hand after leaving the foundation and working in a Google Scholar program wanted to change that. So he laid down the idea of Deno that targets and tries to remove all the shortcomings of NodeJS that it faces as a Javascript runtime some of which are

No native support for promises.
Lack of security.
A poorly designed module system.

A lot of learning has been taken from NodeJS since Node is out in the wild for almost 11 years. Some of the great features of Deno that tackles the problem that Node has are:

It has no default access to the file system, your environment variables, and your network so better security. Everything is turned off by default and is not granted access until you specifically provide the access.
Inspired by GoLang(Another great systems language turned server-side tech) it uses a URL path to resolve the modules. So instead of a module being hosted on a registry on a single platform, the dev can host the dependency anywhere they like and can import them using the URL just like we do in browsers. There is no central repository for it and since Deno always caches the dependency it provides an extra layer of robustness in case the platform hosting the dependency is down. Deno uses typescript as the default language.
Typescript has been a boon for JS devs out there. It allows a dev to write bug-free type-safe code by restricting the developer to follow certain rules and guidelines so they can be sure that the code they ship is going to work all time.
The underlying implementation is written in Rust(the equivalent of C++ for Node) and the event loop is Tokio(Rust). Since the development in the Rust community is far active than ever before and as the language claims that it helps developers write memory-safe code without letting them worry much about the memory manager management and thread-safety issues like that in C++, it holds a lot of promises for Deno. Deno's first version was released on 13th May 2020 and the repo has already gained 55k+ stars on Github and a lot of love and support from the community. But Deno is not here to dethrone NodeJS. Node has already been adopted by a large number of organizations and has set its root deep in the web dev industry it would be very very hard for Deno to dethrone Node. If used with caution Node is a powerful runtime that enables a developer to do any task that its counterparts do and it is performant too.

The open-source community has opened a great realm of competition that has made the coexistence of a great variety of tech solving a variety of problem possible in different ways. It is up to you to decide what path you want to take. Since every community is trying to get better than the other it keeps them on toes to make the product even better which always enhances the product. Hence it is a win-win situation for us developers.

All you need to do is instead of going blindly behind a tech decide that does it solve the problem that you want to solve in an efficient way and will it be feasible to put out the cost of learning and implementing the tech as a use case in your scenario possible.

In conclusion, I would like to say that do not get overwhelmed by the knowledge coming at you. Change is constant and all you need to do is adapt according to the situation.

Yearn, Learn, and Earn.

I would highly encourage you to take a look at the references that I have referred to make this article possible.

If you’re excited by this and want to work on cutting edge technology that touches millions of users daily, drop us a hello at careers@gamezop.com. We are hiring for both backend (Golang / Elixir / Node) and frontend (React / Node) roles.

Part 2. Managing multiple stores using GenServer processes — Elixir/OTP

Arpit Dubey — Tue, 25 Feb 2020 12:15:33 +0000

Part 2. Managing multiple stores using GenServer processes — Elixir/OTP

Hey, have you stumbled upon this post directly without reading the previous article because you searched for something and Google decided to teleport you here? Well, then I would suggest you read the previous article which is the first part of the 7 part series that I am writing.

Now, if you have read the previous article then let’s begin directly with this article.

So previously we created a server process that stores some values in a key-value store. Now what we will aim for is to create a server process with a Manager process which will help us manage multiple key-value stores.

With the introduction of a manager process, we will be able to create multiple stores and store key-values in them separately because guess what, we are not learning Elixir to just to deal with a single process.

Let’s get started 🏎️

Before beginning make sure you have Elixir and Mix installed in your system.

Now create a mix project by typing

mix new ex2_key_val_with_multiple_stores

This command should create your project structure like this

Delete all the files in the lib folder and create a file with name manager.ex and store.ex. Also copy the previously created server.ex in the same folder.

The reason why we are creating these many modules is that it is always good to decouple your code in the form of modules and functions wherever possible. It makes testing a lot easier and makes parts of your app to plug and play anywhere you want.

With the introduction of a manager process to manager multiple instances of stores we are looking to achieve an architecture like in the image below.

init/1 accepts one argument. This is the second argument provided to GenServer.start/2, and you can use it to pass data to the server process while starting it.
The result of init/1 must be in the format {:ok, initial_state}.
handle_cast/2 accepts the request and the state and should return the result in the format {:noreply, new_state}.
handle_call/3 takes the request, the caller information, and the state. It should return the result in the format {:reply, response, new_state}.

Show me some code 👨‍💻

Here we now have 3 entities to care about.

Manager
Server
Store

The Manager process is a GenServer whose task would be to create a store as a server process and maintain a map of stores created with their name and PID which the client would use to communicate.

In the Manager file we startup a GenServer and this time we pass an extra parameter of name: MODULE to it. This registers the GenServer’s name as it’s module name so we won’t be needing a PID to communicate to it. Since we are going to create only one Manager there would only be one Manager with that name.

The internal state of our Manager will store all the store related information in the form of Map with keys being the name of the store and values being their respective PIDs.

Whenever a command to create a store is issued to the Manager it first checks if the store with the same name is already present in its state or not. If yes then it simply returns the PID else it starts up a new server process and stores the name and PID of that server process in its an internal state and then returns the PID of the newly created server process.

The Server is the simple GenServer that handle the task to manipulate the store values when the client issues a specific operation to it to perform.
The server code is the** same as before. It handles all the operations like **put, get and del but in a more abstracted way with the help of a Store module.

To promote code reuse we took out the Store operations in a different module called KeyVal.Store. This makes the app simple.

The final run 📟

In order to start the whole system, we first initiate an elixir session
by typing

iex -S mix

Now we start the Manager process with the startup command. After starting the manager process we create two stores with different names and store their PIDs in local variables.

Then we communicate to the servers with their respective PIDs to store, retrieve or delete values in the store. Since both the server processes are different the values stored in them are completely different. You won’t be able to query for a value stored in Server 1 using the PID od Server 2 since they do not share the stores.

So this is how you can Manager multiple stores using GenServers.

I hope this post has helped you get a little bit better understanding of the whole process.

The complete source code of all the parts are here.

References 📝

Part 1. GenServer — Elixir/OTP

Arpit Dubey — Mon, 24 Feb 2020 19:20:35 +0000

Part 1. GenServer — Elixir/OTP

With the beginning of 2020 Elixir has been gaining a lot of popularity in the field of functional programming language. Being a software engineer who always looks out to try something new I instantly bought the idea of learning the language when it was introduced at my workplace by a fellow game developer.

As I started learning the language I found it a little difficult as I never wrote a single line in a functional way using Elixir. It promotes a coding style that helps developers write code that is short, concise, and maintainable which was pretty new to me.

As I dived into the language further I was introduced to the concepts of scalability, concurrency and fault tolerance. Everything about the language seems perfect the tooling to write, test and maintain code is smooth. Also when companies like Discord and Square-Enix are using it to handle large user base in real-time it gives a lot of credibility to the language.

The area where elixir shines is handling scalability and fault tolerance. As it can be pretty daunting to wrap your head around the concepts I decided to document every task that I did to get hold of the concepts in the easiest possible manner. This post and the posts to come deal with the concepts of writing concurrent and fault-tolerant code using Erlang’s OTP(Open Telecom Platform) behaviours, so if you are new to Elixir you might want to skip this post as it might be a little challenging to get directly into the intermediate parts of the language.

I am not going to dive deep into what Elixir is and how it runs on a system as this post is more about using OTP behaviours in Elixir to achieve concurrency and fault tolerance.

What to expect from this article 🤔

This will be a 7 part article where we will be learning Elixir concepts by building and continuously improvising a key-value store application. The application would let you simply create, update and delete a value in a store.

Part 1. Creating a key-value store using GenServer.
Part 2. Managing multiple stores using GenServer processes.
Part 3. Persisting data in key-value store using GenServer.
Part 4. Making operations on the key-value store faster by using worker processes.
Part 5. Adding Supervisors to the key-value store.
Part 6. Managing multiple Supervisors and a supervision tree for the key-value store.
Part 7. Writing your first Webserver to expose the key-value store to the web.

Let’s get started 🏎️

Before beginning make sure you have Elixir and Mix installed in your system.

Now create a mix project by typing

mix new ex1_a_gen_server

This command should create your project structure like this

delete all the files in the lib folder and create a file with name server.ex

This is the file where we are going to write all of our code.
Before starting to write the code let’s start with what are we going to build and how are we going to implement it using a GenServer.

The key-value store that we are going to build would be a simple process that should do the following three tasks.

Put a value against a key in the store.
Retrieve a value against a key in the store.
Delete a value against a key in the store.

Now let’s discuss what is a Genserver.

A Genserver is behaviour module for implementing the server of a client-server relation.

It is a process like any other Elixir process and it can be used to keep state, execute code asynchronously and so on. The advantage of using a generic server process (GenServer) implemented using this module is that it will have a standard set of interface functions and include functionality for tracing and error reporting. It will also fit into a supervision tree(We will discuss this later in the coming posts).

So we are going to maintain our state i.e. the key-value store in the form of a map in a GenServer process and we are going to talk to the process in an async way by message passing.

In order to make we create out key-value store app we need to implement these three callbacks in our server. init/1, handle_cast/2, and handle_call/3.

init/1 accepts one argument. This is the second argument provided to GenServer.start/2, and you can use it to pass data to the server process while starting it.
The result of init/1 must be in the format {:ok, initial_state}.
handle_cast/2 accepts the request and the state and should return the result in the format {:noreply, new_state}.
handle_call/3 takes the request, the caller information, and the state. It should return the result in the format {:reply, response, new_state}.

Show me some code 👨‍💻

Ok, so here we start our GenServer by calling GenServer.start/2. The first argument is the module itself while the second argument can be any arbitrary argument that you want to pass to the server while it’s initialization.

The GenServer.start/2 will call the init/1 function which would accept the argument passed to it while starting the server. It eventually returns {:ok, %{}} which is our empty initial state.
The put, get and del functions are implemented to accept a PID and some relevant information and pass the information to the GenServer process running on the PID.
After receiving the info depending on whether is a cast or a call request the server call the appropriate handler.
Since the state is not mutable every time when the server state is changed a new state is returned from the handle call or cast functions to the update the server state.
As the function names suggest Map.fetch, Map.put and Map.delete do what their function name describes.
The reason why we are using handle_call for the get request is because it is the request where we care about the result returned to us. So we wait for the process to give us back the result.

The final run 📟

On starting the server we store the PID in a variable and then all the communication is happening using the PID of the process.

You can use the exposed put, get and del functions on the Server module to perform operations on the key-value store all you need to remember is the PID of the store process that you started.

This is the basic implementation of a GenServer in Elixir. I hope this post has helped you get a little bit better understanding of the whole process.

The complete source code of all the parts are here.