Elxpro

Posted on Jun 24, 2022 • Updated on May 4, 2023

First Steps to Using Distributed Systems in Elixir

#elixir #genserver #phoenix #beam

Greeting

Hello #devElixir!!! Welcome to #FullStackElxpro

Here we discuss strategies and tips for your Elixir learning journey. This is from scratch until your first wave as an elixir developer

Do you want to learn Elixir in three months? https://elxpro.com/sell

I am Gustavo and today's theme is: Distributed Systems in Elixir

_ps: You can follow the article with video

Which are Distributed Systems?

A distributed system is a collection of computer programs that utilize computing resources at several different central computing points to achieve a shared goal. Distributed systems centralized bottlenecks or central points of failure of a system.

In this case, in the language, we chose (Elixir). We will use the resources of NODE

https://hexdocs.pm/elixir/1.12/Node.html

Why is it so important to understand?

What is the principle of programming in Elixir? And what are the advantages?

Everyone who chooses to work with Elixir is always the same subject:

Functional Language
competition
Parallelism

The important thing is what I see that could be a trend in the future in advanced Elixir developers and use mainly of concurrency and parallelism in distributed systems to decrease the cost in a company.

And how does Elixir help us with distribution?

A list of frameworks is below, but before using these frameworks, you need to understand a very important concept.

MNESIA - https://elixirschool.com/en/lessons/storage/mnesia
PUBSUB - https://hexdocs.pm/phoenix_pubsub/Phoenix.PubSub.html
LibCluster - https://github.com/bitwalker/libcluster

And reading this article: https://dashbit.co/blog/you-may-not-need-redis-with-elixir

I could talk about MNESIA, Pubsub, and Libcluster, but before you understand and before you start distributing systems with Elixir you need to understand what's behind all these frameworks.

The why and in case you still don't understand the distribution of systems It is low cost and uses performance of computer systems and hardware of a fantastic language that is not mainly and Elixir

How did I come to that conclusion?

I arrived at this final computation, this solver different one Feature had three applications running (in Kubernetes) I needed to read a file and send this to an external service, and an external service that saves a file to the database when the service was running on staging, sometimes these files were not read, so we tried to understand what the data was doing sometimes.

Our problem is that we had 3nodes running and the file was sometimes sent to the batch at the time of background processing, and when trying to read the file it was not healthy.

The simplest solution for this case would be to save this data in Storage (S3). When we talk about a functional language and a powerful language that facilitates concurrency, parallelism and distribution, 3 minutes to read files is a lot of time and that's where the idea for the next version came from using distributed systems and few resources. These resources that require a lot of knowledge of how Elixir works behind the scenes that I've come to complete and I want to share my experience with you and how you can make decisions after this article.

Getting Started with Elixir Distribution

This article is an example of how to use Nodes and RPC. Our first step is to know how to creators and RPC (remote procedure call).

Let's start with IEX

iex --sname gus #this way you create a shortname

iex --name gus@ip #this way you create a name

the biggest difference is access between the internet and local.

Let's learn some of the commands above:

Node.ping: app_name

You will do a ping checking if it is possible to connect with the APP

Node.connect: app_name

You will connect to a Node.

list of nodes

you will see the list of connections you have. What's interesting is that in this case when you're going to connect the nodes, they create communication between all of them. as you see in the Last Node.list

Calling functions.

Before we start talking about processes, something that Elixir gives us for free is a :rpc. something that is very complex in other applications.

Let's see the scenario below:

In the example above we have the RPC that follows the following pattern to be called:

:rpc.call app_name, MODULE, :function, [parameters]

And look at the module and when it exists, the stop for the interesting node that is always not there.

Using processes in Calls.

You can create a project in Elixir using

mix new product_stock

defmodule ProductStock do
  def handle_stock(stock) do
    receive do
      {:add, product} ->
        stock_updated = [product] ++ stock
        handle_stock(stock_updated)
      :status ->
        IO.inspect stock
        handle_stock(stock)
    end
  end
end

Let's start with the start of the application:

❯ iex --sname product_stock -S mix

in this example we start a normal application in elixir but with a short-name

a process has been created and registered with a name.

> pid = generate ProductStock, :handle_stock, [[]]
> Process.register pid, :product_stock

ps: The article below will help you understand more about spawn, send and receive

https://www.youtube.com/watch?v=em4QECkQx4s&t=801s

And now we start sending data to update our stock, but in the same module, we started the application.

> send :product_stock, :status
> send :product_stock, {:add, %{name: "nike", qty: 30}}
> send :product_stock, :status
> send :product_stock, {:add, %{name: "adidas", qty: 35}}

What is interesting to note is what is done after connecting to the server application. And how are we going to update the stock. You will notice that to call a process in another app you need to include in your send the following structure.

 send {:pid_name, :app_name}, message

And when calling N times the service always goes to the app server as shown in the image above.

Handling Cross-Process Responses

defmodule ProductStock do
    def handle_stock(stock) do
      receive do
        {:add, from, product} ->
          stock_updated = [product] ++ stock
          send(from, stock_updated)
          handle_stock(stock_updated)

        {:status, from} ->
          send(from, stock)
          handle_stock(stock)
      end
    end
end

In the example above, we either update our code to always return the response to some process whatever. We use functions like Process.info(PID, :messages) to know if you have messages and flush to read as messages.

About communication of Nodes and Processes

defmodule ProductStock do
  def handle_stock(stock) do
    receive do
      {:add, from, product} ->
        stock_updated = [product] ++ stock
        log(from, :add)
        send(from, stock_updated)
        handle_stock(stock_updated)

      {:status, from} ->
        log(from, :status)
        send(from, {:show, stock})
        handle_stock(stock)
    end
  end

  defp log(pid, :add) do
    IO.inspect("#{:erlang.pid_to_list(pid)} added a new item on stock")
  end

  defp log(pid, :status) do
    IO.inspect("#{:erlang.pid_to_list(pid)} checked the stock")
  end
end

In the image, then we apply what we learned above from this entire article. What has changed is the communication between the nodes and processes now whenever a PID calls a pid you can see that it has a number like: <19214.112.> which translates to: .

Let's share a little bit about distribution.

Imagine that you have a central to stock the products of several stores, and every time a customer requests a product in the store, the stores check if the central has the product, and return the message to the customer. Follow the example below.

defmodule ProductStock do
  def handle_stock(stock) do
    receive do
      {:add, from, product} ->
        stock_updated = [product] ++ stock
        log(from, :add)
        send(from, stock_updated)
        handle_stock(stock_updated)

      {:status, from} ->
        log(from, :status)
        send(from, stock)
        handle_stock(stock)
    end
  end

  defp log(pid, :add) do
    IO.inspect("#{:erlang.pid_to_list(pid)} added a new item on stock")
  end

  defp log(pid, :status) do
    IO.inspect("#{:erlang.pid_to_list(pid)} checked the stock")
  end
end

defmodule Pos do
  def ask() do
    receive do
      {:ask, from, product} ->
        stock_address = :"server@Gustavos-MacBook-Pro"
        stock_info = {:server, stock_address}
        IO.inspect("#{:erlang.pid_to_list(from)} wants to know if get a product")
        send(stock_info, {:status, self()})
        return_itens(from, product)
    end

    ask()
  end

  def return_itens(from, product) do
    receive do
      products ->
        case Enum.find(products, &(&1.name == product)) do
          nil -> send(from, {:print, "There is no Product #{product}"})
          product -> send(from, {:print, product})
        end
    end
  end

  def show_msg do
    receive do
      {:print, msg} -> msg
    end
  end
end