DEV Community: Santiago Cardona

Elixir background processes synchronization with GenServer and DynamicSupervisor

Santiago Cardona — Fri, 16 Jul 2021 14:50:56 +0000

In this post we are gonna see how to handle background processes synchronization using powerful Elixir tools such as GenServer and DynamicSupervisor. Using these tools we can see real concurrent capabilities of Elixir.

For the example application, we are gonna sync GitHub Mentions for a specific user in a given GitHub repository. This sync should be made every X period of time to make sure we have Mentions up to date. We will be using GitHub's REST API to fetch all Pull Request comments on the given repository, and then filter these comments by Mentions for the given user.

We have to make sure we have Mentions up to date, that's why we are gonna use a GenServer to implement the background synchronization. Also, we need to spawn a new GenServer every time a new client wants to sync mentions, that's why we are gonna use DynamicSupervisor to spawn a new GenServer for each client.

Finally, in this example, we are gonna use Phoenix LiveView as the client that's going to request Mentions, display those Mentions, and receive updates when new Mentions are found. Since Phoenix also uses DynamicSupervisor to spawn a process every time a client requests a route (LiveView process in this case), many users can sync Mentions at the same time.

Let's go to the code...

Implementing the Mentions GenServer

defmodule GithubPrMentions.Mentions do
  use GenServer

  ...

  def child_spec(lv_pid) do
    %{
      id: {__MODULE__, lv_pid},
      start: {__MODULE__, :start_link, [lv_pid]},
      restart: :temporary
    }
  end

  def start_link(lv_pid) do
    GenServer.start_link(
      __MODULE__,
      lv_pid,
      name: via(lv_pid)
    )
  end

  def via(lv_pid) do
    {:via, Registry, {GithubPrMentions.Registry.Mentions, lv_pid}}
  end

  def init(_opts) do
    state = %{
      id: nil,
      interval: :timer.seconds(60),
      timer: nil,
      prs_curr_page: %{},
      base_url: nil,
      username: nil,
      token: nil
    }

    {:ok, state}
  end

  ...
end

Here lays the magic behind starting the GenServer. We need a child_spec/1 callback function to specify an identifier of the child process (the GenServer), the starting function, and the restarting policy. This function is going to be executed every time our DynamicSupervisor spawns a new GenServer.

We need also a start_link/1 callback to start the new GenServer linked to its parent supervisor (the DynamicSupervisor, in our case). This function is going to be executed by child_spec/1 passing it the arguments provided by this function as well. There, we name the GenServer via a Registry to have each GenServer process registered and identified by a LiveView PID, for later calling purposes. This is a handy way for identifying a GenServer process without having to remember its PID.

Then, to start syncing Mentions we need to dynamically create a new GenServer that will do the job. Each GenServer is going to be spawn every time the public API function get_mentions/4 is called because that function calls the DynamicSupervisor.start_child/2 function, creating a new GenServer process for the given LiveView PID:

def get_mentions(repo_url, username, token, lv_pid) do
    DynamicSupervisor.start_child(
      GithubPrMentions.Supervisor.Mentions,
      {__MODULE__, lv_pid}
    )

    GenServer.cast(via(lv_pid), {:set_initial_state, repo_url, username, token, lv_pid})
    GenServer.cast(via(lv_pid), {:fetch_pulls, 1})
end

That's all we need to start spawning GenServers dynamically. For the rest of the logic of fetching Pull Request comments, you can see the repo GitHub PR Mentions. But there are 2 important GenServer function callbacks and 1 private function that are a key part of the background processes synchronization: handle_cast({:set_initial_state, repo_url, username, token, lv_pid}, state), handle_info({:DOWN, ref, :process, pid, reason}, state) and schedule_refresh(state).

def handle_cast({:set_initial_state, repo_url, username, token, lv_pid}, state) do
    Process.monitor(lv_pid)

    initial_state = %{
      base_url: GitHub.get_base_url(repo_url),
      username: username,
      token: token,
      id: lv_pid
    }

    state = Map.merge(state, initial_state)

    {:noreply, schedule_refresh(state)}
end

This handle_cast callback perform actions that involves the other 2 functions. The first one starts a process monitor for the current LiveView PID (lv_pid). This process monitoring is going to invoke a handle_info callback when the current associated LiveView PID has died:

def handle_info({:DOWN, _ref, :process, _pid, _reason}, state) do
    {:stop, :normal, state}
end

With the help of this notification, we can then stop the current GenServer spawned specifically for the LiveView that has just died. That way, we make sure that we kill background processes that are no longer required.

The second key action is the one that invokes schedule_refresh(state), making sure that every 60 seconds we refresh mentions:

defp schedule_refresh(state) do
    pid = GenServer.whereis(via(state.id))

    %{state | timer: Process.send_after(pid, :refresh_mentions, state.interval)}
end

Finally, we also need a Registry, a DynamicSupervisor, and the Mentions GenServer in the list of the main app's Supervisor (supervision tree), to be treated as children and be securely restarted when an eventual crash happens:

defmodule GithubPrMentions.Application do
  ...
  def start(_type, _args) do
    children = [
      ...
      {Registry, [name: GithubPrMentions.Registry.Mentions, keys: :unique]},
      {DynamicSupervisor, [name: GithubPrMentions.Supervisor.Mentions, strategy: :one_for_one]},
      GithubPrMentions.Mentions
    ]

    ...
  end
  ...
end

Implementing the LiveView Client

For the LiveView client is just as simple as calling Mentions.get_mentions/4 when the LiveView is going to be mounted to start the background sync.

Once the background sync for Mentions is started, it's going to eventually send messages to its specific LiveView process. Having this in mind, we need to add a handle_info callback which is going to receive mentions that the GenServer sends and start displaying them.

defmodule GithubPrMentionsWeb.MentionsLive do
  use GithubPrMentionsWeb, :live_view

  alias GithubPrMentions.Mentions

  @impl true
  def mount(%{"repo" => repo, "username" => username}, %{"current_user" => current_user}, socket) do
    Mentions.get_mentions(repo, username, current_user.access_token, self())

    {:ok,
     socket
     |> assign(prs: [], username: username)
     |> put_flash(:info, "Searching mentions..."), temporary_assigns: [prs: []]}
  end

  @impl true
  def handle_info({:new_mentions, pr_number, mentions}, socket) do
    send_update(GithubPrMentionsWeb.ShowMentions, id: pr_number, mentions: mentions)

    {:noreply,
     socket
     |> assign(:prs, [pr_number])
     |> put_flash(:info, "Looking for more mentions...")}
  end
end

For the logic of displaying those Mentions, you can see the rest of the implementation in the repo GitHub PR Mentions, or I even recommend you to take a look at the Phoenix LiveView Comment and Reply article to have a better understanding of how this works (btw, it's a fantastic demonstration of LiveView's capabilities).

Happy Coding!

To see the full implementation you can visit the repo:

santiagocardo / github-pr-mentions

Get all mentions of a user in a given repository

GithubPrMentions

To start your Phoenix server:

Install dependencies with mix deps.get
Install Node.js dependencies with npm install inside the assets directory
Start Phoenix endpoint with mix phx.server

Now you can visit localhost:4000 from your browser.

Ready to run in production? Please check our deployment guides.

Learn more

Official website: https://www.phoenixframework.org/
Guides: https://hexdocs.pm/phoenix/overview.html
Docs: https://hexdocs.pm/phoenix
Forum: https://elixirforum.com/c/phoenix-forum
Source: https://github.com/phoenixframework/phoenix

View on GitHub

Autocomplete input + caching results with Phoenix LiveView

Santiago Cardona — Mon, 07 Jun 2021 18:41:52 +0000

In this post, I want to share with you all the powerful machinery I've used to implement a geolocation search input with autocomplete using Here Maps API. With this approach, I've also implemented a browser's geolocation to improve precision, as well as results caching for better performance and cost saving when hitting geolocation third-party backends.

First thing first, I have to say that for this post I took a huge amount of inspiration and code from the fantastic book Programming Phoenix 1.4 by Chris McCord, Bruce Tate, and José Valim. Especially, the cache and API request parts. So if you want to have a deeper explanation don't doubt to read it from the source.

Setting Up the Search Form View

<form phx-change="suggest" phx-submit="search">

      <input
        type="text"
        name="origin"
        value="<%= @origin %>"
        placeholder="Origen"
        list="origins"
        autocomplete="off"
        phx-debounce="700"
      />
      <datalist id="origins">
        <%= for place <- @origins do %>
          <option value="<%= place.title %>"><%= place.title %></option>
        <% end %>
      </datalist>

      <input
        type="text"
        name="destination"
        value="<%= @destination %>"
        placeholder="Destino"
        list="destinations"
        autocomplete="off"
        phx-debounce="700"
      />
      <datalist id="destinations">
        <%= for place <- @destinations do %>
          <option value="<%= place.title %>"><%= place.title %></option>
        <% end %>
      </datalist>

      <%= live_component @socket, CarpoolingWeb.PositionComponent, id: "position" %>

      <button type="submit" phx-disable-with="Buscando...">Buscar</button>

</form>

This form uses a LiveView Component to get the browser's position and using that position (latitude and longitude) as part of the params we sent from the form:

defmodule CarpoolingWeb.PositionComponent do
  use CarpoolingWeb, :live_component

  def mount(socket) do
    {:ok, socket}
  end

  def render(assigns) do
    ~L"""
    <input
      id="current-position"
      name="position"
      type="hidden"
      phx-update="ignore"
      phx-hook="SetCurrentPosition"
    >
    """
  end
end

In this PositionComponent we use a phx-hook to have JavaScript interoperability by accessing the navigator geolocation API and get the current geolocation's position from the user's browser. Also, we use phx-update since we don't want to trigger the phx-hook every time the form changes any of its inputs.

We set up the hook in app.js, in this case, the one called SetCurrentPosition and then we add it as part of the liveSocket:

const setPosition = ({ coords }) => {
  const x = document.getElementById("current-position")
  x.value = coords.latitude + "," + coords.longitude
}

const Hooks = {
  SetCurrentPosition: {
    mounted() {
      if (navigator.geolocation) {
        navigator.geolocation.getCurrentPosition(setPosition)
      }
    }
  }
}

const csrfToken = document.querySelector("meta[name='csrf-token']").getAttribute("content")
const liveSocket = new LiveSocket("/live", Socket, {
  hooks: Hooks,
  params: { _csrf_token: csrfToken }
})

Setting Up the LiveView

defmodule CarpoolingWeb.RideLive.Index do
  use CarpoolingWeb, :live_view

  alias Carpooling.Locations

  @impl true
  def mount(_params, _session, socket) do
    {:ok,
     assign(socket,
       origin: "",
       origins: [],
       destination: "",
       destinations: [],
     )}
  end

  @impl true
  def handle_params(params, _url, socket) do
    {:noreply, apply_action(socket, socket.assigns.live_action, params)}
  end

  @impl true
  def handle_event("suggest", params, socket) do
    %{
      "origin" => origin,
      "destination" => destination,
      "position" => position
    } = params

    origins =
      Locations.get_locations(origin, position)

    destinations =
      Locations.get_locations(destination, position)

    {:noreply,
     assign(socket,
       origins: origins,
       origin: origin,
       destinations: destinations,
       destination: destination,
     )}
  end

  defp apply_action(socket, :index, _params) do
    socket
    |> assign(:page_title, "Transporte Solidario")
  end
end

In the mount callback we initialize origins and destinations fields as empty lists to render the form view and hoping to fill those every time the user types in the location text inputs.

We have a handle_event callback that listens for a "suggest" event to be triggered every time the form changes (this is triggered by the form's phx-change binding). Here we use Locations.get_locations/2 function in order to obtain the origins and destinations lists, and then update those lists in the socket fields.

At this point, we can see how Phoenix LiveView manages to handle the client logic very smoothly 😍. Just by using a couple of LiveView Bindings we got autocomplete inputs ready.

Implementing the Locations Module (Locations Context: Geolocation API Requests + Caching Results)

Here is where the magic occurs. The Locations Context is the one responsible for doing the HTTP requests to get the list of possible locations that matches with the user's input, but also responsible for caching these possible results.

defmodule Carpooling.Locations do
  @backends [Carpooling.Locations.HereMaps]

  defmodule Result do
    defstruct backend: nil, locations: nil
  end

  alias Carpooling.Locations.Cache

  def get_locations(query, point), do: search(query, point)

  defp search(query, point) do
    if String.length(query) >= 5 do
      compute(query, point, [])
      |> Enum.map(fn item -> item.locations end)
      |> List.flatten()
    else
      []
    end
  end

  defp compute(query, point, opts \\ []) do
    timeout = opts[:timeout] || 10_000
    opts = Keyword.put_new(opts, :limit, 10)
    backends = opts[:backends] || @backends

    {uncached_backends, cached_results} = fetch_cached_results(backends, query, opts)

    uncached_backends
    |> Enum.map(&async_query(&1, query, point, opts))
    |> Task.yield_many(timeout)
    |> Enum.map(fn {task, res} ->
      res || Task.shutdown(task, :brutal_kill)
    end)
    |> Enum.flat_map(fn
      {:ok, results} -> results
      _ -> []
    end)
    |> write_results_to_cache(query, opts)
    |> Kernel.++(cached_results)
    |> Enum.take(opts[:limit])
  end

  defp fetch_cached_results(backends, query, opts) do
    {uncached_backends, results} =
      Enum.reduce(
        backends,
        {[], []},
        fn backend, {uncached_backends, acc_results} ->
          case Cache.fetch({backend.name(), query, opts[:limit]}) do
            {:ok, results} ->
              {uncached_backends, [results | acc_results]}

            :error ->
              {[backend | uncached_backends], acc_results}
          end
        end
      )

    {uncached_backends, List.flatten(results)}
  end

  defp async_query(backend, query, point, opts) do
    Task.Supervisor.async_nolink(
      Carpooling.TaskSupervisor,
      backend,
      :compute,
      [query, point, opts],
      shutdown: :brutal_kill
    )
  end

  defp write_results_to_cache(results, query, opts) do
    Enum.map(results, fn %Result{backend: backend} = result ->
      :ok = Cache.put({backend.name(), query, opts[:limit]}, result)

      result
    end)
  end
end

In this case, I'm using Here Maps API to get possible locations for the autocomplete's suggestions. But we can integrate more APIs such as Google Maps just by implementing a particular module for each API + results parsing and including them as part of the @backends list. Then, we include a Result module to identify each different backend's results in case we need it someway in the future.

We have four important functions here: compute, fetch_cached_results, async_query and write_results_to_cache. Its names are pretty declarative in what each one of them do.

The compute function is the one that invokes the others, but also the one where we set up different configurations such as results limitation, requests timeouts, and so on. This function is going to process cached and uncached results to concatenate them and return them as results.

The fetch_cached_results function is going to loop over each geolocation backend and get cached results from the Cache module (cached_results) for both, the given query and the current backend module. In case that the current backend module doesn't have cached results for that specific query, is going to be pushed in the uncached_backends list.

The async_query function is the one in charge to get results from geolocation backends by calling its corresponding compute callback (each geolocation backend module must have this callback, we are going to cover it later). In this case, we use an Elixir's Task to perform the request, but more importantly, use the async_nolink function from this module because we want this request to be asynchronous but we don't want it to let the process crash by an HTTP request error, we just simply kill that specific task that errored. Since we are constantly typing in the geolocation search input which fires Locations.get_locations/2 function execution, and also we may have different backend integrations, we don't mind if a single HTTP request crashes, we just kill it because we don't want a whole restarting of the current process caused by this specific crash.

Lastly, the async_query function is the one in charge to put uncached results in the cache. Here we loop over each backend's results, set those results in the cache, and then return the same results. This is a function that simply returns the same that receives, but in the middle performs a side effect, making sure that each backend's result is going to be cached.

Implementing the Backend Module (Backend's Behaviour)

As I mentioned above, we can integrate as many geolocation backends as we wish. But to use them correctly we can create a behavior that each one of them must follow. This module is going to have to callbacks: name and compute. The name/0 callback is going to return the backend's name. The compute/3 is going to return a list of results with the structure of the Result module.

defmodule Carpooling.Locations.Backend do
  @callback name() :: String.t()
  @callback compute(query :: String.t(), point :: String.t(), opts :: Keyword.t()) :: [%Carpooling.Locations.Result{}]
end

Implementing HereMaps Module

Since we are using HereMaps to get geolocations, we implement the module where HTTP requests are made, as well as making sure it follows the Backend's behavior, returning a list of results with the shape of the Result structure.

defmodule Carpooling.Locations.HereMaps do
  alias Carpooling.Locations.Result

  @behaviour Carpooling.Locations.Backend

  @base_discover "https://discover.search.hereapi.com/v1/discover"
  @base_geocode "https://geocode.search.hereapi.com/v1/geocode"

  @impl true
  def name, do: "here_maps"

  @impl true
  def compute(query_str, point, _opts) do
    fetch(query_str, point)
    |> build_results()
  end

  defp fetch(query, point) do
    url(query, point)
    |> HTTPoison.get()
    |> handle_response()
  end

  defp url(query, point) do
    query = URI.encode_query(q: query, apiKey: api_key()) <> "&in=countryCode:COL"

    if String.length(point) > 0 do
      "#{@base_discover}?#{query}&at=#{point}"
    else
      "#{@base_geocode}?#{query}"
    end
  end

  defp api_key, do: Application.fetch_env!(:carpooling, :here_maps)[:apikey]

  defp handle_response({:ok, %{status_code: status_code, body: body}}) do
    {
      status_code |> check_for_error(),
      body |> Poison.Parser.parse!()
    }
  end

  def check_for_error(200), do: :ok
  def check_for_error(_), do: :error

  defp build_results({:ok, %{"items" => items}}) do
    locations =
      items
      |> Enum.map(fn item ->
        %{
          address: item["address"],
          position: item["position"],
          title: item["title"]
        }
      end)

    [%Result{backend: __MODULE__, locations: locations}]
  end

  defp build_results(_response), do: []
end

Implementing the Cache Module

We almost finish but that doesn't mean it's less important. The Cache module plays a pretty important role in our solution. Here we can see some of the Elixir's beauties 😍: GenServer and ETS. This module is going to take advantage of these two fantastic built-in tools.

We create a GenServer to handle put and fetch calls to the ETS table which is the one in charge of persist geolocation results. This GenServer is also going to perform a background job that will clean this table every 600 seconds for this case.

defmodule Carpooling.Locations.Cache do
  use GenServer

  @clear_interval :timer.seconds(600)

  def put(name \\ __MODULE__, key, value) do
    true = :ets.insert(tab_name(name), {key, value})

    :ok
  end

  def fetch(name \\ __MODULE__, key) do
    {:ok, :ets.lookup_element(tab_name(name), key, 2)}
  rescue
    ArgumentError -> :error
  end

  def start_link(opts) do
    opts = Keyword.put_new(opts, :name, __MODULE__)
    GenServer.start_link(__MODULE__, opts, name: opts[:name])
  end

  def init(opts) do
    state = %{
      interval: opts[:clear_interval] || @clear_interval,
      timer: nil,
      table: new_table(opts[:name])
    }

    {:ok, schedule_clear(state)}
  end

  def handle_info(:clear, state) do
    :ets.delete_all_objects(state.table)
    {:noreply, schedule_clear(state)}
  end

  defp schedule_clear(state) do
    %{state | timer: Process.send_after(self(), :clear, state.interval)}
  end

  defp new_table(name) do
    name
    |> tab_name()
    |> :ets.new([
      :set,
      :named_table,
      :public,
      read_concurrency: true,
      write_concurrency: true
    ])
  end

  defp tab_name(name), do: :"#{name}_cache"
end

Once everything is settled, we only need to add the Cache Module and the Task Supervisor (for the async_nolink/5 call) to the App's supervision tree:

...

children = [
  Carpooling.Locations.Cache,
  {Task.Supervisor, name: Carpooling.TaskSupervisor},
  ...
]

...

That's it! Enjoy it!

I hope this can help you someway to build a robust autocomplete search mechanism while enjoying developing with Elixir 💜 and Phoenix LiveView 🧡

To see the full implementation you can visit the repo:

santiagocardo / carpooling

Carpooling app that connects workers and students for a collaborative economy

Carpooling

To start your Phoenix server:

Install dependencies with mix deps.get
Create and migrate your database with mix ecto.setup
Install Node.js dependencies with npm install inside the assets directory
Start Phoenix endpoint with mix phx.server

Now you can visit localhost:4000 from your browser.

Ready to run in production? Please check our deployment guides.

Learn more

Official website: https://www.phoenixframework.org/
Guides: https://hexdocs.pm/phoenix/overview.html
Docs: https://hexdocs.pm/phoenix
Forum: https://elixirforum.com/c/phoenix-forum
Source: https://github.com/phoenixframework/phoenix

View on GitHub

Conditional rendering of components with Phoenix LiveView

Santiago Cardona — Tue, 27 Apr 2021 15:34:18 +0000

When we find a view that has many HTML elements, we usually want to separate them into their components to have a better organization of the code that facilitates its understanding and maintenance, as well as decouple the large logic into smaller pieces, and why not, reuse these components in another view (very much in the style of React.js mental model). This can be easily achieved by using LiveComponent from Phoenix LiveView.

The problem is that many times these components share some logic between them, and many times they depend on each other to know if it should be rendered or not. This is called conditional rendering, and with Phoenix LiveView we can achieve this by handling the conditional logic in the parent LiveView, through the handle_info callback function.

Use Case

Suppose we have a view for registering a vehicle in a car workshop. This view is composed of two forms, one for the registration of the vehicle owner and one for the registration of the vehicle itself. Each form has its independent registration view, and so we use a LiveComponent for each of these, within which its business logic is handled independently.

These views must be rendered independently and in order:

User registration (customer_component.ex)
Vehicle registration (vehicle_component.ex)

Once the user is registered, the user's id must be passed to the vehicle component, so that we can successfully associate the user with the vehicle when we are registering the vehicle.

How do we achieve this? Let's go to the code...

The conditional logic is handled in the parent view of the user and vehicle components, in this case, it will be the LiveView vehicle_live/index.ex and its respective file vehicle_live/index.html.leex where the HTML elements to be rendered are located.

In the file vehicle_live/index.ex:



defmodule CarWorkshopWeb.VehicleLive.Index do
  use CarWorkshopWeb, :live_view

  alias CarWorkshop.{Vehicles.Vehicle, Accounts.Customer}

  @impl true
  def mount(_params, _session, socket) do
    {:ok, assign(socket, customer: %Customer{}, view_to_show: :customer_view)}
  end

  @impl true
  def handle_params(params, _url, socket) do
    {:noreply, apply_action(socket, socket.assigns.live_action, params)}
  end

  defp apply_action(socket, :new, _params) do
    socket
    |> assign(:page_title, "Register Vehicle")
    |> assign(:vehicle, %Vehicle{})
    |> assign(:customer, %Customer{})
  end

  @impl true
  def handle_info({:customer_registered, customer, view_to_show}, socket),
    do: {:noreply, assign(socket, customer: customer, view_to_show: view_to_show)}
end

In the mount function, we assign to the socket the property view_to_show, using this, we will know the component to render in vehicle_live/index.html.leex. We give it a value of :customer_view, making the first view to be rendered the one of the user component.

The callback function handle_info will be in charge of changing the view_to_show property of the socket, and thus, the conditional rendering of each component will be performed according to the arguments we pass to this function.

In the file vehicle_live/index.html.leex:



<%= if @view_to_show == :vehicle_view do %>
  <%= live_component @socket, CarWorkshopWeb.VehicleComponent,
    id: @vehicle.id || :new,
    title: "Register Vehicle",
    action: @live_action,
    vehicle: @vehicle,
    customer_id: @customer.id
  %>
<% else %>
  <%= live_component @socket, CarWorkshopWeb.CustomerComponent,
    id: @customer.id || :new,
    title: "Register Customer",
    action: @live_action,
    customer: @customer
  %>
<% end %>

With this, we already have the user view rendered. What we would need to do is call the handle_info callback from the user component once all our business logic has been executed, and then allow the rendering of the vehicle view.

Suppose we want to render the vehicle component immediately after the user has been successfully registered. For this, in the customer_component.ex file:



  ...
  @impl true
  def handle_event("save", %{"customer" => customer_params}, socket) do
    case Accounts.create_customer(customer_params) do
      {:ok, customer} ->
        send(self(), {:customer_registered, customer, :vehicle_view})

        {:noreply, socket}

      {:error, %Ecto.Changeset{} = changeset} ->
        {:noreply, assign(socket, changeset: changeset)}
    end
  end
  ...

Using the send() function we make the parent LiveView execute the handle_info callback that matches the {:customer_registered, customer, view_to_show} parameter contract. From which we will be able to know the identifier of the handle_info to execute (:customer_registered), the newly created user (customer), and the view to render (:vehicle_view).

That's it, we have a conditional rendering of components!

To see the full implementation you can visit the repo:

santiagocardo / car-workshop

Car Workshop Managment Web App

CarWorkshop

To start your Phoenix server:

Install dependencies with mix deps.get
Create and migrate your database with mix ecto.setup
Install Node.js dependencies with npm install inside the assets directory
Start Phoenix endpoint with mix phx.server

Now you can visit localhost:4000 from your browser.

Ready to run in production? Please check our deployment guides.

Learn more

Official website: https://www.phoenixframework.org/
Guides: https://hexdocs.pm/phoenix/overview.html
Docs: https://hexdocs.pm/phoenix
Forum: https://elixirforum.com/c/phoenix-forum
Source: https://github.com/phoenixframework/phoenix

View on GitHub

Renderizar componentes condicionalmente con Phoenix LiveView

Santiago Cardona — Tue, 27 Apr 2021 15:33:38 +0000

Cuando nos encontramos con una vista que tiene muchos elementos HTML, normalmente queremos separarlos en sus propios componentes para tener una mejor organización del código que facilite su comprensión y mantenimiento, así como desacoplar la lógica grande en piezas más pequeñas, y por qué no, reutilizar estos componentes en otra vista (muy al estilo del modelo mental de React.js). Esto se puede conseguir fácilmente utilizando LiveComponent de Phoenix LiveView.

El problema, es que muchas veces estos componentes comparten cierta lógica entre sí, y muchas veces dependen el uno del otro para saber si debe ser renderizado o no. A esto se le llama renderizado condicional, y con Phoenix LiveView podemos lograrlo manejando la lógica condicional en la LiveView padre, por medio de la función callback handle_info.

Caso de Uso

Supongamos que tenemos una vista para registrar un vehículo en un taller de mecánica. Esta vista se compone de dos formularios, uno para el registro del dueño del vehículo y otro para el registro del vehículo como tal. Cada formulario tiene su propia vista de registro independiente, y por ello usamos un LiveComponent por cada una de estos, dentro del cual se maneja su propia lógica de negocios independientemente.

Estas vistas deben ser renderizadas independientemente y en orden:

Registro del usuario (customer_component.ex)
Registro del vehículo (vehicle_component.ex)

Una vez registrado el usuario, se le debe pasar el id del usuario al componente de vehículo, y así poder asociar exitosamente el usuario con el vehículo cuando estamos registrando el vehículo.

Cómo logramos esto? Vamos al código...

La lógica condicional la manejamos en la vista padre de los componentes usuario y vehículo, en este caso será la LiveView vehicle_live/index.ex y su respectivo archivo vehicle_live/index.html.leex donde se encuentran los elementos HTML a renderizar.

En el archivo vehicle_live/index.ex:

defmodule CarWorkshopWeb.VehicleLive.Index do
  use CarWorkshopWeb, :live_view

  alias CarWorkshop.{Vehicles.Vehicle, Accounts.Customer}

  @impl true
  def mount(_params, _session, socket) do
    {:ok, assign(socket, customer: %Customer{}, view_to_show: :customer_view)}
  end

  @impl true
  def handle_params(params, _url, socket) do
    {:noreply, apply_action(socket, socket.assigns.live_action, params)}
  end

  defp apply_action(socket, :new, _params) do
    socket
    |> assign(:page_title, "Register Vehicle")
    |> assign(:vehicle, %Vehicle{})
    |> assign(:customer, %Customer{})
  end

  @impl true
  def handle_info({:customer_registered, customer, view_to_show}, socket),
    do: {:noreply, assign(socket, customer: customer, view_to_show: view_to_show)}
end

En la función mount, le asignamos al socket la propiedad view_to_show por medio de la cual sabremos el componente a renderizar en vehicle_live/index.html.leex. Le damos un valor de :customer_view, haciendo que la primera vista en ser renderizada sea la del componente de usuario.

La función callback handle_info será la encargada de cambiar la propiedad view_to_show del socket, y así, se realizará el renderizado condicional de cada componente según los argumentos que le pasemos a esta función.

En el archivo vehicle_live/index.html.leex:

<%= if @view_to_show == :vehicle_view do %>
  <%= live_component @socket, CarWorkshopWeb.VehicleComponent,
    id: @vehicle.id || :new,
    title: "Register Vehicle",
    action: @live_action,
    vehicle: @vehicle,
    customer_id: @customer.id
  %>
<% else %>
  <%= live_component @socket, CarWorkshopWeb.CustomerComponent,
    id: @customer.id || :new,
    title: "Register Customer",
    action: @live_action,
    customer: @customer
  %>
<% end %>

Con esto, ya tenemos renderizada la vista de usuario. Lo único que nos faltaría sería llamar al callback handle_info desde el componente de usuario una vez que se haya ejecutado toda nuestra lógica de negocio, y así, luego permitir el renderizado de la vista de vehículo.

Supongamos que queremos renderizar el componente de vehículo inmediatamente después que se registre el usuario exitosamente. Para esto, en el archivo customer_component.ex:

  ...
  @impl true
  def handle_event("save", %{"customer" => customer_params}, socket) do
    case Accounts.create_customer(customer_params) do
      {:ok, customer} ->
        send(self(), {:customer_registered, customer, :vehicle_view})

        {:noreply, socket}

      {:error, %Ecto.Changeset{} = changeset} ->
        {:noreply, assign(socket, changeset: changeset)}
    end
  end
  ...

Usando la función send() hacemos que en el LiveView padre se ejecute el callback handle_info que cumple con el contrato de parámetros {:customer_registered, customer, view_to_show}. Del cual podremos saber el indentificador del handle_info a ejecutar (:customer_registered), el usuario recién creado (customer) y la vista a renderizar (:vehicle_view).

Esto es todo, tenemos renderizado condicional de componentes!

Para ver la implementación completa puedes visitar el repo:

santiagocardo / car-workshop

Car Workshop Managment Web App

CarWorkshop

To start your Phoenix server:

Install dependencies with mix deps.get
Create and migrate your database with mix ecto.setup
Install Node.js dependencies with npm install inside the assets directory
Start Phoenix endpoint with mix phx.server

Now you can visit localhost:4000 from your browser.

Ready to run in production? Please check our deployment guides.

Learn more

Official website: https://www.phoenixframework.org/
Guides: https://hexdocs.pm/phoenix/overview.html
Docs: https://hexdocs.pm/phoenix
Forum: https://elixirforum.com/c/phoenix-forum
Source: https://github.com/phoenixframework/phoenix

View on GitHub

Upload files to Google Drive with Phoenix LiveView

Santiago Cardona — Wed, 21 Apr 2021 00:34:44 +0000

Phoenix LiveView has a great abstraction when it comes to uploading files. It makes our life much easier both from the client side (user interface) and from the server side (our server or even to third party services such as cloud services).

This post is focused on helping you with uploading files to the Google Drive service since many times Google services are difficult to understand and it takes time to figure out how to make successful integration, as was my case. That's why I would like to share with the community how I managed to do it after a long time of searching and testing.

You can also see this post in Spanish 🇨🇴

Cargar archivos a Google Drive con Phoenix LiveView

Santiago Cardona ・ Apr 19 '21

#elixir #phoenix #liveview #javascript

Let's start

This guide is not intended to show the detail of how the file upload process works in Phoenix LiveView. Rather, it is intended to show the integration of Google Drive with what the Phoenix LiveView External Uploads and Phoenix LiveView Uploads Deep Dive guides by Chris McCord already show in a super clear and easy to understand way.

For details on how the file upload process works in Phoenix LiveView, you can refer to the guides mentioned above.

Prerequisites

The first thing we must take into account is that we must enable access to the Google Drive API, this can be achieved by visiting the Google Drive API documentation.

Make sure you create a Google Cloud Service Account and finally have the .json file with your Google Cloud Service Account credentials. This file should contain something like this:

{
  "type": "service_account",
  "project_id": "<your google cloud project>",
  "private_key_id": "<your private key id>",
  "private_key": "<your private key>",
  "client_email": "<your client email>",
  "client_id": "<your client id>",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "<your client x509 cert url>"
}

Dependencies configuration

In order to authenticate with Google services, we will use the Goth library (Google + Auth).

Add Goth to your dependencies in the file mix.exs

  defp deps do
    [
      ...
      {:goth, "~> 1.2.0"}
    ]
  end

Run the following line in the console to download the dependencies:

$ mix deps.get

And finally, we must expose an environmental variable called GOOGLE_APPLICATION_CREDENTIALS in order to let Goth obtain an access token:

$ export GOOGLE_APPLICATION_CREDENTIALS=<your_service_account_file>.json

Where <your_service_account_file> is the path to the .json file containing your Google Cloud Service Account credentials.

(This file has sensitive info, you shouldn't add it to your code repository)

Let's go to the code...

On the LiveView side

In our LiveView file we must enable file uploads. To do this in the mount function, we modify:

  def mount(socket) do
    {:ok,
     allow_upload(
       socket,
       :photos,
       accept: ~w(.png .jpeg .jpg),
       max_entries: 2,
       external: &presign_entry/2
     )}
  end

The accept property enables file uploads by accepting only .png, .jpeg or .jpg formats, in this case. The max_entries property allows the upload of a maximum of two (2) files.

The external property must be a callback function with two parameters. This function will actually be in charge of calling the function that uploads the files to Google Drive, passing it the necessary data to perform the upload. When we use external it is because the function that will be in charge of uploading the files will be a JavaScript function (on the client side). We will do it with JavaScript, since later we want to know the upload progress of each of the files, to know if there is an error in this upload process, or if there is an error in the validation of the files. All this using JavaScript's XMLHttpRequest.

Add the presign_entry function in your LiveView file:

  @google_drive_url "https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart"
  @google_drive_scope "https://www.googleapis.com/auth/drive.file"
  defp presign_entry(entry, socket) do
    {:ok, %{token: token}} = Goth.Token.for_scope(@google_drive_scope)

    fields = %{
      name: "#{entry.uuid}.#{ext(entry)}",
      content_type: entry.client_type,
      token: token
    }

    {:ok, %{uploader: "GoogleDriveMultipart", url: @google_drive_url, fields: fields}, socket}
  end

  defp ext(entry) do
    [ext | _] = MIME.extensions(entry.client_type)
    ext
  end

Add the handle_event callback function to handle the event of canceling the upload of one of the files:

def handle_event("cancel-entry", %{"ref" => ref}, socket) do
  {:noreply, cancel_upload(socket, :photos, ref)}
end

Add the Phoenix HTML components for files uploading:

    ...
    <%= live_file_input @uploads.photos %>

    <%= for {_ref, msg} <- @uploads.photos.errors do %>
      <p class="alert alert-danger">
        <%= Phoenix.Naming.humanize(msg) %>
      </p>
    <% end %>

    <%= for entry <- @uploads.photos.entries do %>
      <%= live_img_preview(entry) %>
      <progress max="100" value="<%= entry.progress %>" />
      <a
        href="#"
        phx-click="cancel-entry"
        phx-value-ref="<%= entry.ref %>"
      >
        Cancel
      </a>
    <% end %>

On the JavaScript side

In the app.js file enable external file uploads by adding the following:

import { uploadPhotosToGoogleDrive } from "./uploaders/google-drive"

const Uploaders = {
  GoogleDriveMultipart: uploadPhotosToGoogleDrive
}

let liveSocket = new LiveSocket("/live", Socket, {
  uploaders: Uploaders,
  params: { _csrf_token: csrfToken }
})

In the uploaders folder create the google-drive.js file adding the following:

const createRequestPayload = (fields, photo) => {
  const boundary = 'uploading photos'
  const multipartRequestHeaders = [
    ['Content-Type', `multipart/related; boundary="${boundary}"`],
    ['Authorization', `Bearer ${fields.token}`]
  ]
  const delimiter = "\r\n--" + boundary + "\r\n"
  const close_delim = "\r\n--" + boundary + "--"
  const contentType = fields.content_type
  const metadata = {
    'name': fields.name,
    'mimeType': contentType,
    'parents': [fields.parent]
  }

  const base64Data = btoa(photo)
  const multipartRequestBody =
    delimiter +
    'Content-Type: application/json; charset=UTF-8\r\n\r\n' +
    JSON.stringify(metadata) +
    delimiter +
    'Content-Type: ' + contentType + '\r\n' +
    'Content-Transfer-Encoding: base64\r\n' +
    '\r\n' +
    base64Data +
    close_delim

  return {
    multipartRequestHeaders,
    multipartRequestBody
  }
}

export const uploadPhotosToGoogleDrive = (entries, onViewError) => {
  entries.forEach(entry => {
    const { file, meta: { url, fields } } = entry

    const reader = new FileReader()

    reader.readAsBinaryString(file)
    reader.onload = () => {
      const {
        multipartRequestHeaders,
        multipartRequestBody
      } = createRequestPayload(fields, reader.result)

      const xhr = new XMLHttpRequest()
      onViewError(() => xhr.abort())

      xhr.onprogress = event => {
        if (event.lengthComputable) {
          const percent = Math.round((event.loaded / event.total) * 100)
          entry.progress(percent)
        }
      }

      xhr.open("POST", url, true)
      multipartRequestHeaders.forEach(([key, value]) => {
        xhr.setRequestHeader(key, value)
      })

      xhr.send(multipartRequestBody)

      xhr.onload = () => {
        if (xhr.status !== 200) {
          return entry.error()
        }
      }
      xhr.onerror = () => entry.error()
    }
  })
}

That's it! Let's try...

When testing the file upload we can see how the upload progress shows the bar at 100% complete (in green color).

And finally, in the DevTools we can see a successful response obtained by Google Drive API from which we can know the kind of upload, the ID of the file on Google Drive, the name and format of the file.

Great, you have your file in Google Drive!

To see the full implementation you can visit the repo:

santiagocardo / car-workshop

Car Workshop Managment Web App

CarWorkshop

To start your Phoenix server:

Install dependencies with mix deps.get
Create and migrate your database with mix ecto.setup
Install Node.js dependencies with npm install inside the assets directory
Start Phoenix endpoint with mix phx.server

Now you can visit localhost:4000 from your browser.

Ready to run in production? Please check our deployment guides.

Learn more

Official website: https://www.phoenixframework.org/
Guides: https://hexdocs.pm/phoenix/overview.html
Docs: https://hexdocs.pm/phoenix
Forum: https://elixirforum.com/c/phoenix-forum
Source: https://github.com/phoenixframework/phoenix

View on GitHub

Cargar archivos a Google Drive con Phoenix LiveView

Santiago Cardona — Mon, 19 Apr 2021 21:37:17 +0000

Phoenix LiveView tiene una gran abstracción a la hora de realizar subida de archivos. Nos facilita bastante la vida tanto desde el lado del cliente (interfaz de usuario) como desde el lado del servidor (nuestro servidor o incluso a servicios de terceros como servicios en la nube).

Este post está enfocado en ayudarte con la subida de archivos al servicio de Google Drive, ya que muchas veces los servicios de Google son difíciles de entender y lleva tiempo el poder descubrir la manera de hacer una integración exitosa, como lo fue mi caso. Es por esto que quisiera compartir con la comunidad la manera de como logré hacerlo después de mucho tiempo de búsqueda y ensayos.

Comencemos...

Esta guía no pretende mostrar el detalle de cómo funciona el proceso de subida de archivos en Phoenix LiveView. Mejor, pretende mostrar la integración de Google Drive con lo que ya las guías External Uploads de Phoenix LiveView y Phoenix LiveView Uploads Deep Dive por Chris McCord muestran de una manera super clara y fácil de entender.

Para conocer el detalle de cómo funciona el proceso de subida de archivos en Phoenix LiveView puedes mirar las guías anteriormente mencionadas.

Requerimientos previos

Lo primero que debemos tener en cuenta es que debemos habilitar el acceso a la API de Google Drive, esto lo puedes lograr visitando la documentación de Google Drive API.

Asegúrate de crear una Cuenta de Servicio en Google Cloud y por último tener el archivo .json con las credenciales de tu Cuenta de Servicio de Google Cloud. Este archivo debe contener algo parecido a esto:

{
  "type": "service_account",
  "project_id": "<your google cloud project>",
  "private_key_id": "<your private key id>",
  "private_key": "<your private key>",
  "client_email": "<your client email>",
  "client_id": "<your client id>",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "<your client x509 cert url>"
}

Configuración de dependencias

Para poder realizar la autenticación con los servicios de Google usaremos la librería Goth (Google + Auth).

Añadimos Goth a nuestras dependencias en el archivo mix.exs

  defp deps do
    [
      ...
      {:goth, "~> 1.2.0"}
    ]
  end

Corremos la siguiente línea en la consola para descargar nuestras dependencias:

$ mix deps.get

Y por último, debemos exponer una variable de ambiente llamada GOOGLE_APPLICATION_CREDENTIALS con la cual Goth tendría lo necesario para obtener un access token:

$ export GOOGLE_APPLICATION_CREDENTIALS=<your_service_account_file>.json

Donde <your_service_account_file> es el path al archivo .json que contiene las credenciales de tu Cuenta de Servicio de Google Cloud.

(Este archivo es sencible, no lo deberías añadir en tu repositorio de código)

Vamos al código...

Del lado de nuestra LiveView

En nuestro archivo de LiveView debemos habilitar la carga de archivos. Para ello en la función mount, modificamos:

  def mount(socket) do
    {:ok,
     allow_upload(
       socket,
       :photos,
       accept: ~w(.png .jpeg .jpg),
       max_entries: 2,
       external: &presign_entry/2
     )}
  end

La propiedad accept habilita la subida de archivos aceptando únicamente formatos .png, .jpeg o .jpg, en este caso. La propiedad max_entries permite la subida de máximo dos (2) archivos.

La propiedad external debe ser una función callback con dos parámetros. Esta función será en realidad la encargada de llamar a la función que sube los archivos a Google Drive, pasándole los datos necesarios para realizar la subida. Cuando usamos external es porque la función que se encargará de subir los archivos será una función de JavaScript (en el lado del cliente). Lo haremos con JavaScript, ya que más adelante queremos saber el progreso de subida de cada uno de los archivos, saber si hay un error en este proceso de subida, o si hay un error en la validación de los archivos. Todo esto usando XMLHttpRequest de JavaScript.

Añadimos nuestra función presign_entry en nuestro archivo de LiveView:

  @google_drive_url "https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart"
  @google_drive_scope "https://www.googleapis.com/auth/drive.file"
  defp presign_entry(entry, socket) do
    {:ok, %{token: token}} = Goth.Token.for_scope(@google_drive_scope)

    fields = %{
      name: "#{entry.uuid}.#{ext(entry)}",
      content_type: entry.client_type,
      token: token
    }

    {:ok, %{uploader: "GoogleDriveMultipart", url: @google_drive_url, fields: fields}, socket}
  end

  defp ext(entry) do
    [ext | _] = MIME.extensions(entry.client_type)
    ext
  end

Añadimos la función callback handle_event para manejar el evento de cancelar la subida de alguno de los archivos:

def handle_event("cancel-entry", %{"ref" => ref}, socket) do
  {:noreply, cancel_upload(socket, :photos, ref)}
end

Añadimos los componentes Phoenix HTML para la subida de los archivos:

    ...
    <%= live_file_input @uploads.photos %>

    <%= for {_ref, msg} <- @uploads.photos.errors do %>
      <p class="alert alert-danger">
        <%= Phoenix.Naming.humanize(msg) %>
      </p>
    <% end %>

    <%= for entry <- @uploads.photos.entries do %>
      <%= live_img_preview(entry) %>
      <progress max="100" value="<%= entry.progress %>" />
      <a
        href="#"
        phx-click="cancel-entry"
        phx-value-ref="<%= entry.ref %>"
      >
        Cancel
      </a>
    <% end %>

Del lado de JavaScript

En el archivo app.js habilitamos la carga externa de archivos añadiendo lo siguiente:

import { uploadPhotosToGoogleDrive } from "./uploaders/google-drive"

const Uploaders = {
  GoogleDriveMultipart: uploadPhotosToGoogleDrive
}

let liveSocket = new LiveSocket("/live", Socket, {
  uploaders: Uploaders,
  params: { _csrf_token: csrfToken }
})

En la carpeta uploaders creamos el archivo google-drive.js añadiendo lo siguiente:

const createRequestPayload = (fields, photo) => {
  const boundary = 'uploading photos'
  const multipartRequestHeaders = [
    ['Content-Type', `multipart/related; boundary="${boundary}"`],
    ['Authorization', `Bearer ${fields.token}`]
  ]
  const delimiter = "\r\n--" + boundary + "\r\n"
  const close_delim = "\r\n--" + boundary + "--"
  const contentType = fields.content_type
  const metadata = {
    'name': fields.name,
    'mimeType': contentType,
    'parents': [fields.parent]
  }

  const base64Data = btoa(photo)
  const multipartRequestBody =
    delimiter +
    'Content-Type: application/json; charset=UTF-8\r\n\r\n' +
    JSON.stringify(metadata) +
    delimiter +
    'Content-Type: ' + contentType + '\r\n' +
    'Content-Transfer-Encoding: base64\r\n' +
    '\r\n' +
    base64Data +
    close_delim

  return {
    multipartRequestHeaders,
    multipartRequestBody
  }
}

export const uploadPhotosToGoogleDrive = (entries, onViewError) => {
  entries.forEach(entry => {
    const { file, meta: { url, fields } } = entry

    const reader = new FileReader()

    reader.readAsBinaryString(file)
    reader.onload = () => {
      const {
        multipartRequestHeaders,
        multipartRequestBody
      } = createRequestPayload(fields, reader.result)

      const xhr = new XMLHttpRequest()
      onViewError(() => xhr.abort())

      xhr.onprogress = event => {
        if (event.lengthComputable) {
          const percent = Math.round((event.loaded / event.total) * 100)
          entry.progress(percent)
        }
      }

      xhr.open("POST", url, true)
      multipartRequestHeaders.map(([key, value]) => {
        xhr.setRequestHeader(key, value)
      })

      xhr.send(multipartRequestBody)

      xhr.onload = () => {
        if (xhr.status !== 200) {
          return entry.error()
        }
      }
      xhr.onerror = () => entry.error()
    }
  })
}

Eso es todo! Probemos...

Al probar la carga de archivos podremos ver cómo el progreso de subida muestra la barra al 100% completa (en color verde).

Y por último, en los DevTools podremos ver una respuesta exitosa obtenida por Google Drive API de la cual podremos saber el tipo de subida, el ID del archivo dentro de Google Drive, el nombre y el formato de este.

Y listo, tienes tu archivo en Google Drive!

Para ver la implementación completa puedes visitar el repo:

santiagocardo / car-workshop

Car Workshop Managment Web App

CarWorkshop

To start your Phoenix server:

Install dependencies with mix deps.get
Create and migrate your database with mix ecto.setup
Install Node.js dependencies with npm install inside the assets directory
Start Phoenix endpoint with mix phx.server

Now you can visit localhost:4000 from your browser.

Ready to run in production? Please check our deployment guides.

Learn more

Official website: https://www.phoenixframework.org/
Guides: https://hexdocs.pm/phoenix/overview.html
Docs: https://hexdocs.pm/phoenix
Forum: https://elixirforum.com/c/phoenix-forum
Source: https://github.com/phoenixframework/phoenix

View on GitHub