DEV Community: Tam Nhat Ly

Build a small chat service using Elixir and deploy it on Amazon ec2 using AWS (Last part)

Tam Nhat Ly — Sat, 17 Aug 2024 12:21:14 +0000

Introduction

In Part 2, we covered the implementation of chat functionality with LiveView, LiveComponent, PubSub, and Channels. In this final article, we’ll focus on compiling the application locally and deploying it to AWS EC2.

Create Dockerfile

A Dockerfile outlines a series of commands for Docker to build the application environment. By using a Dockerfile, we can create a consistent image of a test environment, which can be effortlessly and reliably recreated without the need for manual setup and configuration for each test run.

This is the sample Dockerfile .



ARG ELIXIR_VERSION=1.16.3
ARG OTP_VERSION=26.1.2
ARG DEBIAN_VERSION=bullseye-20240612-slim

ARG BUILDER_IMAGE="hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-debian-${DEBIAN_VERSION}"
ARG RUNNER_IMAGE="debian:${DEBIAN_VERSION}"

FROM --platform=linux/amd64 ${BUILDER_IMAGE} as builder

# install build dependencies
RUN apt-get update -y && apt-get install -y build-essential git \
    && apt-get clean && rm -f /var/lib/apt/lists/*_*

# prepare build dir
WORKDIR /app
...

Run application by docker-compose

For ease of local testing, create a prod.env file with the following content:



export SECRET_KEY_BASE=<key>
export HOST=chat.vn
export PHX_HOST=chat.vn
export PHX_HTTP_PORT=8080
export PHX_HTTPS_PORT=443
export MIX_ENV=prod
export DATABASE_FILE="/data/chat_service_dev.db"
export DOCKER_DEFAULT_PLATFORM=linux/amd64

Replace <key> with own secret key by running the command mix phx.gen.secret. This prod.env file will automatically set the necessary environment variables for configuring our application.

Next, Docker Compose will help us run the application locally. Create a docker-compose.yml file in our project root with the following content:



version: '3'
services:
  elixir:
    build:
      network: host
      context: .
      dockerfile: Dockerfile
    env_file: prod.env
    environment:
      - DATABASE_FILE=/data/chat_service_dev.db
    platform: linux/amd64
    ports:
      - "4433:443"
      - "8080:8080"
    container_name: chat_service
    volumes:
      - ./data:/data
      - ./priv/cert:/cert

In this setup, Docker Compose will use the prod.env file to export the required environment variables into the running container.

To start the application, run docker-compose up or docker-compose up -d to run it in the background.

Once everything is up and running, we can open our browser and navigate to http://localhost:8080/chat_room?user_id=john&channel_id=game_1 to see the application in action.

After we will save image we just created to tar file:

docker save -o chat_service.tar <image_id> and we need to run docker image ls to get image_id.

Deploy Docker image to Amazon EC2

First, in AWS, navigate to the EC2 dashboard and launch an instance by visiting this link (https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#LaunchInstances).

Select Debian 12 with a 64-bit (x86) architecture, which is eligible for the free tier. This option is sufficient for our needs as our application doesn’t require a high-performance machine.

For the instance type, choose t2.micro. When setting up the Key Pair for login, create a new one and ensure we save it to our local machine. For all other settings, we can leave them as default.

Navigate to the EC2 instance dashboard to view our instances.

In the Security Groups section, we'll see that the SSH port is pre-configured by default. Since we’ll be hosting a web app on this instance, we need to add another configuration. By default, Phoenix’s web server runs on port 8080. To allow traffic on this port, go to the Security tab and add a new Inbound rule for port 8080.

Finally, we can start our EC2 instances by command:
aws ec2 start-instances --instance-ids <instance_id>.

Remember install aws command and get instance-ids name from dashboard.

Wait few seconds, check instance state in EC2 dashboard, we can see it will change status from Stopped -> Pending -> Running,

Locate your private key file. The key used to launch this instance is .pem. Run this command, if necessary, to ensure your key is not publicly viewable: chmod 400 "<name>.pem".

Then we will connect to the EC2 machine by command using its Public DNS and host name is admin:

ssh -i "<name>.pem" admin@<Public IPv4 DNS>

In EC2 machine, we also need to create prod.env and docker_compose.yml file:

prod.env:



SECRET_KEY_BASE="<key>"
HOST=chat.vn
PHX_HOST=chat.vn
PHX_HTTP_PORT=8080
PHX_HTTPS_PORT=443
MIX_ENV=prod
DATABASE_FILE="/data/chat_service_dev.db"

docker_compose.yml:



version: '3'
services:
  elixir:
    image: <image_id>
    env_file: prod.env
    platform: linux/amd64
    environment:
      - DATABASE_FILE=/data/chat_service_dev.db
    ports:
      - "4433:443"
      - "8080:8080"
    container_name: chat_service
    volumes:
      - ./data:/data
      - ./priv/cert:/cert

We’ll update in the next step. Previously, we saved the chat_service.tar file. To copy it to the EC2 instance, use the following command:



scp -i "<name>.pem"  chat_service.tar  admin@<Public IPv4 DNS>:/home/admin

Once the file is on the EC2 instance, load the image from the tar file using:



docker load -i chat_service.tar

Next, retrieve the image ID by running:



docker image ls

Update the in your docker-compose.yml file with the ID you obtained.

Next, run docker-compose up -d to start the application. You can then access the chat service at:

<Public IPv4 address>:8080/chat_room?user_id=jack&channel_id=game_1

If there are no errors and no spinner is displayed on the LiveView page, it means the WebSocket connection has been successfully established!

It’s been a productive day! Don’t forget to terminate/stop the EC2 instance if you no longer need it to avoid incurring additional costs.

Overall Key Takeaways

Complete Workflow: The series provides a comprehensive guide from building a chat service with Elixir and Phoenix to deploying it on AWS EC2 using Docker.
Real-Time Capabilities: Emphasizes the use of Phoenix Channels for real-time communication, a crucial feature for chat applications.
Containerization Benefits: Highlights the benefits of using Docker for packaging and deploying applications, ensuring consistent environments and simplifying deployment.
Cloud Deployment: Covers the deployment process to AWS EC2, including instance configuration, security settings, and cost management.

Repo: https://github.com/ohhi-vn/embedded_chat

Build a small chat service using Elixir and deploy it on Amazon ec2 using AWS (Part 2)

Tam Nhat Ly — Wed, 14 Aug 2024 12:56:48 +0000

Introduction

At Part 1, we explored how to implement the database for our chat service. In this article, we'll dive into how to implement chat functionality using LiveView, LiveComponent, PubSub, and Channels.

Channel

In Endpoint.ex, we will define the socket handler that will manage connections on a specified URL.

  socket "/live", ChatServiceWeb.UserSocket, websocket: true

  socket "/api/socket", ChatServiceWeb.UserSocket,
    websocket: true,
    longpoll: false

On the client side, you will establish a socket connection to the route above, the url of application will be like this: http://localhost:8080/chat_room?user_id=jack&channel_id=game_1

with user_id is your id and channel_id is name of room.

const userId = params.get('user_id');
const channelId = params.get('channel_id');

let channel = liveSocket.channel("channel:" + channelId, {params: {user_id: userId}})

channel.join()
  .receive("ok", resp => { console.log("Joined successfully", resp) })
  .receive("error", resp => { console.log("Unable to join", resp) })

On the server, Phoenix will call ChatServiceWeb.UserSocket.connect/2, passing your parameters along with the initial socket state. Within this function, you can authenticate and identify the socket connection, as well as set default socket assigns. The socket is also where you define your channel routes.

The asterisk * serves as a wildcard matcher. For example, in the following channel route, requests for channel:channel_id will all be directed to the game_channel.ex. In your UserSocket, you would have:

defmodule ChatServiceWeb.UserSocket do
  use Phoenix.LiveView.Socket

  channel "lv:*", Phoenix.LiveView.Channel
  channel "channel:*", ChatServiceWeb.GameChannel

  @impl true
  def connect(_params, socket, _connect_info) do
    {:ok, socket}
  end

  @impl true
  def id(_socket), do: nil
end

In game_channel.ex, we will implement the join/3 callback, allowing anyone to join the room using the channel_id. After joining the channel, the user will be redirected to the LiveView page, where we'll now discuss LiveView and LiveComponent.

defmodule ChatServiceWeb.GameChannel do
  use Phoenix.Channel
  alias Phoenix.PubSub
  alias ChatService.Room.RoomActions
  alias Phoenix.LiveView.Socket

  use Phoenix.LiveView

  @pubsub_chat_room Application.compile_env(:chat_service,
                                            [:pubsub, :chat_room])

  @impl true
  def join("channel:" <> channel_id,
           %{"params" => payload},
           %Phoenix.Socket{transport_pid: transport_pid} = socket) do

    user_id = Map.get(payload, "user_id", :not_found)
    is_embedded = Map.get(payload, "is_embedded", :not_found)
    if authorized?(user_id, payload) do
      topic = "channel:" <> channel_id
      if connected?(%Socket{transport_pid: transport_pid}), do:
        PubSub.broadcast(@pubsub_chat_room, topic, {:join_room,
                                                    %{channel_id: channel_id,
                                                      is_embedded: is_embedded,
                                                      user_id: user_id}})
      {:ok, socket}
    else
      {:error, %{reason: "unauthorized"}}
    end
  end

However, when game_channel.ex communicates with LiveView, it needs to use PubSub to transfer messages between them. In the ChatRoomLiveView module, you’ll need to subscribe to the PubSub topic within the mount function.

  def mount(params, session, socket) do
    {channel_id, user_id} =
      {Map.get(params, "channel_id", :not_found),
       Map.get(params, "user_id", :not_found)}

    topic = "channel:" <> channel_id
    :ok = PubSub.subscribe(@pubsub_chat_room, topic)
...
end

Using this Pubsub, whenever we need to send message from channel to liveview, we just use PubSub.broadcast/3.

We will design simple chat service page like this, this is using liveview page.

There are two main components: the discussion area on the left side and the list of current users online for the room on the right side. Everything else is part of the LiveView page.

Whenever a user joins the room or sends a message, the LiveView page receives the message from the channel via PubSub, updates the message history in the message component, and refreshes the list of online users.

For example, when a user sends a new message, we handle the keypress event and use channel.push to send the message from JavaScript (app.js) to the channel:

let chatInput = document.querySelector("#chat-input")

chatInput.addEventListener("keypress", event => {
    if(event.key === 'Enter' && chatInput.value != ""){
        channel.push("new_msg", {body: chatInput.value, user_id: userId})
      chatInput.value = ""
    }
  })

In game_channel.ex, we will implement the handle_in/3 callback to receive new messages from the client side, along with the channel_id as the topic. The new message will be saved, associating it with the user, channel_id, and message body. We use the spawn function to run the save_message function asynchronously, as we don't need to wait for it to complete.

  @impl true
  def handle_in("new_msg", %{"body" => body, "user_id" => user_id},
                %Phoenix.Socket{transport_pid: transport_pid,
                                topic: "channel:" <> channel_id} = socket) do
    if connected?(%Socket{transport_pid: transport_pid}) do
      topic = "channel:" <> channel_id
      spawn(fn -> save_message(channel_id, body, user_id) end)
      PubSub.broadcast(@pubsub_chat_room, topic, {:send_msg,
                                                  %{body: body,
                                                    user_id: user_id,
                                                    channel_id: channel_id}})
     end
    {:noreply, socket}
  end

  # save message into database.
  defp save_message(channel_id, body, user_id) do
    [room_id] = RoomActions.get_room_id_of_channel(channel_id)
    {:ok, _} = RoomActions.add_chat(%{message: body, user_id: user_id,
                                      room_id: room_id})
  end

On the LiveView page, we will use the handle_info/2 callback to capture messages from the channel and then update the message history accordingly.

  @impl true
  def handle_info({:send_msg,
                   %{body: body, user_id: user_id}},
                   %{assigns: %{message: old_msg}} = socket) do
    new_msg = old_msg ++ [%{messages: body, user_name: user_id}]
    # it will send update message to ChatServiceWeb.MessagesComponent
    # because we assign new :message for socket (socket is changed)
    {:noreply, assign(socket, :message, new_msg)}
  end

At message live componnent page, we just need to assign message to socket.

  @impl true
  def update(%{message: message}, socket) do
    {:ok, assign(socket, :message, message)}
  end

Finally, when we run command iex -S mix phx.server, and we can access url http://localhost:8080/chat_room?user_id=jack&channel_id=game_1 in browser, and chat something with your friend.

Next part, we will discover how to run the application in EC2 Amazon AWS (Part 3).

Repo: https://github.com/ohhi-vn/embedded_chat

Build a small chat service using Elixir and deploy it on Amazon ec2 using AWS (Part 1)

Tam Nhat Ly — Mon, 12 Aug 2024 13:53:01 +0000

Introduction

Let's build a distributed chat service application using Elixir. It will be a simple project where users can join rooms and send messages to others, but it will also introduce some fascinating concepts and foundational building blocks in Elixir development.

This application will leverage key techniques such as LiveView, LiveComponent, Channels, PubSub, and Ecto. We'll deploy it on AWS EC2 instances, and the application will be compiled locally using Docker.

This blog series will cover:

Building the database with Ecto (This blog).
Implementing chat functionality using LiveView, LiveComponent, PubSub, and Channels.
Compiling the application locally and deploying it on AWS EC2.

This is not an exhaustive guide and won't cover every use case. It may not adhere to all security best practices, as the goal is to explore interesting concepts and provide a starting point for your development journey. Additionally, the application may not handle all error cases and could report errors unexpectedly.

Interested? Let's get started!

Ecto Introduction

Ecto is split into 4 main components:

Ecto.Repo - repositories are wrappers around the data store. Via the repository, we can create, update, destroy and query existing entries. A repository needs an adapter and credentials to communicate to the database

Ecto.Schema - schemas are used to map external data into Elixir structs. We often use them to map database tables to Elixir data but they have many other use cases

Ecto.Query - written in Elixir syntax, queries are used to retrieve information from a given repository. Ecto queries are secure and composable

Ecto.Changeset - changesets provide a way to track and validate changes before they are applied to the data

In summary:
Ecto.Repo - where the data is
Ecto.Schema - what the data is
Ecto.Query - how to read the data
Ecto.Changeset - how to change the data

Implement Database

The database of the application will be simple including 4 tables:
Chat, User, Room and Member.

Overview of the table as bellow diagram:

Besides, we will use sqlite3 for communicating to the database instead of Postgres because when using sqlite3 we do not need to set up user name and password for it, and it is suitable for this small chat application.

We will name the application ChatService and create a room directory under lib/chat_service. In this directory, we will define the schemas and changesets for four tables within lib/chat_service/room, such as: chat, member_info, room_info and user_info.

We will take a look at user_info.ex and room_info.ex:

room_info.ex



defmodule ChatService.Room.RoomInfo do
  use Ecto.Schema
  import Ecto.Changeset

  @primary_key {:room_id, :id, autogenerate: true}
  schema "room" do
    field :channel_id, :string
    has_many :chat, ChatService.Room.ChatInfo, foreign_key: :room_id, references: :room_id
    has_many :member, ChatService.Room.MemberInfo, foreign_key: :room_id, references: :room_id

    timestamps()

  end

  def changeset(room, params \\ %{}) do
    room
    |> cast(params, [:channel_id])
    |> validate_required([:channel_id])
    |> unique_constraint(:room_id)
  end

end

user_info.ex:



defmodule ChatService.Room.UserInfo do
  use Ecto.Schema
  import Ecto.Changeset

  @primary_key {:user_id, :string, autogenerate: false}
  schema "user" do
    has_many :chat, ChatService.Room.ChatInfo, foreign_key: :user_id, references: :user_id
    has_many :member, ChatService.Room.MemberInfo, foreign_key: :user_id, references: :user_id
    timestamps()

  end

  def changeset(chat, params \\ %{}) do
    chat
    |> cast(params, [:user_id])
    |> unique_constraint(:user_id)
    |> validate_required([:user_id])
  end

  def registration_changeset(user, params \\ %{}) do
    cast(user, params, [])
  end
end

As you can see, we will use @primary_key in the user and room schemas to specifically configure the schema's primary key and
to indicate a one-to-many association with another schema we will use has_many.

After creating all schemas and changesets, we will generate a migration using the command mix ecto.gen.migration create_room_table. Once generated, you can check the priv/repo/migrations directory, where you’ll find a new migration file with a timestamp like <time_stamp>_create_room_table.ex.

In this migration file, we will create four tables, and we'll also add unique indexes for the user, room, and member tables.

To migrate a repository we will use command mix ecto.migrate.

API interaction with database

We will create a set of APIs to interact with the database in the lib/chat_service/room directory, and we'll name this module room_action.ex, such as: add_room, add_chat, add_user, etc.



defmodule ChatService.Room.RoomActions do

  alias ChatService.Room.ChatInfo
  alias ChatService.Room.RoomInfo
  alias ChatService.Room.UserInfo
  alias ChatService.Room.MemberInfo
  alias ChatService.Repo
  import Ecto.Query

  @doc """
  Add RoomInfo entry with channel id.
  """
  @spec add_room(map()) ::
    {:ok, Ecto.Schema.t()} | {:error, Ecto.Changeset.t()}
  def add_room(attrs \\ %{}) do
    %RoomInfo{}
    |> RoomInfo.changeset(attrs)
    |> Repo.insert()
  end

  @doc """
  Add ChatInfo entry.
  """
  @spec add_chat(map()) ::
    {:ok, Ecto.Schema.t()} | {:error, Ecto.Changeset.t()}
  def add_chat(attrs \\ %{}) do
    %ChatInfo{}
    |> ChatInfo.changeset(attrs, [:user_id, :room_id])
    |> Repo.insert()
  end

  ...
end

Next part, we will discover more technique to build the application such as: LiveView, LiveComponent, PubSub, and Channels. (Part 2)

Repo: https://github.com/ohhi-vn/embedded_chat