Perfect Elixir: Foundations of a Web App

Today we'll make a simple web app and explore the nuances and tradeoffs between different solutions. By the end, we'll have examples of apps that call into our database and return some HTML. Let's see which solution stands out as the most applicable to our needs.

Table of Contents

• Nothing At All
• Cowboy
• Plugged Cowboy
• Bandit
  • Connecting To Database
• Phoenix Framework
  • What Phoenix Gives Us
• Conclusion

Nothing At All

What if we don't choose a solution? It is technically possible to serve HTML directly using the Erlang TCP/IP socket module. This isn't a viable solution, but it's illuminating to see what it takes to work without abstractions.

We'll need to listen on a TCP socket and wait for traffic, then send data when a request arrives. Here's a script that does that:

defmodule SimpleServer do
  def start(port) do
    # Listen on a TCP socket on the specified port
    #   :binary       - Treat data as raw binary, instead of
    #                   being automatically converted into
    #                   Elixir strings (which are UTF-8 encoded).
    #                   It'd be unnecessary to convert, as the
    #                   HTTP protocol uses raw bytes.
    #   packet: :line - Frame messages using newline delimiters,
    #                   which is the expected shape of HTTP-data
    #   active: false - Require manual fetching of messages. In
    #                   Erlang, active mode controls the
    #                   automatic sending of messages to the
    #                   socket's controlling process. We disable
    #                   this behavior, so our server can control
    #                   when and how it reads data
    {:ok, socket} = :gen_tcp.listen(port, [
      :binary, packet: :line, active: false
    ])
    IO.puts("Listening on port #{port}")
    loop_handle_client_connection(socket)
  end

  defp loop_handle_client_connection(socket) do
    # Wait for a new client connection. This is a blocking call
    # that waits until a new connection arrives.
    # A connection returns a `client_socket` which is connected
    # to the client, so we can send a reply back.
    {:ok, client_socket} = :gen_tcp.accept(socket)
    send_hello_world_response(client_socket)
    :gen_tcp.close(client_socket)

    # Recursively wait for the next client connection
    loop_handle_client_connection(socket)
  end

  defp send_hello_world_response(client_socket) do
    # Simple HTML content for the response.
    content = "<h1>Hello, World!</h1>"

    # Generate the entire raw HTTP response, which includes 
    # calculating content-length header.
    response = """
    HTTP/1.1 200 OK
    content-length: #{byte_size(content)}
    content-type: text/html

    #{content}
    """
    :gen_tcp.send(client_socket, response)
  end
end

SimpleServer.start(8080)

We can now starts the script in one terminal and probe it with curl in another to see HTML being returned:

$ elixir simple_server.exs    | $ curl http://localhost:8080
Listening on port 8080        | <h1>Hello, World!</h1>%

This works, if by "works" we mean this is technically returning HTML.

Is it actually useful? Not in any practical sense no, but it does hint at the hard work that gets solved by the libraries we'll explore next.

ℹ️ BTW the full code to this section is here.

Cowboy

Cowboy is a minimalist and popular HTTP server, written in Erlang and used in many Elixir projects.

To try Cowboy, we scaffold an Elixir project with mix:

$ mix new cowboy_example
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/cowboy_example.ex
* creating test
* creating test/test_helper.exs
* creating test/cowboy_example_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd cowboy_example
    mix test

Run "mix help" for more commands.

And then we add Cowboy as a dependency and install dependencies:

$ cd cowboy_example
$ git-nice-diff -U1
cowboy_example/mix.exs
L#23:
     [
+      {:cowboy, "~> 2.11"}
       # {:dep_from_hexpm, "~> 0.3.0"},
$ mix deps.get
Resolving Hex dependencies...
Resolution completed in 0.043s
New:
  cowboy 2.12.0
  cowlib 2.13.0
  ranch 1.8.0
* Getting cowboy (Hex package)
* Getting cowlib (Hex package)
* Getting ranch (Hex package)
You have added/upgraded packages you could sponsor, run 
`mix hex.sponsor` to learn more

ℹ️ BTW git-nice-diff is just a small script that works like git diff but simplifies the output to make it easier to show diffs in this article. You can find it here if you're curious.

And then we add Cowboy as a dependency and install it:

defmodule CowboyExample do
  def start_server do
    # Set up the routing table for the Cowboy server, so root 
    #requests ("/") direct to our handler.
    dispatch = :cowboy_router.compile([{:_, [
      {"/", CowboyExample.HelloWorldHandler, []}
    ]}])

    # Start the Cowboy server in "clear mode" aka plain HTTP
    #   options - Configuration options for the server itself
    #             (this also supports which IP to bind to,
    #             SSL details, etc.)
    #   `env`   - Configuration map for how the server
    #             handles HTTP requests
    #             (this also allows configuring timeouts,
    #             compression settings, etc.)
    {:ok, _} =
      :cowboy.start_clear(
        :my_name,
        [{:port, 8080}],
        %{env: %{dispatch: dispatch}}
      )

    IO.puts("Cowboy server started on port 8080")
  end
end

defmodule CowboyExample.HelloWorldHandler do
  # `init/2` is the entry point for handling a new HTTP request
  # in Cowboy
  def init(req, _opts) do
    req = :cowboy_req.reply(200, %{
      "content-type" => "text/html"
    }, "<h1>Hello World!</h1>", req)

    # Return `{:ok, req, state}` where `state` is
    # handler-specific state data; here, it's `:nostate`
    # as we do not maintain any state between requests.
    {:ok, req, :nostate}
  end
end

Start the server:

$ iex -S mix                  | $ curl http://localhost:8080
Generated cowboy_example app. | <h1>Hello World!</h1>%
Erlang/OTP 26 [erts-14.2.2] [ |
source] [64-bit] [smp:12:12]  |
[ds:12:12:10] [async-threads: |
1] [dtrace]                   |
                              |
Interactive Elixir (1.16.1) - |
 press Ctrl+C to exit (type h |
() ENTER for help)            |
iex(1)> CowboyExample.start_s |
erver                         |
Cowboy server started on port |
 8080                         |
:ok                           |
iex(2)>                       |

We have moved up an abstraction level with Cowboy handling sockets instead of having to deal with them manually. It's still quite complex-looking code, and to work efficiently with it we'll have to learn Erlang-specific syntax and patterns.

ℹ️ BTW the full code to this section is here.

Plugged Cowboy

Cowboy is actually often used with Plug, an Elixir library that makes it easy to write HTML-responding functions. The plug_cowboy library combines both Cowboy and Plug dependencies, so let's give that a try.

Generate a new project with a supervisor:

$ mix new --sup plugged_cowboy
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/plugged_cowboy.ex
* creating lib/plugged_cowboy/application.ex
* creating test
* creating test/test_helper.exs
* creating test/plugged_cowboy_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd plugged_cowboy
    mix test

Run "mix help" for more commands.

And then we add our dependency:

$ cd plugged_cowboy
$ git-nice-diff -U1
/plugged_cowboy/mix.exs
L#24:
     [
+      {:plug_cowboy, "~> 2.0"}
       # {:dep_from_hexpm, "~> 0.3.0"},
$ mix deps.get
Resolving Hex dependencies...
Resolution completed in 0.104s
New:
  cowboy 2.10.0
  cowboy_telemetry 0.4.0
  cowlib 2.12.1
  mime 2.0.5
  plug 1.15.3
  plug_cowboy 2.7.0
  plug_crypto 2.0.0
  ranch 1.8.0
  telemetry 1.2.1
* Getting plug_cowboy (Hex package)
* Getting cowboy (Hex package)
* Getting cowboy_telemetry (Hex package)
* Getting plug (Hex package)
* Getting mime (Hex package)
* Getting plug_crypto (Hex package)
* Getting telemetry (Hex package)
* Getting cowlib (Hex package)
* Getting ranch (Hex package)
You have added/upgraded packages you could sponsor, run `mix hex.sponsor` to learn more

And we write a request-handling plug:

defmodule PluggedCowboy.MyPlug do
  import Plug.Conn

  # Plugs must define `init/1`, but we have nothing to configure so it's just a no-op implementation
  def init(options), do: options

  # `call/2` is the main function of a Plug, and is expected to process the request and generate a response
  def call(conn, _options) do
    conn
    # Both functions below are part of `Plug.Conn`s functions, they're available because we imported it
    |> put_resp_content_type("text/plain")
    |> send_resp(200, "Hello, World!")
  end
end

Configure and register Plug.Cowboy to use our plug:

$ git-nice-diff -U1
/plugged_cowboy/lib/plugged_cowboy/application.ex
L#10:
     children = [
+      # Connect  Plug.Cowboy plug handler
+      {Plug.Cowboy, plug: PluggedCowboy.MyPlug, scheme: :http, options: [port: 8080]}
       # Starts a worker by calling: PluggedCowboy.Worker.start_link(arg)

And… that all just works:

iex -S mix                    | $ curl http://localhost:8080
Erlang/OTP 26 [erts-14.2.2] [ | Hello, World!%
source] [64-bit] [smp:12:12]  | 
[ds:12:12:10] [async-threads: |
1] [dtrace]                   |
Interactive Elixir (1.16.1) - |
 press Ctrl+C to exit (type h |
() ENTER for help)            |
iex(1)>                       |

Plug definitely makes it easier to start the server, aligning our code with more idiomatic Elixir.

ℹ️ BTW the full code to this section is here.

Bandit

Bandit is a modern Elixir web server that integrates well with Erlang's concurrency model, offering great performance.

ℹ️ BTW the naming is fun: Bandit is an alternative to Cowboy. Internally Cowboy uses a dependency called Ranch to orchestrate sockets, and Bandit decided to call their socket-wrangling dependency… Thousand Island 😂 (non-native English speakers might have to look up the dictionary definitions to follow these puns).

Let's generate a new project and add dependencies:

$ mix new --sup bandit_example > /dev/null
$ cd bandit_example
$ git-nice-diff -U1
/bandit_example/mix.exs
L#24:
     [
+      {:bandit, "~> 1.0"},
+      {:plug, "~> 1.0"}
       # {:dep_from_hexpm, "~> 0.3.0"},
$ mix deps.get

ℹ️ BTW the > /dev/null just silences the mix new command, so we don't have to cover as much output.

And then we write the code:

$ git-nice-diff -U1
/bandit_example/lib/bandit_example/application.ex
L#10:
     children = [
+      {Bandit, plug: BanditExample.MyPlug, port: 8080}
       # Starts a worker by calling: BanditExample.Worker.start_link(arg)
/bandit_example/lib/bandit_example/my_plug.ex
L#1:
+defmodule BanditExample.MyPlug do
+  import Plug.Conn
+
+  def init(options), do: options
+
+  def call(conn, _options) do
+    conn
+    |> put_resp_content_type("text/plain")
+    |> send_resp(200, "Hello, World!")
+  end
+end

And that just works:

$ iex -S mix                  | $ curl http://localhost:8080    
                              | Hello, World!%
Erlang/OTP 26 [erts-14.2.2] [ |
source] [64-bit] [smp:12:12]  |
[ds:12:12:10] [async-threads: |
1] [dtrace]                   |
                              |
Interactive Elixir (1.16.1) - |
 press Ctrl+C to exit (type h |
() ENTER for help)            |
iex(1)>                       |

This turned out very similar to before, because Bandit is designed to be almost a drop-in replacement. But this time, what if we also try to connect to our database?

Connecting To Database

Add the Postgres adapter Postgrex:

$ git-nice-diff -U1
bandit_example/mix.exs
L#25:
       {:bandit, "~> 1.0"},
-      {:plug, "~> 1.0"}
+      {:plug, "~> 1.0"},
+      {:postgrex, ">= 0.0.0"}
       # {:dep_from_hexpm, "~> 0.3.0"},
$ mix deps.get > /dev/null

Initialize the database:

$ mkdir -p priv/db 
$ initdb -D priv/db
$ pg_ctl -D priv/db -l logfile start

And we also need a user and a database so there's something to connect to:

$ createuser -d bandit 
$ createdb -O bandit bandit

Register the Postgrex process:

$ git-nice-diff -U1
bandit_example/lib/application.ex
L#10:
     children = [
-      {Bandit, plug: BanditExample.MyPlug, port: 8080}
+      {Bandit, plug: BanditExample.MyPlug, port: 8080},
+      {Postgrex,
+        [
+          name: :bandit_db,
+          hostname: "localhost",
+          username: "bandit",
+          password: "bandit",
+          database: "bandit"
+        ]
+      }
     # Starts a worker by calling: BanditExample.Worker.start_link(arg)

Add a database query to the plug:

$ git-nice-diff -U1
bandit_example/lib/my_plug.ex
L#6:
   def call(conn, _options) do
+    %Postgrex.Result{rows: [[current_time]]} =
+      Postgrex.query(:bandit_db, "SELECT NOW() as current_time", [])
+
     conn
     |> put_resp_content_type("text/plain")
-    |> send_resp(200, "Hello, World!")
+    |> send_resp(200, "Hello, World! It's #{current_time}")
   end

Start the server:

20:04:22.598 [info] Running B | $ curl http://localhost:8080
anditExample.MyPlug with Band | Hello, World! It's 2024-03-17
it 1.3.0 at 0.0.0.0:8080 (htt |  19:04:24.547992Z%
p)                            | 
Erlang/OTP 26 [erts-14.2.2] [ |
source] [64-bit] [smp:12:12]  |
[ds:12:12:10] [async-threads: |
1] [dtrace]                   |
                              |
Interactive Elixir (1.16.1) - |
 press Ctrl+C to exit (type h |
() ENTER for help)            |
iex(1)>                       |

🎉 We now have a basic web app connected to its database.

ℹ️ BTW the full code to this section is here.

Phoenix Framework

We can't skip Phoenix, the dominant framework for Elixir web development. Phoenix is a powerful, flexible, and highly ergonomic framework for writing very scalable web applications.

Generate and bootstrap a Phoenix sample app:

$ mix local.hex --force && mix archive.install hex phx_new --force
$ mix phx.new my_app  # answer yes when prompted
$ cd my_app
$ mkdir -p priv/db && initdb -D priv/db && pg_ctl -D priv/db -l logfile start && createuser -d postgres
$ mix deps.get && mix ecto.create

ℹ️ BTW if you get database errors, you might need to reset Postgres:

$ lsof -ti :5432 | xargs -I {} kill {}; rm -rf priv/db # Kill all processes on Postgres' default port
$ rm -rf priv/db # Delete the local DB data folder

Now start the app with iex -S mix phx.server and visit http://localhost:4000:

What Phoenix Gives Us

The Phoenix generator connects to Postgres using Ecto which is a wrapper to efficiently create database queries.

And Ecto actually uses Postgrex under the hood; config/dev.exs has the same pattern of hardcoded database credentials that we used in the Bandit section:

$ cat config/dev.exs
…
# Configure your database
config :my_app, MyApp.Repo,
  username: "postgres",
  password: "postgres",
  hostname: "localhost",
  database: "my_app_dev",
…

Let's use the database in a module and call that function from the controller:

$ git-nice-diff -U1
my_app/lib/my_app/query.ex
L#1:
+defmodule MyApp.Query do
+  import Ecto.Query
+
+  alias MyApp.Repo
+
+  def get_db_time do
+    # The SELECT 1 is a dummy table to perform a query without a table
+    query = from u in fragment("SELECT 1"), select: fragment("NOW()")
+    query |> Repo.all() |> List.first()
+  end
+end
my_app/lib/my_app_web/controllers/page_controller.ex
L#6:
     # so skip the default app layout.
-    render(conn, :home, layout: false)
+    db_time = MyApp.Query.get_db_time()
+    render(conn, :home, layout: false, db_time: db_time)
   end
my_app/lib/my_app_web/controllers/page_html/home.html.heex
L#42:
   <div class="mx-auto max-w-xl lg:mx-0">
+    <h1 class="text-lg">Database Time: <%= @db_time %></h1>
     <svg viewBox="0 0 71 48" class="h-12" aria-hidden="true">

This setup displays a database query result on the page:

ℹ️ BTW the full code to this section is here.

Conclusion

It's been a lot of fun trying out various solutions, and Phoenix ends up as the most realistic choice for our needs: It's simple, scalable, and has impressive features with great documentation. For our needs, Phoenix is a perfect fit, and I'll continue this article series based on Phoenix.