DEV Community: Sophie DeBenedetto

What's New in Elixir 1.16

Sophie DeBenedetto — Tue, 09 Jan 2024 12:00:00 +0000

The Elixir 1.16 release candidate is out now, and it comes with some compelling improvements to diagnostics, documentation, and a few other enhancements that make Elixir an even better choice for developers.

We'll dive into some of these changes and highlight Elixir's continued focus on developer happiness and community building.

Let's get started!

Better Diagnostics for Compiler Errors in Elixir

We'll begin with the improvements to compiler diagnostics provided in Elixir 1.16. Now, your mix compile errors come with detailed code snippets that tell you exactly where your code goes wrong.

You'll benefit from these improvements when you have syntax errors, mismatched delimiter errors, and generic compiler errors.

Syntax Errors

First up, you can see below that syntax errors now come with a pointer to exactly where the error happened. Let's say we have the following code:

defmodule Doctor do
  def show_prescription_codes do
    ["DR", "DS", "amp", &]
  end
end

If you run mix compile, you'll see a syntax error with an associated code snippet, like this:

sophiedebenedetto[ 7:30PM] ~/personal-projects/doctor  💖 mix compile
Compiling 1 file (.ex)

== Compilation error in file lib/doctor.ex ==
** (SyntaxError) invalid syntax found on lib/doctor.ex:48:26:
    error: syntax error before: ']'
    │
 48 │     ["DR", "DS", "amp", &]
    │                          ^
    │
    └─ lib/doctor.ex:48:26
    (elixir 1.16.0-rc.1) lib/kernel/parallel_compiler.ex:428: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

Note the ^ pointer to the exact location of the syntax error. With this, it's even easier to quickly spot and correct such errors.

Mismatched Delimiter Errors

Another diagnostic improvement comes with the code snippet representation for MismatchedDelimiterError occurrences. Given the following code with a mismatched delimiter:

defmodule Doctor do
  def show_prescription_codes do
    ["DR", "DS", "amp", "ad")
  end
end

Running mix compile will now show you the following:

sophiedebenedetto[ 7:29PM] ~/personal-projects/doctor  💖 mix compile
Compiling 1 file (.ex)

== Compilation error in file lib/doctor.ex ==
** (MismatchedDelimiterError) mismatched delimiter found on lib/doctor.ex:48:29:
    error: unexpected token: )
    │
 48 │     ["DR", "DS", "amp", "ad")
    │     │                       └ mismatched closing delimiter (expected "]")
    │     └ unclosed delimiter
    │
    └─ lib/doctor.ex:48:29
    (elixir 1.16.0-rc.1) lib/kernel/parallel_compiler.ex:428: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/8

This error includes a display of the code snippet that is causing it, and even highlights the unclosed and mismatched delimiters associated to the MismatchedDelimiterError.

Other Errors

You'll now see similarly detailed code snippets for all kinds of compiler errors, including errors that describe undefined variables. Let's look at an example of that now.

Given the following code:

def list_patients do
  Patient.for_doctor(doctor)
end

Running mix compile will display the following error:

sophiedebenedetto[ 7:36PM] ~/personal-projects/doctor  💖 mix compile
Compiling 1 file (.ex)
    error: undefined variable "doctor"
    │
 53 │     Patient.for_doctor(doctor)
    │                        ^^^^^^
    │
    └─ lib/doctor.ex:53:24: Doctor.list_patients/0


== Compilation error in file lib/doctor.ex ==
** (CompileError) lib/doctor.ex: cannot compile module Doctor (errors have been logged)

These improved diagnostics make it even easier for Elixir developers to understand compilation errors, catch and remedy basic mistakes, and write clean and functioning code. These improvements show Elixir's continued focus on developer experience.

ExDoc Content Matures with Elixir

Some of the most exciting changes coming in Elixir 1.16 are the improvements to official Elixir language docs, offered through ExDoc. The "Getting Started" guide from elixir-lang.org has been incorporated into the official Elixir documentation on Hex docs.

While this may not seem too exciting on the surface, it represents a concerted effort to refine and unify official guidance on the Elixir programming language. The language and the community have matured to such a stage that we can better align on official guides like this and provide the best agreed-upon direction for new developers.

Anti-Patterns

In this same vein, the official Elixir docs now include robust content on Elixir anti-patterns. This content is divided into four separate sections, covering code, design, processes, and metaprogramming.

Once again, this content is made possible by the maturity of the Elixir language. As Elixir has grown up, and as adoption and the community have grown, we're at a point where Elixir developers can align on what good and bad Elixir code looks like.

With this extensive guidance in the official docs, including framing context and code samples, Elixir developers have the resources they need to ship clean code that meets generally agreed-upon standards. It will be even easier for experienced and novice Elixir developers alike to write and maintain clean code in various scenarios.

And that's not it for docs enhancements.

ExDoc Cheatsheets In the Official Elixir Docs

The official docs now include an ExDoc cheatsheet for the first time. ExDoc cheatsheets allow developers to author quick, summary-type guides to modules and libraries.

The official Elixir docs now include an Enum module cheatsheet, and that's just the beginning.

Elixir maintainers are looking for more community members to contribute cheatsheets to the official docs. This is a great way for first-timers to get an Elixir language contribution! Get started by opening an issue that outlines what cheatsheet you'd like to add.

Taken together, these docs improvements all point to the maturity of the Elixir language and community and the continued focus on education and support for Elixir newcomers. The official introduction docs, the anti-pattern docs, and the inclusion of cheatsheets provide clear and accessible guidance to Elixir developers at every level. This kind of alignment and clarity makes Elixir even more accessible to novices. At the same time, it empowers experienced Elixir devs to deliver clean, standardized Elixir code that performs well.

More Enhancements

You might be interested in taking advantage of some of the smaller enhancements from this release in your upgraded Elixir apps. Let's explore them briefly now.

The `String.replace_invalid/2` Function

First up, there is a new String.replace_invalid/2 function that (you guessed it) lets you replace invalid characters in a string. Let's take a look at an example:

iex> String.replace_invalid("asd" <> <<0xFF::8>>)
"asd�"

The default behavior is to replace the invalid character with a �, but you can replace it with any character you like:

iex> String.replace_invalid("asd" <> <<0xFF::8>>, "😵")
"asd😵"

This will come in handy if you find yourself processing strings of text from external sources. Such text may contain invalid characters, and identifying and replacing such characters just got a lot easier.

New Functionality for `Task.yield_many/2`

Next up, let's take a look at the new option you can pass to Task.yield_many. Previously, you could tell Task.yield_many to timeout and stop waiting for tasks if a certain time limit was exceeded. Now, you can provoke that same behavior if a certain number of tasks has been reached. You can do so by providing the limit option.

Let's say you have the following code to spawn ten tasks. Each task will sleep for i number of seconds and then return a message indicating what happened. Then, we yield each task with its result, and iterate over those {task, result} tuples to operate on the result by IO.inspect-ing them:

def MyModule do
  def do_tasks do
    tasks =
      for i <- 1..10 do
        Task.async(fn ->
          Process.sleep(i * 1000)
          "I slept for #{i} seconds"
        end)
      end

    tasks_with_results = Task.yield_many(tasks, timeout: 10000)

    results =
      Enum.map(tasks_with_results, fn {task, res} ->
        # Shut down the tasks that did not reply nor exit
        res || Task.shutdown(task, :brutal_kill)
      end)

    # Here we are matching only on {:ok, value} and
    # ignoring {:exit, _} (crashed tasks) and `nil` (no replies)
    for {:ok, value} <- results do
      IO.inspect(value)
    end
  end
end

This code will continue to yield tasks and their results until all the tasks are done or the timeout is exceeded. Now, you can use the limit: task_num option instead of timeout, like this:

tasks_with_results = Task.yield_many(tasks, limit: 1)

If we execute our function in IEx, you'll see that the code stops waiting for tasks after just one task is yielded:

iex> MyModule.do_tasks()
"I slept for 1 seconds"
["I slept for 1 seconds"]

If the limit is reached before the default timeout (5000), or before all the tasks are done, then Task.yield_many returns immediately without triggering the :on_timeout behaviour. You can dig further into this functionality in the docs.

This new functionality gives you even more fine-grained control over how your program orchestrates async tasks.

The `Logger.levels/0` Function

The last enhancement I'll highlight here is the new Logger.levels/0 function. This function returns the list of all available log levels:

iex> Logger.levels()
[:error, :info, :debug, :emergency, :alert, :critical, :warning, :notice]

This provides a quick reminder of the levels you can use to configure your logger.

Wrap Up

With the 1.16 release, Elixir continues to be a compelling choice for developers looking for:

An excellent developer experience
A welcoming and supportive community
Strong and opinionated development guidelines
Powerful language capabilities to meet the needs of various programming scenarios.

The new developments in diagnostics, documentation, and language functionality continue to make Elixir a pleasure to work with. Better, more detailed diagnostic messages for compiler errors show Elixir's emphasis on developer experience, making it easier than ever before to identify and fix these errors.

Improved docs and the addition of the anti-patterns guide empowers both experienced and newbie Elixir devs to write clean and standardized code. And minor language improvements, like the functions we explored above, show Elixir's continued commitment to language excellence.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Observe Your Phoenix App with Structured Logging

Sophie DeBenedetto — Tue, 25 Jul 2023 12:00:00 +0000

In this post, we'll configure a Phoenix LiveView application to use a structured logger. We'll then use AppSignal to correlate log events with other telemetry signals, like exception reports and traces.

Along the way, you'll learn about the benefits of structured logging, and you'll see how to configure a distinct framework and application logger in your Phoenix app.

Let's get started!

What is Structured Logging?

Structured logs conform to a specific structure in which an agreed-upon set of key/value pairs describe common log attributes. Without structured logging, you might get a log line that looks something like this:

20:34:05.879 request_id=F1ssWNn62XasBTYAAAEj [info] GET /
20:34:05.911 request_id=F1ssWNn62XasBTYAAAEj [info] hello from the page controller
20:34:05.919 request_id=F1ssWNn62XasBTYAAAEj [info] Sent 200 in 39ms

A few bits of information make up each log statement:

a timestamp
a level (in this case, info)
a log message
a bit of metadata (the request_id)

In this unstructured format, all the data is included in one string, so it's difficult to search and analyze log statements based on each data point.

Structured logging solves this problem for us. It breaks each piece of information into its own key/value pair. Here's an example of these same log statements structured using the logfmt format:

timestamp="20:38:15.973 2023-05-01" level=info message="GET /" request_id=F1sskxS1k-w0jUIAAAKB
timestamp="20:38:16.013 2023-05-01" level=info message="hello from the page controller" request_id=F1sskxS1k-w0jUIAAAKB
timestamp="20:38:16.031 2023-05-01" level=info message="Sent 200 in 58ms" request_id=F1sskxS1k-w0jUIAAAKB

Now you can search and analyze log statements based on the timestamp, level, message, and request_id fields. For example, you could search all log statements with a shared request_id, filter log lines by their level key, or search logs within a range of timestamps.

Now that we can see the benefits of using structured logs, we can configure our Phoenix LiveView application to use the logfmt format in development and production. But first, we'll take a closer look at how Phoenix apps configure their loggers out of the box.

Logger Configuration in Phoenix

By default, your Phoenix application will configure its logger to output unstructured logs to the console. Open up your config.exs file, and you'll see something like this:

config :logger, :console,
  format: "$time $metadata[$level] $message\n",
  metadata: [:request_id]

This means the logger will use the console backend to print log messages to the console using the specified (unstructured) format. This configuration says that only the request_id from the default metadata list should be included in each log statement. Elixir's Logger adds a certain set of metadata to all log statements, and you can find the full list here. The request_id metadata, however, is not added by default with the Elixir Logger. Instead, it is added by the Plug.RequestId plug that you'll see added to your application's endpoint.ex file like this:

# endpoint.ex
plug Plug.RequestId

Let's take a quick peek at the rest of the default metadata by setting metadata: :all in config.exs:

config :logger, :console,
  format: "$time $metadata[$level] $message\n",
  metadata: :all

Now, when we fire up our app and visit the root path, we'll see something like this:

21:09:24.947 erl_level=info application=phoenix domain=elixir file=lib/phoenix/logger.ex function=phoenix_endpoint_stop/4 line=231 mfa=Phoenix.Logger.phoenix_endpoint_stop/4 module=Phoenix.Logger pid=<0.815.0> request_id=F1suRjwyEp4CjWAAAACi [info] Sent 200 in 2ms

That's a lot of information! Too much information for our development environment. These logs are so information-dense that they're difficult to parse with our human eyes. For that reason, you'll notice that the dev.exs file overwrites this configuration to simplify it:

config :logger, :console,
  format: "$metadata[$level] $message\n"

Here, the timestamp and metadata are excluded entirely, except for the metadata's $level data point.

Now that we've seen where and how your Phoenix app configures the logger out-of-the-box, let's configure the logger to emit structured logs that conform to logfmt.

Using Logfmt in Phoenix Development

First up, we'll configure our development environment to use the console logger backend with a logfmt formatter. For this, we need the LogfmtEx dependency added to our application. Open up your mix.exs file and add it like this:

{:logfmt_ex, "~> 0.4"}

Go ahead and run mix deps.get to install the new dependency.

Next up, we'll configure our application's console logger to use this formatter. In dev.exs, add the following:

config :logger, :console, format: {LogfmtEx, :format}

Now, when you start up your application, you should see structured logs like this:

timestamp="21:18:36.828 2023-05-01" level=info message="Sent 200 in 45ms" request_id=F1suxrg8tRisBTYAAAmh

Notice that we haven't overwritten the :metadata key from the console backend config in config.exs, so we're still only logging the request_id data point from the metadata.

Now we've successfully emitted structured logs to the console, but this isn't good enough for production. To operate a Phoenix application in production, we need to emit our logs to a third-party service that allows us to search, aggregate, and analyze them at scale. This is where AppSignal comes in.

In the next section, we'll incorporate the AppSignal logger into our Phoenix application and configure it to use logfmt.

Using Logfmt with the AppSignal Logger in Production

If you don't already have AppSignal installed and configured in your application, you can follow the instructions here. Assuming you have an AppSignal account and organization configured, we're ready to set up the AppSignal logger backend by following the steps in AppSignal's Logging from Elixir docs. We'll use the Elixir integration rather than set up a custom source.

In our runtime.exs file, we'll specify the Appsignal.Logger.Backend, like this:

config :logger, :backends, [
  {Appsignal.Logger.Backend, [group: "phoenix", format: "logfmt"]}
]

Also, in runtime.exs, you'll want to set the application's log level to :info, to ensure we send all logs at the info level and below to AppSignal:

config :logger, level: :info

This will forward our application's logs to our AppSignal account and group them under the "phoenix" group designation. It will treat the format of the log statements as logfmt and allow us to search and analyze our logs based on their attributes.

If you log into your AppSignal account, select your organization, and select "Logging" -> "Application", you'll see something like this:

Note that the log line includes the timestamp, hostname, level, and group (in this case, "phoenix"). Since we've specified "logfmt" for the Appsignal.Logger.Backend, each of these attributes is indexed and searchable. All we have to do is double-click any of these attributes on the log line itself, and the search filter on the top of the page will update and execute:

Let's take a closer look at one of these log lines now. The "hello from the page controller" log line is emitted explicitly in application code by the following line added to the PagesController:

# pages_controller.ex
require Logger

def home(conn, _params) do
  Logger.info("hello from the page controller")
  render(conn, :home, layout: false)
end

If we click the "+" icon next to the log line, we see all of the log line's metadata:

You can see that the Appsignal.Logger.Backend emits all of the default metadata, along with the request_id. Since we're using logfmt, each data point is searchable. So, just like we did with the log attributes displayed in the log line itself, we can click on any metadata attributes to get them added to the search. If you click on the request_id_string attribute, you'll see the search query update to look like this:

In fact, we can add additional metadata when we emit log lines (using calls to Logger), like this:

Logger.info("hello from the page controller", %{user_id: conn.assigns.current_user.id})

With that, the user_id attribute is displayed in the details of each log line and searchable just like the request_id metadata.

So far, we've:

Configured our Phoenix application's logger to use the Appsignal.Logger.Backend.
Taught that backend to use logfmt.
Seen logs emitted and consumed into AppSignal, and got a feel for the basic logging search functionality there.

Before we continue exploring the AppSignal logging UI and some of its more useful features, we need to address a problem that's cropped up.

Framework Vs. Application Logs

Taking another look at the log statements our Phoenix app has emitted to AppSignal so far, we see something like this:

05-01-2023 10:21:04.383[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]Access StreamChatWeb.Endpoint at http://localhost:4000
+05-01-2023 10:21:04.363[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]Running StreamChatWeb.Endpoint with cowboy 2.9.0 at 127.0.0.1:4000 (http)
+05-01-2023 10:19:02.709[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]GET /asdfads
+05-01-2023 10:18:51.429[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]Sent 200 in 4ms
+05-01-2023 10:18:51.426[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]GET /
+05-01-2023 10:18:51.388[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]Sent 302 in 252ms
+05-01-2023 10:18:51.137[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]POST /users/log_in
+05-01-2023 10:18:50.201[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]CONNECTED TO Phoenix.LiveView.Socket in 16µs Transport: :websocket Serializer: Phoenix.Socket.V2.JSONSerializer Parameters: %{"_csrf_token" => "UQshLhEWIzUbAXgLQTE5bFNXRW8LZ00w3xVHuWYmHt1avWPZ4e6-97ut", "_live_referer" => "undefined", "_mounts" => "0", "_track_static" => %{"0" => "http://localhost:4000/assets/app.css", "1" => "http://localhost:4000/assets/app.js"}, "vsn" => "2.0.0"}
+05-01-2023 10:18:50.130[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]Sent 200 in 34ms
+05-01-2023 10:18:50.097[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]GET /users/log_in
+05-01-2023 10:18:48.255[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]Sent 200 in 689µs
+05-01-2023 10:18:48.254[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]GET /
05-01-2023 10:18:48.187[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]hello from the page contoller
+05-01-2023 10:18:48.215[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]Sent 200 in 10ms
+05-01-2023 10:18:48.206[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]POST /users/log_out
05-01-2023 10:19:25.413[application][ERROR][SOPHIEs-MacBook-Pro-2.local][phoenix]** (RuntimeError) boom! iex:5: (file) (elixir 1.14.2) src/elixir.erl:309: anonymous fn/4 in :elixir.eval_external_handler/1 (stdlib 4.1.1) erl_eval.erl:748: :erl_eval.do_apply/7 (stdlib 4.1.1) erl_eval.erl:987: :erl_eval.try_clauses/10 (elixir 1.14.2) src/elixir.erl:294: :elixir.eval_forms/4 (elixir 1.14.2) lib/module/parallel_checker.ex:107: Module.ParallelChecker.verify/1 (iex 1.14.2) lib/iex/evaluator.ex:329: IEx.Evaluator.eval_and_inspect/3 (iex 1.14.2) lib/iex/evaluator.ex:303: IEx.Evaluator.eval_and_inspect_parsed/3 (iex 1.14.2) lib/iex/evaluator.ex:292: IEx.Evaluator.parse_eval_inspect/3 (iex 1.14.2) lib/iex/evaluator.ex:187: IEx.Evaluator.loop/1 (iex 1.14.2) lib/iex/evaluator.ex:32: IEx.Evaluator.init/4
+05-01-2023 10:18:47.578[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]Sent 200 in 7ms
+05-01-2023 10:18:47.572[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]GET /
+05-01-2023 10:18:46.901[application][INFO][SOPHIEs-MacBook-Pro-2.local][phoenix]Sent 200 in 52ms

There is a lot of noise in here mixed with some useful bits of information. You can see one error log that will very likely be helpful in debugging, as well as one instance of our "hello from the page controller" log statement. Those two log lines are very hard to detect amidst the many log lines emitted that indicate the server is up, the WebSocket channel is connected, certain web requests are served, and so on.

These noisy logs are emitted not from our application code, but from the Phoenix framework itself. They don't tell us anything interesting, and they are not an essential part of the overall observability picture of our application. Instead, they create noise and contribute to high log volume, making it harder to find and identify the log statements that we intend to emit from application code.

For this reason, framework logs should be emitted at the FATAL level. Application logs that are emitted when exceptions are raised or by explicit calls to Logger should log at the INFO level. Phoenix doesn't expose an easy way for us to do that, but we can build our own custom logger backend to solve this problem. Let's do that now.

Build A Custom Logger Backend

A logger backend is really nothing more than a GenServer, and must conform to the following API:

Implement an init/1 function
Implement a handle_event/3 function

Here's the skeleton for our backend:

# lib/stream_chat/logger/backend.ex
def init({__MODULE__, options}) do
    {:ok, Keyword.merge([group: __MODULE__, format: :plaintext], options)}
  end

  def handle_event(
        {level, _gl, {Logger, message, _timestamp, metadata}},
        options
      ) do
      # ...
  end
end

The init/1 function will be called with whatever config we pass to the call to configure the logger backend in runtime.exs. It returns the {:ok, state} tuple.

The handle_event/2 function will be called with all the data we need to emit a log statement with the Appsignal.Logger, including the:

Level
Log message
Metadata
Internal state of the logger backend process that we established in init/1

Whenever the Phoenix framework itself calls the Phoenix application's logger, the metadata variable passed to handle_event/2 will include an :application key pointing to a value of :phoenix. Where logs are emitted by an explicit call to Logger or by an exception raised in our application, the metadata keyword list will not include an :application key. We want to be able to write some code that says: "Only log statements coming from Phoenix if they are at the configured log level or below". We can do exactly that by adding some logic that looks for metadata with application: :phoenix.

Before we write that code, let's take a look at how we will configure this custom logger backend in our runtime.exs file:

config :logger, :backends, [
  {StreamChat.Logger.Backend,
   [
     format: "logfmt",
     application_config: [
       phoenix: [
         level_lower_than: :fatal
       ],
       default: [
        level_lower_than: :info
       ]
     ]
   ]}
]

Here, we include a key :application_config, which points to another keyword list. That keyword list contains configs for the :phoenix app along with the :default app. This data will be passed to the init/1 function and stored in our backend's state. So, when handle_event/2 is called later on, we can do the following:

Check if the metadata contains application: :phoenix.
If so, check the provided level and only log if it is at or below the level stored in application_config: [phoenix: [level_lower_than: level]] in state.

Here's what our function will look like:

 @levels %{
    fatal: 0,
    error: 1,
    warn: 2,
    info: 3,
    debug: 4,
    trace: 5
  }

def handle_event(
      {level, _gl, {Logger, message, _timestamp, metadata}},
      options
    ) do
  application = metadata[:application] || :default

  log_event(
    @levels[level],
    @levels[options[:application_config][application][:level_lower_than]],
    message,
    metadata,
    options
  )

  {:ok, options}
end

def log_event(level, level_lower_than, message, metadata, options)
    when level <= level_lower_than do
  Appsignal.Logger.log(
    level,
    to_string(metadata[:group] || options[:group]),
    IO.chardata_to_string(message),
    Enum.into(metadata, %{}),
    options[:format]
  )
end

def log_event(_level, _level_lower_than, _message, _metadata, _options),
  do: :noop

We pull the application config out of state and pass that along with the provided level to the log_event helper function. That function includes a guard clause that will cause it to no-op if the provided level is higher than the level configured in runtime.exs for the given application. If we should in fact log, we do so by calling directly on the AppSignal.Logger.log function. This allows us to emit logs to AppSignal at a specified level, for a given group, with a message and any metadata.

With this, we will suppress Phoenix framework logs above the FATAL level, but ensure that logs emitted by application code are emitted at INFO.

If you're hesitant to suppress framework logs in this way, remember that logs are just one piece of the observability puzzle. Rather than recording every instance of a GET / request, we are more likely to care about trends in serving such requests over time. We want the ability to drill down into the details of some such requests, especially where they are anomalous (for example, if a request experiences a high degree of latency).

This is where traces come in, along with other telemetry data, like metrics and exceptions.

In the next section, we'll look at how you can use AppSignal to correlate logs with traces and other types of telemetry, giving you a full picture of how your application is performing.

Correlating Log Events and Other Telemetry with AppSignal

Application logs are just one part of your overall observability posture. They should be used to record specific events of interest in your application's lifecycle and provide additional details and context in the event of an error or exception.

You should leverage application logs alongside other telemetry data like metrics, traces, and exception reports to fully observe your application in production.

Here's a very high-level overview of how you can view each type of telemetry data that your application emits:

Logs record that an event occurred (for example, a failed user log-in attempt).
Metrics record trends in event occurrences over time. For example, you can record the distribution of request latency to a particular endpoint, or monitor the capacity utilization of your system.
Traces record details of the lifecycle for a particular piece of code flow. For example, drill down into the trace of a specific request to understand possible causes of latency.
Exception reports show when an unexpected or "exceptional" error occurred. If your code encounters an exception, you can see the details of that exception, including the backtrace and full error message.

These telemetry signals become especially powerful when they can be correlated. For example, when viewing a specific exception report, you may want to gain more insight into the performance of your overall system at the time the exception occurred. In this case, the ability to view logs and/or traces from that same timeframe can provide you with the extra information you need to remedy bugs and improve overall application performance.

Monitoring Your Logs Using AppSignal

Correlating telemetry data can be a particularly hard problem to solve in the observability domain, especially when using different third-party services for things like logging, tracing, and exception reporting. AppSignal provides functionality for managing all of these types of telemetry and empowers you to correlate different types of telemetry data to get the most out of various signals. Let's take a look at an example now.

We'll use the "Errors" -> "Issue list" as the entry point for this exercise. I've selected an exception report from this list, displayed here:

We can correlate this exception report with some traces emitted by the live view that encountered this error. Note that I've configured our application to emit LiveView traces to AppSignal by following the instructions here. By clicking on "samples" and selecting an exception sample to examine, we can see the following exception details:

Now, in addition to the original exception report, we can see other useful information like the event that the live view was trying to handle when the exception occurred and the payload of that event, in addition to the full exception backtrace.

Finally, we can use the Time Detective feature to explore the full range of telemetry data, including traces, that our application emitted around the time of this exception. If we click on the "Time Detective" action here:

We'll see something like this:

The "Performance" view brings up some trace samples from the time period that we can drill down into. Here, we can see a list of traces that occurred around the time of the incident, including traces for the various LiveView lifecycle callbacks. From the details in the exception sample, we know this error occurred when an event handler in our live view tried to handle the "update" message. So, let's select the handle_event trace. We'll see something like this:

You can also select the "Logs" tab to view logs from this time period.

The Time Detective tool can be accessed via the logging UI as well. Just expand a given log line and select "Time Detective" from the bottom of the expanded view:

Then you'll have access to all of the telemetry data that can be correlated to your log statement based on timeframe. This could be especially useful if you see a high volume of a particular request being logged, or note an interesting error log. For any interesting or informative log statements, a deep dive into correlated telemetry data is just a click away.

Logging in Phoenix: A Few Final Tips

You're now ready to leverage logging as one part of the overall observability story in your Phoenix application. Just a few guiding rules to keep in mind when you do:

Prefer structured over unstructured logs so that you can query and analyze your application's logs with tools like AppSignal.
Suppress framework logs to the FATAL level and emit application logs at INFO.
Treat logs as just one part of your observability posture. Logs are most useful when correlated with other telemetry data, like exceptions and traces.

Wrap Up

We've covered a lot of ground in this post, so let's wrap up before you go.

We discussed the benefit of structured over unstructured logging, and configured our Phoenix application to use logfmt with the console logger backend. We then saw the benefits of structured logging in action in AppSignal.

Finally, we took our log analysis skills to the next level when we suppressed framework logs and explored how our meaningful application logs correlated with other telemetry data in AppSignal.

Now take on the world of Phoenix logging. Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Sanitize Strings in Elixir with Pattern Matching and Recursion

Sophie DeBenedetto — Tue, 08 Nov 2022 14:48:08 +0000

In this post, we'll use two of Elixir's most powerful features - pattern matching and recursion - to sanitize a string by removing invalid Unicode "Specials" characters.

Let's dive straight in!

The Problem: Strings with Unicode Specials

Unicode Specials is a short Unicode block of characters allocated at characters U+FFF0-FFFF. If one of these characters is present in a string of text, that text is not correctly encoded Unicode text. Here's the full set of those characters:

0xFFF0
0xFFF1
0xFFF2
0xFFF3
0xFFF4
0xFFF5
0xFFF6
0xFFF7
0xFFF8
0xFFF0
0xFFF10
0xFFF11
0xFFF12
0xFFF13
0xFFF14
0xFFFA
0xFFFB
0xFFFC
0xFFFD
0xFFFE
0xFFFF

You might encounter incorrectly encoded strings that contain a member of the Unicode Specials block. Let's say you need a way to process those strings anyway. Maybe you have some code that reads file contents or receives string input from another external source, like a form file upload or external API. In this case, you will want to remove these invalid characters to continue processing the text.

Elixir makes this easy with the help of binary pattern matching and recursion. We'll build out a function that removes Unicode specials from a string and replaces them with the "�" replacement character, so your program can continue to process the string input. When we're done, you'll have a handy function you can use to sanitize any string and a deeper understanding of two Elixir language features that will prove useful in many other scenarios.

Let's get started!

The Solution: Sanitize Your String with Pattern Matching and Recursion in Elixir

Our goal is to produce a function - CleanString.unicode_only/2 - that takes in any string and returns that string, with Unicode Specials replaced with the "�" replacement character. Here are a few examples of our finished product in action:

CleanString.unicode_only("abc" <> <<0xFFFF::16>>)
=> "abc�"
CleanString.unicode_only("a" <> <<0xFFFF::16>> <> "bc" <> <<0xFFFF::16>>)
=> "a�bc�"
CleanString.unicode_only("abc" <> <<0xFFF2::16>> <> "def")
=> "abc�def"
CleanString.unicode_only(<<0xFFF5::16>> <> "def")
=> "�def"
CleanString.unicode_only(<<0xFFF5::16>> <> <<0xFFF4::16>> <> "def")
=> "��def"

Step 1: Break Down the Problem

Before we write any code, let's break down this problem one step at a time:

For each character in a string
Check if that character is present in the @specials list
If it is, remove it from the string
And replace it with the replacement character
If it is not present in the @specials list
Keep that character

Changing elements in place in a string is a tricky and potentially expensive operation, but Elixir gives us another way.

We can use binary pattern matching to iterate over each element in our string, recursively breaking off the "head" or first character of the string. Then, depending on whether or not that character is considered valid, we will add it to the head of a new string.

Pulling the head off a string and adding to the head of a new string are very performant tasks in Elixir--it doesn't matter how long the string is, Elixir will only ever need to remove from or add to the beginning of that string.

Let's break down our problem one more time, this time with our new strategy in place:

For each character in a string
Check if that character is present in the @specials list
If it is, add a replacement character to the head of a new string
If it is not, add that character to the head of a new string

This will result in a brand new string that contains only replacement or valid Unicode characters.

With our plan in place, we're ready to write some code.

Step 2: Define the Module in Elixir

First up, we'll define a module, CleanString, that sets a module attribute, @specials, to a list of binary strings that represent the characters in the specials block listed above:

defmodule CleanString do
    @specials [
    <<255, 240>>,
    <<255, 241>>,
    <<255, 242>>,
    <<255, 243>>,
    <<255, 244>>,
    <<255, 245>>,
    <<255, 246>>,
    <<255, 247>>,
    <<255, 248>>,
    <<255, 240>>,
    <<255, 16>>,
    <<255, 17>>,
    <<255, 18>>,
    <<255, 19>>,
    <<255, 20>>,
    <<255, 250>>,
    <<255, 251>>,
    <<255, 252>>,
    <<255, 253>>
    <<255, 254>>,
    <<255, 255>>
  ]
end

This list of binary strings is generated by taking each specials character and evaluating its hexadecimal bitstring version like this:

iex> <<0xFFFF::16>>
<<255, 255>>

We want our module to store this list of binary strings so that we can check each character of the given string against this list and remove it if it's found.

Now that we have the basics of our module in place, let's start building the unicode_only/2 function.

Step 3: Define the `unicode_only/2` Function in Elixir

The unicode_only/2 function will take in two arguments: the original string and the new "clean" string we're building.

We plan to call our function recursively. The first time, we call it with only one argument--the argument of the original string--like this:

CleanString.unicode_only("abc" <> <<0xFFFF::16>>)
=> "abc�"

In this scenario, we need to default the "clean string" second argument to an empty string. Let's define that default argument function head first:

defmodule CleanString do

  @specials [
    # ...
  ]

  def unicode_only(string, new_string \\ "")
end

Great, now we're ready to define the remainder of our function heads. Here's where Elixir's powerful binary pattern matching feature comes in.

We'll define one version of the function that pattern matches a scenario where the character at the head of the string is included in the @specials list, and one in which it isn't. Let's start with that first scenario:

@replacement_character "�"

def unicode_only(<<head::binary-size(2)>> <> tail, new_string)
    when head in @specials do
  unicode_only(tail, @replacement_character <> new_string)
end

First, we've added a new module attribute, @replacement_character, that stores our replacement character. Then, we've added a version of the unicode_only/2 function that uses binary pattern matching to extract the first two bytes of the string and set them equal to a variable, head.

Why do we need to capture the first two bytes of the string to check if the first character is part of the Unicode Specials block? This is because each individual specials character is represented by a two-byte binary string.

Let's take a closer look at how we capture those two bytes now. The magic happens in the first argument given to unicode_only/2:

<<head::binary-size(2)>> <> tail

Elixir allows you to pattern match in the function head. This means that when unicode_only/2 is called, the first argument given will be pattern matched against this definition. For example, if we call the function like this:

original_string = "abc" <> <<0xFFFF::16>>
CleanString.unicode_only(original_string)

Then the following pattern match will be executed:

<<head::binary-size(2)>> <> tail = original_string

As mentioned above, this results in setting the variable head equal to the first two bytes of the string. Meanwhile, the tail variable is bound to the remainder of the string. Then, we use a guard clause to check if that variable is included in the @specials list.

If it is, we don't want to add it to the new clean string we're constructing. Instead, we want to add the replacement character:

@replacement_character <> new_string

Now we need to perform this same operation for the next character in the original string, and the next one, and so on. This is where recursion comes in. We will call unicode_only/2 again. This time with only the tail of the original string as a first argument, and the newly constructed clean string as a second argument:

unicode_only(tail, @replacement_character <> new_string)

In this way, we recursively pull the head off the string, until the string is empty. Now, we have a function head that runs when the first two bytes of the string are included in @specials.

Let's define another version of our function that will run when the first character isn't included there.

 def unicode_only(<<head::binary-size(1)>> <> tail, new_string) do
  new_string = head <> new_string
  unicode_only(tail, new_string)
end

Once again, we use binary pattern matching in the function head. This time, we pattern match the first argument given to unicode_only/2 against the following:

<<head::binary-size(1)>> <> tail

Here, we pull off the first byte of the string and set it equal to the head variable, and we set the remainder of the string equal to tail. We only want to grab the first byte of the string here because non-specials characters are represented by a single byte, for example:

iex> <<99>>
"c"

Moving our attention back to the function body, let's talk about how this function behaves. In the scenario where the first character is not included in @specials we do want to retain that character in our new string. So we add it to the head of the new string, like this:

head <> new_string

Then, we're ready to recursively call our function again, with the remaining tail of the original string and our updated new string:

unicode_only(tail, head <> new_string)

We're almost done. We just need a way to break out of our recursive loop when the time is right. When do we want to stop recursing? When we've processed every character in the original string. When this happens, the original string will be empty, and the first argument passed to unicode_only/2 in our recursive loop will be "". Let's implement that function head now, once again using pattern matching:

def unicode_only("", new_string), do: String.reverse(new_string)

When the original string is finally empty, we don't want to call unicode_only/2 again. Instead, we'll return the reversed new string.

We need to reverse the string because we added each processed character to the head of the new string to keep our code performant. So, we added each retained or replacement character in the opposite order from their placement in the original string. Reversing the string when we're done puts everything back in the right order.

Putting it all together, we end up with this:

defmodule CleanString do

  @specials [
    <<255, 240>>,
    <<255, 241>>,
    <<255, 242>>,
    <<255, 243>>,
    <<255, 244>>,
    <<255, 245>>,
    <<255, 246>>,
    <<255, 247>>,
    <<255, 248>>,
    <<255, 240>>,
    <<255, 16>>,
    <<255, 17>>,
    <<255, 18>>,
    <<255, 19>>,
    <<255, 20>>,
    <<255, 250>>,
    <<255, 251>>,
    <<255, 252>>,
    <<255, 254>>,
    <<255, 255>>
  ]

  @replacement_character "�"

  def unicode_only(string, new_string \\ "")

  def unicode_only(<<head::binary-size(2)>> <> tail, new_string)
      when head in @specials do
    unicode_only(tail, @replacement_character <> new_string)
  end

  def unicode_only(<<head::binary-size(1)>> <> tail, new_string) do
    unicode_only(tail, head <> new_string)
  end

  def unicode_only("", new_string), do: String.reverse(new_string)
end

Wrap Up

With a set of functions implemented in around ten lines of code, we've created an elegant solution for a common, tough problem. Thanks to Elixir's powerful pattern matching functionality, which we used here to pattern match binary strings in function heads, we have a handy function for cleaning bad characters out of any string.

String sanitization is something that you may need to do in the wild, and Elixir is a perfect fit for the job.
Beyond that specific scenario though, this solution demonstrates how nicely Elixir pattern matching and recursion go hand in hand.

Instead of implementing complex and difficult-to-read if statements, our recursive code flow was built entirely with pattern matching. Each scenario was mapped to a different unicode_only/2 function head.

Reach for function head pattern matching whenever you want to execute different function behavior based on different inputs, and you'll end up with clean, highly readable, and maintainable code. You can even pattern match on single characters, or groups of characters, in a binary string with the syntax you've seen here.

Taken together, pattern matching and recursion are a match made in Elixir heaven. Add these tools to your Elixir toolbox and take them out whenever you have a complex input processing problem to solve.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Phoenix LiveView 0.18: New Special HTML Attributes

Sophie DeBenedetto — Tue, 18 Oct 2022 11:39:24 +0000

Phoenix LiveView 0.18 just shipped, with lots of new goodies to make developing LiveView an even better experience.

In this post, I'll take you through a lesser-known new feature - LiveView's new special HTML attributes - and show you how to write cleaner HTML with :if, :for, and :let.

When we're done, you'll have an eloquent, ergonomic, and dynamic function component you can use to render a list anywhere in your LiveView app.

Let's dive in!

What are Special HTML Attributes in LiveView 0.18?

LiveView special HTML attributes are inspired by a Surface feature called "directives". Directives are built-in attributes that modify the translated code of a tag or component at compile time.

Directives are just one of a few new features, like component slots and declarative assigns, that LiveView has drawn from Surface. Surface is LiveView's independently developed, open-source component library, and it has often driven LiveView framework development forward by introducing (and battle-testing) new concepts quickly.

The :if and :for special attributes provide syntactic sugar for the <%= if ... do %> and <%= for ... do %> EEx calls you would normally expect to write in your templates.

The :let attribute works a little bit differently. It is used in component slots when you want to yield a variable from within the function component back up to the caller. You'll see this most often when you use the .form/1 function component, but you can also use it to make your own function components even more dynamic.

Let's take a look at how you can use these attributes to write eloquent, ergonomic HTML in your LiveView templates. Along the way, you'll combine the usage of all three of these attributes to build a clean and dynamic LiveView function component.

Cleaner LiveView Templates with `:if` and `:for`

You can use these two attributes in regular Phoenix templates, in components, and in slots to write cleaner HTML code. We'll take a look at an example using the :if directive first:

<div class="alert alert-danger" :if={@error_message}>
  <p><%= @error_message %></p>
</div>

The div with the alert alert-danger class that contains the error message markup will only render if the @error_message is present and evaluates to true.

This approach replaces the more verbose one here:

<%= if @error_message do %>
<div class="alert alert-danger">
  <p><%= @error_message %></p>
</div>
<% end %>

The code that uses the :if attribute is easier to read--more eloquent--and easier to write--more ergonomic.

You can also use the :if attribute to conditionally render a function component, all in one simple line of code. Let's say we have a function component, .error/1, that wraps up the error message markup above. We can conditionally render it like this:

<.error message={@error_message} :if={@error_message}>

Now, let's take a look at an example that uses the :for attribute. The :for attribute can be used to iteratively render some content for each member in a collection, like this:

<tbody id="books">
  <tr :for={book <- @books}>
    <td><%= book.title %></td>
  </tr>
</tbody>

Thanks to the :for attribute, you don't have to establish your for loop explicitly within EEx tags, like this:

<tbody id="books">
  <%= for book <- @books %>
  <tr>
    <td><%= book.title %></td>
  </tr>
  <% end %>
</tbody>

Once again, we're left with code that is both more eloquent and more ergonomic. You can even combine the usage of :if and :for if you need to. Let's say we have a list of books to render, but we only want to display a given book if it is available in our online store. We can achieve that like this:

<ul>
  <li :for={book <- @books} :if={book.available}><%= book.title %></li>
</ul>

Since both :if and :for can be used in either plain HTML or in LiveView function components, we can wrap our list item HTML markup in a function component. Given the following function component:

def list_item(assigns) do
  ~H"""
  <li>
    <%= @book.title %>
  </li>
  """
end

We can call on it like this:

<ul>
  <.list_item :for={book <- @books} :if={book.available} />
</ul>

This is a nice way to start compartmentalizing our markup into reusable function components made even cleaner with special HTML attributes.

We can do better with our function component, though. Right now, we've only encapsulated the markup for list items, not the entire list. On top of that, our .list_item/1 function component isn't very reusable--it expects to be called with an assigns that contains an @book attribute set to something that responds to .title.

We'll build a more dynamic function component that uses component slots and the :let attribute to encapsulate the entire unordered list into one dynamic component. When we're done, you'll have a dynamic function component that uses :for and :let to render any collection, anywhere in your LiveView app.

For this next example, we'll simplify our logic a bit by dropping the :if attribute and the requirement that a book should only render if it's available. We'll just focus on using :let and :for. Let's get started.

Dynamic Function Components with `:let`

First up, let's use a named component slot to create a function component that renders an unordered list and its list items. Start by defining a top-level .unordered_list/1 function component that renders a named slot, :list_item, inside the appropriate markup:

def unordered_list(assigns) do
  ~H"""
  <ul>
    <li>
      <%= render_slot(@list_item, ...) %>
    </li>
  </ul>
  """
end

We're on track to render our unordered list component like this:

<.unordered_list>
  # ...
</unordered_list>

When we call on the component, we'll set an assigns - items - equal to the list of books. Now, we can use :for in the function component HEEx to render a list item slot for each item in the @items assigns:

def unordered_list(assigns) do
  ~H"""
  <ul :for={item <- @items}>
    <li>
      <%= render_slot(@list_item, item) %>
    </li>
  </ul>
  """
end

Putting it all together, we can call on our function component like this:

<.unordered_list items={@books}>
  <:list_item :let={book}>
    <%= book.title>
  </:list_item>
</.unordered_list>

Here's where the :let special attribute comes in. In our function component, we are using :for to iterate over the list of books in the @items assignment. For each book in the list, bound to the item variable at each step in the iteration, we are calling render_slot/2 with a second argument of item.

The call to :let={book} when we call on our :list_item slot sets a variable - book - equal to the second argument passed into render_slot/2. In this way, you yield variables from your function components back to the caller with the :let attribute. So, where we call :list_item when rendering the function component, we can operate on the book variable to display its title.

By combining :for and :let, we end up with an eloquent, ergonomic, and highly dynamic function component that we can use again and again in our application to render any collection into an unordered list.

Before we wrap up, there's one limitation that I'd like to point out here. Bringing back our "only render a book title if the book is available" logic is a little tricky here. You might want to do something like this:

<.unordered_list items={@books}>
  <:list_item :let={book} :if={book.available}>
    <%= book.title>
  </:list_item>
</.unordered_list>

This is not possible, however. The component slot call cannot combine :let with :if in that manner. We can't take advantage of the :if attribute here and instead have to write something like this:

def unordered_list(assigns) do
  ~H"""
  <ul :for={item <- @items}>
    <%= if item.available %>
      <li>
        <%= render_slot(@list_item, item) %>
      </li>
    <% end %>
  </ul>
  """
end

The downside is that the component becomes aware of the need to call .available on the item, making it less reusable.

Wrap Up

LiveView is being adopted so quickly in the Elixir community (and beyond) partly because of its gentle learning curve and emphasis on developer happiness.

LiveView's new special HTML attributes make writing LiveView templates even easier and more fun. By borrowing ergonomic features from Surface, we end up with code that is painless to write (fewer onerous EEx tags!) and easy to understand and maintain.

Reach for these special HTML attributes when writing plain HEEx templates, LiveView HEEx templates, or function components. You'll end up with beautiful code.

Until next time, happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Fix Process Bottlenecks with Elixir 1.14's Partition Supervisor

Sophie DeBenedetto — Tue, 27 Sep 2022 11:18:57 +0000

Elixir v1.14 shipped earlier this month with a bunch of new goodies.

In this post, we'll explore Elixir's new PartitionSupervisor. We'll take a look at some code that suffers from the exact bottleneck issue that partitions supervisors are designed to solve. Then, we'll fix that bottleneck. Along the way, you'll learn how partition supervisors work under the hood to prevent process bottlenecks.

Let's get started!

The Problem: Dynamic Supervisor Bottlenecks in Your Elixir App

You may be familiar with using dynamic supervisors to start child processes on-demand in your application. Telling the dynamic supervisor to start its children can become a bottleneck, however.

With just one dynamic supervisor, concurrent processes each have to wait their turn to tell your app to start up children. If the "start up child" process takes a long time, or your app handles a very high volume of "tell the dynamic supervisor to start children" requests, you've got a bottleneck. The single dynamic supervisor can only initialize one child process at a time.

Let's take a look at a simple example that artificially replicates such a bottleneck.

The Slow Worker and Its Dynamic Supervisor

In my sample app here, I have a worker - SlowGreeter.Worker. The worker module is a genserver that initializes by sleeping for five seconds and then printing out a greeting:

defmodule SlowGreeter.Worker do
  use GenServer

  def start_link(%{name: name, from: from}) do
    GenServer.start_link(__MODULE__, %{name: name, from: from})
  end

  def init(state) do
    :timer.sleep(5000)
    send(self(), :greet)
    {:ok, state}
  end

  def handle_info(:greet, %{name: name, from: from}) do
    IO.puts("Hello from #{from}, #{name}")
    {:noreply, name}
  end
end

Disclaimer: This is meant to replicate a genserver with some time-consuming work to do on initialization. It should be noted that a "slow-starting" genserver is something of a code smell. You should avoid doing time-consuming work in the #init/1 function and leverage handle_continue/2 to prevent long-running tasks from blocking genserver start up.

But, sometimes doing such work when the genserver starts up is unavoidable. And regardless, this setup helps us replicate a bottleneck scenario that can happen:

1. When your dynamic supervisor is either starting up a slow-to-initialize worker, or

2. If your app is handles many concurrent requests telling the same dynamic supervisor to start up its children.

Okay, with that disclaimer out of the way, let's proceed with illustrating our bottleneck. We have our slow-to-initialize worker, and we also have a SlowGreeter.DynamicSupervisor module that starts it up:

defmodule SlowGreeter.DynamicSupervisor do
  use DynamicSupervisor
  alias SlowGreeter.Worker

  def start_link(init_arg) do
    DynamicSupervisor.start_link(__MODULE__, init_arg, name: __MODULE__)
  end

  @impl true
  def init(_init_arg) do
    DynamicSupervisor.init(strategy: :one_for_one)
  end

  def start_greeter(name) do
    spec = %{id: Worker, start: {Worker, :start_link, [%{name: name, from: "DynamicSupervisor"}]}}
    DynamicSupervisor.start_child(__MODULE__, spec)
  end
end

The SlowGreeter.DynamicSupervisor starts when the application starts up. Then, we can tell it to start the greeter worker like this:

SlowGreeter.DynamicSupervisor.start_greeter("Frankenstein")
# sleep for 5 seconds...
"Hello from DynamicSupervisor, Frankenstein"

Now we're ready to demonstrate our bottleneck.

Demo: The Bottleneck

Let's spawn five processes, each of which will make a call to SlowGreeter.DynamicSupervisor.start_greeter/1. Since each of our spawned processes requests the same dynamic supervisor, we'll see that each process must wait its turn.

for _i <- 1..5 do
  name = Faker.Person.first_name()
  # The DynamicSupervisor starts a worker that sleeps for 5 seconds then puts a greeting
  spawn(fn -> SlowGreeter.DynamicSupervisor.start_greeter(name) end)
end

When the SlowGreeter.DynamicSupervisor starts up its child SlowGreeter.Worker, it must wait for the worker to initialize. The worker initialization process is slow--we told it to sleep for five seconds and then print out a greeting. We'll see that each spawned process's call to the dynamic supervisor is only processed when the preceding one finishes. As a result, each greeting prints out at a five-second interval, as you can see in this video.

This approach is bottlenecked by the dynamic supervisor itself. It can only process one "start up child" request at a time, and must wait until the initialization of a child worker is complete before moving on to the next one.

We can remove this bottleneck with the help of Elixir 1.14's new PartitionSupervisor. Let's implement it now. Then, we'll take a deeper dive into how it works.

The Fix: Supervise a Fleet of Dynamic Supervisors with a Partition Supervisor

Instead of initializing just one dynamic supervisor when our application starts up, we'll use the new PartitionSupervisor to spin up a set of dynamic supervisors.

Then, instead of making direct calls to the dynamic supervisor to start its child worker, we will go through the partition supervisor, which will route the request to one of the many dynamic supervisors it's overseeing. Let's take a look at the code.

Using `PartitionSupervisor` in the Dynamic Supervisor Module

We'll implement a new module, SlowGreeter.DynamicSupervisorWithPartition, which uses the DynamicSupervisor behaviour. This time, however, the #start_greeter/1 function will call on the dynamic supervisor to start its child via the PartitionSupervisor.

defmodule SlowGreeter.DynamicSupervisorWithPartition do
  use DynamicSupervisor
  alias SlowGreeter.Worker

  def start_link(init_arg) do
    DynamicSupervisor.start_link(__MODULE__, init_arg, name: __MODULE__)
  end

  @impl true
  def init(_init_arg) do
    DynamicSupervisor.init(strategy: :one_for_one)
  end

  def start_greeter(name) do
    spec = %{id: Worker, start: {Worker, :start_link, [%{name: name, from: "PartitionSupervisor"}]}}

    DynamicSupervisor.start_child(
      {:via, PartitionSupervisor, {__MODULE__, self()}},
      spec
    )
  end
end

Here's where the magic happens:

DynamicSupervisor.start_child(
  {:via, PartitionSupervisor, {__MODULE__, self()}},
  spec
)

Instead of calling on the SlowGreeter.DynamicSupervisorWithPartition directly, we call it through the partition supervisor using the {:via, PartitionSupervisor, {dynamic_supervisor_name, routing_key}} tuple.

This will route the "start child request" to one of the running SlowGreeter.DynamicSupervisorWithPartition processes supervised by the partition supervisor that starts up with our application. Let's configure our app to start such a supervisor now:

# application.ex
def start(_type, _args) do
  children = [
      {PartitionSupervisor, child_spec: DynamicSupervisor, name: SlowGreeter.DynamicSupervisorWithPartition},
    ]

    opts = [strategy: :one_for_one, name: SlowGreeter.Supervisor]
    Supervisor.start_link(children, opts)
end

With this, when the app starts, a partition supervisor will start up and begin supervising a set of SlowGreeter.DynamicSupervisorWithPartition dynamic supervisors. The default behavior is to create a partition for each available scheduler (usually one per core of your machine). It will then place a dynamic supervisor on each partition.

With our code in place, let's see a demo of our bottleneck fix.

Demo: No More Bottleneck

Once again, we'll spawn five processes. Each process will call on the SlowGreeter.DynamicSupervisorWithPartition#start_greeter/1 function, like this:

for _i <- 1..5 do
  name = Faker.Person.first_name()
  # The PartitionSupervisor tells one of its dynamic supervisors to start a worker that sleeps for 5 seconds then puts a greeting
  spawn(fn -> SlowGreeter.DynamicSupervisorWithPartition.start_greeter(name) end)
end

This function goes through the partition supervisor to route a "start child worker" request to dynamic supervisors across partitions. This means we don't have to wait for a single dynamic supervisor to finish initializing a child worker before moving on to process the next request from a spawned process.

Instead, the partition supervisor will route these requests so that they're processed more or less concurrently by the dynamic supervisors spread across partitions. So, we'll see that all five greetings are printed out simultaneously after just one five-second sleep. You can see this behavior in this video.

With that, we've fixed our bottleneck! Remember that our worker's "sleep for five seconds on init" behavior is artificial, and you'll want to avoid lengthy worker startups in general. But, this example serves to illustrate the kind of bottleneck that can occur in your application if you deal with lots of concurrent requests to the same dynamic supervisor.

Elixir 1.14's new partition supervisor neatly solves this problem by partitioning your dynamic supervisors and distributing requests to them, so that they can handle a greater load.

Where you don't need to share state between child processes supervised by dynamic supervisors, this approach can help your dynamic supervisors handle scale and avoid bottlenecks.

Now that we've seen partition supervisors in action, let's take a closer look at how they work.

`PartitionSupervisor` in Elixir 1.14: Under the Hood

Keep reading for a deeper dive into how partition supervisors establish partitions and route requests to dynamic supervisors.

Partitioning Dynamic Supervisors

As we saw earlier, you can tell your app to start a partition supervisor, overseeing some dynamic supervisors, in your application.ex's #start/2 function like this:

def start(_type, _args) do
  children = [
    {PartitionSupervisor, child_spec: DynamicSupervisor, name: SlowGreeter.DynamicSupervisorWithPartition},
  ]

  opts = [strategy: :one_for_one, name: SlowGreeter.Supervisor]
  Supervisor.start_link(children, opts)
end

Here, we start a partition supervisor that starts a SlowGreeter.DynamicSupervisorWithPartition for each core in our machine. This is the default start_link behavior.

PartitionSupervisor#start_link/1 supports options that include a :partitions key. The :partitions key should be set to a positive integer representing the number of partitions and defaults to System.schedulers_online(), which should return the number of cores on your machine.

Under the hood, a partition supervisor is just a regular supervisor that generates a child spec for each partition and then stores the partition info in ETS or a Registry. Later, when you tell a dynamic supervisor to start its child via the partition supervisor, it selects a partition based on the routing algorithm and routes the call to a dynamic supervisor in that partition. So, it effectively spins up a set of dynamic supervisors when your app comes up, and then starts child processes across those dynamic supervisors.

Let's take a closer look at the routing behavior now.

Routing Requests to Dynamic Supervisors

When you're ready to start a dynamic supervisor's child through a partition supervisor, you do so like this:

DynamicSupervisor.start_child(
  {:via, PartitionSupervisor, {dynamic_supervisor_name, routing_key}},
  spec
)

The :via tuple specifies the name of the dynamic supervisor to start up and a routing key.

In our example app, we used self() as a routing key, which will return the PID of the current process, but you can also provide a positive integer here. If key is an integer, it is routed using rem(abs(key), num_partitions). Otherwise, the routing is performed by calling :erlang.phash2(key, num_partitions). So, when you use the same PID as a routing key, you can be assured that your call will be directed to the same partition every time.

Wrap Up

We've seen how you can use partition supervisors to avoid bottlenecks in your dynamic supervisor code. Provided the state of your child processes can be partitioned (i.e., you don't need to share state among them), you can reach for a partition supervisor to elegantly scale up your dynamic supervisors.

In fact, partition supervisors can be used to partition and scale any process in your Elixir application. If a process can be started up, and have its state partitioned, then you can use a partition supervisor. Start the partition supervisor with its child spec (just like we did for our dynamic supervisor) and dispatch messages to the partition supervisor's children using the same :via tuple we demonstrated.

This is just one of the many exciting new features of Elixir v1.14. Read about another new feature, debugging with dbg()!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Elixir 1.14: Better Debugging with dbg/2 and More

Sophie DeBenedetto — Wed, 21 Sep 2022 11:52:49 +0000

The latest Elixir release introduces new features to improve your developer and debugging experience.

In this post, we'll take a look at the new dbg() functionality, along with some improvements to Inspect and binary evaluation error messaging. All these changes come together to make you an even more productive Elixirist.

Let's get started!

Debugging with `Kernel.dbg/2` in Elixir

The Elixir Kernel now exposes a dbg/2 macro that you can use with or without IEx for enhanced debugging. Let's walk through a few examples that illustrate just how useful this tool can be in your debugging toolkit.

In this example, we have a Phoenix LiveView application called Live Library. Users can browse, checkout, and return books. Unfortunately for us, however, it looks like we have a bug--when a user tries to check out a book, the live view crashes.

We can use the dbg/2 function to gain insight into what's going wrong. We'll start with a few well-placed calls to dbg/2. Then, we'll move on to leveraging dbg/2 within an IEx session.

Debugging the Current Binding

First, we'll make a call to dbg/2 in the live view's event handler function that gets called when the user clicks "checkout". This is the entry point of our "checkout book" code flow, so we'll start our debugging process there.

# lib/live_library_web/live/book_live/checkout_component.ex

def handle_event("checkout", _, %{assigns: %{book: book}} = socket) do
  dbg() # this is new!
  case Library.checkout_book(book, socket.assigns.current_user.id) do
    {:ok, book} ->
      {:noreply, socket |> assign(:book, book)}

    {:err, _err} ->
      {:noreply, socket}
  end
end

Now, if we try once again to check out a book, we'll see the following output in our terminal where we're running the Phoenix server via mix phx.server:

[(live_library 0.1.0) lib/live_library_web/live/book_live/checkout_component.ex:33: LiveLibraryWeb.CheckoutComponent.handle_event/3]
binding() #=> [
  book: %LiveLibrary.Library.Book{
    updated_at: ~N[2022-09-11 17:42:19],
    inserted_at: ~N[2022-02-28 16:33:45],
    book_loans: [
      %LiveLibrary.Library.BookLoan{
        updated_at: ~N[2022-09-11 17:39:18],
        inserted_at: ~N[2022-09-11 17:39:18],
        book_id: 7,
        user_id: 4,
        id: 51,
        __meta__: #Ecto.Schema.Metadata<:loaded, "book_loans">
      }
    ],
    author_id: 4,
    title: "My Test Book!!!",
    description: "A book I made up. Different from the book I wrote which I also technically made up.",
    cover: "live_view_upload-1657544569-150520820351856-7",
    available: true,
    id: 7,
    __meta__: #Ecto.Schema.Metadata<:loaded, "books">
  },
  socket: #Phoenix.LiveView.Socket<
    id: "phx-FxPfPGQDjqGwUAyB",
    endpoint: LiveLibraryWeb.Endpoint,
    view: LiveLibraryWeb.BookLive.Index,
    parent_pid: nil,
    root_pid: #PID<0.780.0>,
    router: LiveLibraryWeb.Router,
    assigns: %{
      __changed__: %{},
      book: %LiveLibrary.Library.Book{
        updated_at: ~N[2022-09-11 17:42:19],
        inserted_at: ~N[2022-02-28 16:33:45],
        # ...
      },
      current_user: #LiveLibrary.Accounts.User<
        updated_at: ~N[2022-02-18 14:49:02],
        inserted_at: ~N[2022-02-18 14:49:02],
        admin: true,
        confirmed_at: nil,
        email: "admin@email.com",
        id: 4,
        __meta__: #Ecto.Schema.Metadata<:loaded, "users">,
        ...
      >,
      flash: %{},
      id: "checkout-7",
      myself: %Phoenix.LiveComponent.CID{cid: 5}
    },
    transport_pid: #PID<0.768.0>,
    # ...
  >
]

When called without any arguments, dbg/2 defaults to debugging the binding for the current context. In this case, that's the handle_event/3 function (and the variables available to us by the time the dbg/2 call is hit on the first line of that function).

As you can see in the output above, calling dbg/2 does a few things for us - printing out the:

Location of the debugged code:

[(live_library 0.1.0) lib/live_library_web/live/book_live/checkout_component.ex:33: LiveLibraryWeb.CheckoutComponent.handle_event/3]

Value of the current context's binding:

binding() #=> [
  book: %LiveLibrary.Library.Book{
    #...

The call to dbg/2 also returns the value of the debugged code itself. We'll see how helpful this is in a bit.

For now, we're already experiencing some of the benefits of dbg/2 with just this basic usage. Let's examine the value of the arguments that handle_event/3 is called with. This way, we'll better understand how our code is executing and what might be going wrong.

Based on the dbg/2 output, it looks like the input to handle_event/3 is exactly what we expect. We have a socket whose assigns correctly contain a book and a current user. So, let's move our call to dbg/2 downstream a bit, into the Library.checkout_book/2 context function.

Debugging Provided Code

At this point in our debugging journey, we're ready to turn our attention to the behavior of the Library.checkout_book/2 function. This is the next step in our broken "checkout book" code flow.

We're interested in a few things. We'd like to understand what exact arguments the function is being called with, and what it returns. We can easily reveal this information with two calls to debug/2.

We'll call dbg/2 with the current default context at the beginning of the checkout_book/2 function to look at the arguments' values.

Then, we'll pipe the code in Library.checkout_book/2 into a call to dbg/2 at the end of the function. This will allow us to see the return value of the function. And, since dbg/2 returns the value of the executed code you pass to it as a first argument, the remainder of our "checkout book" code flow will be unaffected by this call to dbg/2.

Here's what our debugged function looks like:

def checkout_book(book, user_id) do
  dbg()
  with {:ok, _loan} <- Library.create_book_loan(%{user_id: user_id, book_id: book.id}),
       {:ok, book} <- Library.update_book(book, %{available: false}) do
        book
  else
    err -> err
  end
  |> dbg()
end

Now if we try to check out a book, we'll see the following output in our terminal:

[(live_library 0.1.0) lib/live_library/library.ex:13: LiveLibrary.Library.checkout_book/2]
binding() #=> [
  book: %LiveLibrary.Library.Book{
    updated_at: ~N[2022-09-11 18:27:59],
    inserted_at: ~N[2022-02-28 16:33:45],
    author_id: 4,
    title: "My Test Book!!!",
    description: "A book I made up. Different from the book I wrote which I also technically made up.",
    cover: "live_view_upload-1657544569-150520820351856-7",
    available: true,
    id: 7,
    __meta__: #Ecto.Schema.Metadata<:loaded, "books">,
    # ...
  },
  user_id: 4
]

# ...

[(live_library 0.1.0) lib/live_library/library.ex:21: LiveLibrary.Library.checkout_book/2]
with {:ok, _loan} <- Library.create_book_loan(%{user_id: user_id, book_id: book.id}),
     {:ok, book} <- Library.update_book(book, %{available: false}) do
  book
else
  err -> err
end #=>
%LiveLibrary.Library.Book{
  updated_at: ~N[2022-09-11 18:28:00],
  inserted_at: ~N[2022-02-28 16:33:45],
  # ...
}

From this, we can clearly see that the checkout_book/2 function returns a single book struct. We're close to solving our bug now.

Let's return to the handle_event/3 function that calls checkout_book/2. This time, we'll run our server in an IEx session with iex -S mix phx.server. Then, we'll place another dbg/2 call in the handle_event/3 function to freeze code execution and interact with our code in real time.

Debugging with `dbg/2` and IEx in Elixir

The dbg/2 function is configurable, and you can extend it with advanced behavior. IEx comes with one such extension baked in.

In Elixir 1.14, IEx extends dbg/2 with an interactive shell, similar to IEx.pry.

Let's add back our dbg/2 call to the event handler function and play around:

# lib/live_library_web/live/book_live/checkout_component.ex

def handle_event("checkout", _, %{assigns: %{book: book}} = socket) do
  dbg()
  case Library.checkout_book(book, socket.assigns.current_user.id) do
    {:ok, book} ->
      {:noreply, socket |> assign(:book, book)}

    {:err, _err} ->
      {:noreply, socket}
  end
end

Now when we try to check out a book in the browser, we'll see that we have a request to pry in our terminal:

Request to pry #PID<0.816.0> at LiveLibraryWeb.CheckoutComponent.handle_event/3 (lib/live_library_web/live/book_live/checkout_component.ex:33)

   32:   def handle_event("checkout", _, %{assigns: %{book: book}} = socket) do
   33:     dbg()
   34:     case Library.checkout_book(book, socket.assigns.current_user.id) do
   35:       {:ok, book} ->
   36:         Endpoint.broadcast(@checkout_topic, "checkout_event", %{book: book})

Allow? [Yn] Y

We can now interact with and execute our code. Let's manually execute the checkout_book/2 function like this:

pry(1)> Library.checkout_book(book, socket.assigns.current_user.id)
iex(1)> [debug] QUERY OK db=1.0ms idle=1240.2ms
INSERT INTO "book_loans" ("book_id","user_id","inserted_at","updated_at") VALUES ($1,$2,$3,$4) RETURNING "id" [7, 4, ~N[2022-09-11 18:46:06], ~N[2022-09-11 18:46:06]]
[debug] QUERY OK db=1.0ms queue=0.3ms idle=1241.5ms
UPDATE "books" SET "available" = $1, "updated_at" = $2 WHERE "id" = $3 [false, ~N[2022-09-11 18:46:06], 7]
%LiveLibrary.Library.Book{
  updated_at: ~N[2022-09-11 18:46:06],
  inserted_at: ~N[2022-02-28 16:33:45],
  # ...
  ],

Here, we can see that we are returning the newly checked-out book. A call to whereami will show us the next bit of code flow that will try to execute:

pry(2)> whereami
Location: lib/live_library_web/live/book_live/checkout_component.ex:33

   31:
   32:   def handle_event("checkout", _, %{assigns: %{book: book}} = socket) do
   33:     dbg()
   34:     case Library.checkout_book(book, socket.assigns.current_user.id) do
   35:       {:ok, book} ->

    (live_library 0.1.0) lib/live_library_web/live/book_live/checkout_component.ex:33: LiveLibraryWeb.CheckoutComponent.handle_event/3

This reminds us that the case statement expects to match on a return value not of a single book struct, but rather of an {:ok, book} tuple. With that, we've found our bug! By updating the checkout_book/2 function to return a tuple instead of a book struct, we'll fix our problem.

Before we move on to some other new Elixir 1.14 features that will improve your development and debugging experience, let's take a last look at some additional dbg/2 functionality.

More `dbg/2` Goodies

Let's examine how dbg/2 makes it easier than ever before to debug pipelines. Then, we'll peek at some of the more advanced dbg/2 configuration capabilities.

Debugging Pipelines with `dbg/2`

The dbg/2 macro makes it easier to debug pipelines. Before dbg/2, if you wanted to examine return values at each step of your pipeline, you'd have to add an explicit call to IO.inspect at each step - like this:

# lib/live_library_web/live/book_live/index.ex
def mount(_params, %{"user_token" => user_token}, socket) do
  {:ok,
   socket
   |> assign_user(user_token)
   |> IO.inspect()
   |> assign_books()
   |> IO.inspect()
   |> assign_search()
   |> IO.inspect()
   |> assign_search_changeset()
   |> IO.inspect()
   |> assign_categories()
   |> IO.inspect()
   |> assign_category_ids()}
end

This isn't very eloquent and it can be tedious to implement. Now in Elixir 1.14, we can place just one call to dgb/2 at the end of our pipeline. This will print out the value at each individual step of the pipeline and ensure that the pipeline still returns the value of the executed code. Let's add in our dbg/2 call here:

def mount(_params, %{"user_token" => user_token}, socket) do
  {:ok,
   socket
   |> assign_user(user_token) # %{assigns: %{user: #...}}
   |> assign_books() # %{assigns: %{books: #...}}
   |> assign_search() # %{assigns: %{search: #...}}
   |> assign_search_changeset() # # %{assigns: %{search_changeset: #...}}
   |> assign_categories() # %{assigns: %{categories: #...}}
   |> assign_category_ids() # %{assigns: %{category_ids: #...}}
   |> dbg()}
end

Now, when we visit the /books index page that brings up this live view, we'll see that the return value of each step of the pipeline has been printed for us to examine.

Additionally, if you run this code in an IEx session, you will be able to step, or pry, through each individual step in the pipeline. Code execution will freeze at each successive line in the pipeline.

Pipelines represent one of Elixir's most powerful and eloquent language features, and debugging them just got easier.

Configuring `dbg/2`

You can extend the behavior of dbg/2 in your Elixir application by defining a custom function. Tell dbg/2 to use that custom function via your app's :dbg_callback configuration key. Your app configuration should look like this:

# config/config.exs
config :elixir, :dbg_callback, {MyMod, :debug_fun, [:stdio]}

Where you've defined a function MyMod.debug_fun/4.

Then when dbg/2 is called, your custom function will be called with three arguments prepended to any arguments you specify in your app config:

An AST representing the code to be debugged that was passed to dbg/2 as a first argument.
An AST representing any options given to dbg/2 as a second argument.
The Macro.Env struct representing the context in which dbg/2 was invoked.

You can find a more detailed example of this dbg/2 custom configuration approach in the docs.

Now that we've taken a pretty comprehensive look at what dbg/2 has to offer, let's move on to some other new Elixir 1.14 features that improve the development experience.

Inspect Improvements

Elixir 1.14 has improved the Inspect protocol implementation for a few core library modules: MapSet, Version.Requirement, and Date.Range. Previously, calls to inspect a new MapSet would return notation that is not valid Elixir:

iex> MapSet.new([1, :two, {"three"}])
#MapSet<[1, :two, {"three"}]>

This output can't be copied and pasted in IEx, or operated on by subsequent function calls.

This is no longer the case in Elixir 1.14. Now, you'll see this behavior:

iex> MapSet.new([1, :two, {"three"}])
MapSet.new([1, :two, {"three"}])

Here, the output is valid Elixir that can be operated on by subsequent Elixir code.

Some additional improvements have been made to Inspect protocol for structs:

When you inspect a struct, fields will be displayed in the order in which they are defined in defstruct.
If you want to customize the Inspect behavior for your struct by deriving it, you can use the new :optional keyword to omit struct fields that have not changed from their default value when displaying the inspection results. This simplifies struct representation and adds some much-needed brevity to the representation of larger structs. You can learn more info about the Inspect customization option in the docs.

We have just one more Elixir 1.14 improvement to look at before we wrap up.

Better Binary Evaluation Errors

In keeping with the improved debugging and developer experience we see throughout the Elixir 1.14 release, the latest version of Elixir also improves error messaging on binary construction and evaluation. This brings Elixir in line with Erlang/OTP 25 improvements in the same area.

Instead of getting generic ArgumentError messages when constructing binaries in an invalid manner, we get more informative error messages like this:

iex> 123 <> "456"
** (ArgumentError) expected binary argument in <> operator but got: 123

Want to learn how you can also save time debugging an app in production with AppSignal? Check out how Upside does it.

Wrap Up

Elixir has a devoted following and an increasingly growing community and adoption rate, in part because the language prioritizes developer happiness.

This latest Elixir release is no exception. With new debugging capabilities and improvements to Inspect protocols as well as binary evaluation error messages, Elixir's developer experience toolkit has grown even more.

Thanks to these new features, you'll be more productive in Elixir than ever before. Upgrade your Elixir apps today, and get going!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Phoenix LiveView Under The Hood: The Form Function Component

Sophie DeBenedetto — Tue, 05 Apr 2022 11:59:40 +0000

Thanks to HEEx and function components, LiveView provides developers with a sleek, ergonomic syntax for building and maintaining sophisticated interactive UIs. LiveView's form/1 function component is a great example of this, making it easier than ever before to render complex forms within LiveView.

However, the form/1 function component can feel a little mysterious to anyone unfamiliar with LiveView's function components.

In this post, we'll look under the hood of the form/1 function component. We'll dive into the Phoenix Component functionality that underpins this function and explore features like component slots. When we're done, you'll know exactly how the form/1 function renders your forms and you'll have a deeper understanding of LiveView components, setting you up to build your own components in the future.

What Are Function Components in Phoenix LiveView?

Before we dive into the form/1 function component, let's discuss LiveView's function components at a high level. Function components are defined in modules that use the Phoenix.Component behavior. Any function that takes in an argument of assigns and returns some markup wrapped in a HEEx template is a function component.

Let's take a look at a simple example. We'll define a UserDetails module that implements a function component, contact_info.

defmodule MyAppLive.UserDetails do
  use Phoenix.Component

  def contact_info(assigns) do
    ~H"""
    <div class="contact-info">
      <p><strong>Phone:</strong><%= @user.phone_number %></strong></p>
      <p><strong>Email:</strong><%= @user.email %></strong></p>
    </div>
    """
  end
end

Now, we can call on this component with any HEEx template like this (assuming you have an available @current_user assigns):

<UserDetails.contact_info user="{@current_user}" />

If you import the UserDetails module or if you're calling on this function component somewhere where the function is defined locally (i.e., elsewhere in the UserDetails module), you can leave off the module name from the function call:

<.contact_info user={@current_user}>

With this basic understanding of function components under your belt, we're ready to dive into the .form/1 function component that LiveView makes available to you. We'll take a look at some more advanced features of function components along the way.

Calling the `form/1` Function Component

LiveView implements a function component, form/1. This function component is defined in Phoenix.LiveView.Helper and imported into all of your live views. We'll start with the entry point of the form/1 code flow—the caller.

In this example, we'll construct a form for a book record that has a title, description, and author, like this:

<.form
    let={f}
    for={@changeset}
    id="book-form"
    phx-change="validate"
    phx-submit="save">

  <%= label f, :title %>
  <%= text_input f, :title %>
  <%= error_tag f, :title %>

  <%= label f, :description %>
  <%= textarea f, :description %>
  <%= error_tag f, :description %>

  <%= label f, :author %>
  <%= select f, :author_id, Enum.map(@authors, &{&1.full_name, &1.id}) %>
  <%= error_tag f, :author_id %>

  <div>
    <%= submit "Save", phx_disable_with: "Saving..." %>
  </div>
</.form>

Already, we can see that the form/1 function component offers a clean and easy-to-read syntax for describing forms. Let's break down the function invocation here. Then, we'll trace what's happening under the hood.

The form/1 function component, like all function components, is called with one argument of the assigns. Some of the attributes passed as part of the assigns probably look familiar from working with Phoenix.HTML.form_for/4. We pass in the changeset, an ID, and some LiveView form bindings. In fact, we can optionally pass in any of the same options you would give to Phoenix.HTML.form_for/4, and the form/3 function component will use those options in the same way.

In addition to the form options we're giving to assigns, we also use the let assigns to tell the function component to give a value back to the caller. More on this in a bit.

Lastly, you'll notice that we're actually calling the form/1 function component with an opening and closing <.form> tag, and we've enclosed the content of our form within those tags. This eloquent, declarative syntax is made possible by the Phoenix Component's render_slot/2 functionality. We'll see that in action later on in this post.

Now that we've seen how to call on the form/1 function component, let's dive into the function's implementation.

The `form/1` Component Under the Hood

You already know that a function component takes in an argument of some assigns and returns some markup wrapped in a HEEx template. So, you won't be surprised to see that at its core, the form/1 function component sets up a Phoenix.HTML.Form struct and renders it in a HEEx template that returns an HTML form.

We'll break down this process one step at a time, beginning with the creation of the form struct.

Creating the `Phoenix.Form`

The first thing that the form/1 function does under the hood is to construct a Phoenix.HTML.Form struct. This is constructed using the data in the assigns that we passed into our function call. Let's take a look at the code:

# Extract options and then to the same call as form_for
action = assigns[:action] || "#"
form_for = assigns[:for] || raise ArgumentError, "missing :for assign to form"
form_options = assigns_to_attributes(assigns, [:action, :for])

# Since FormData may add options, read the actual options from form
%{options: opts} =
  form = %Phoenix.HTML.Form{
    Phoenix.HTML.FormData.to_form(form_for, form_options)
    | action: action
  }

First, it casts the data from the assigns into the same format that Phoenix.HTML.form_for/4 uses to construct form structs. Then, it initializes a new struct.

Next up, the function constructs the form method, CSRF token, and multi-part setting from the data in the form struct, like this:

# And then process method, csrf_token, and multipart as in form_tag
{method, opts} = Keyword.pop(opts, :method, "post")
{method, hidden_method} = form_method(method)

{csrf_token, opts} =
  Keyword.pop_lazy(opts, :csrf_token, fn ->
    if method == "post", do: Plug.CSRFProtection.get_csrf_token_for(action)
  end)

opts =
  case Keyword.pop(opts, :multipart, false) do
    {false, opts} -> opts
    {true, opts} -> Keyword.put(opts, :enctype, "multipart/form-data")
  end

With the form struct and options in place, we then move on to constructing the assigns that will be rendered in the HEEx template here:

assigns =
  LiveView.assign(assigns,
    form: form,
    csrf_token: csrf_token,
    hidden_method: hidden_method,
    attrs: [action: action, method: method] ++ opts
  )

For LiveView to track changes to assigns values rendered by a function component, it must render a valid assigns either passed in as the only argument given to the function or created via a call to Phoenix.LiveView.assign/3 or Phoenix.LiveView.assign_new/3. So, the function component uses LiveView.assign/3 here to construct a new set of assigns to render in the HEEx template.

With the assigns in place, the function will then render the template. Let's take a look at that now.

Rendering the HEEx Template

The form/1 function component, like all function components, returns some markup wrapped in a HEEx template. Let's take a look at that return value now:

 ~H"""
<form {@attrs}>
  <%= if @hidden_method && @hidden_method not in ~w(get post) do %>
    <input name="_method" type="hidden" value={@hidden_method}>
  <% end %>
  <%= if @csrf_token do %>
    <input name="_csrf_token" type="hidden" value={@csrf_token}>
  <% end %>
  <%= render_slot(@inner_block, @form) %>
</form>
"""

Unsurprisingly, the form/1 function component simply returns a HEEx template wrapping a call to an HTML <form>. Let's dig into how the template uses some of the assigns established in the previous step.

First up, you can see that the @attrs assigns is interpolated directly into the opening <form> tag. This will render the tag with the appropriate method= and action= attributes. Next up, you can see that the @hidden_method assigns determines if and how to populate the hidden input. Then the @csrf token, if present, is added to the HTML form in another hidden input.

Most of this template has been pretty straightforward so far. The HEEx template renders, and HTML is constructed from various assigns values. Next up, we'll look at how the form fields we specified between our opening and closing <.form> tags are rendered into the template with the component slot functionality.

Render the Inner Block with Phoenix Component Slots

Phoenix Components implement a feature called "slots". Slots enable us to give blocks to our function component calls, nesting them within opening and closing function component tags like regular HTML tags. Slots are the reason for the form/1 syntax we used above:

<.form ...assigns>
  <!-- form fields -->
</.form>

Here's how it works. Suppose you call on a function component with opening and closing function component tags. That function component's template renders the content in those tags with the help of the render_slot/2 function.

Let's take a look at an example. Recall the UserDetails.contact_info/1 function component we defined earlier:

defmodule MyAppLive.UserDetails do
  use Phoenix.Component

  def contact_info(assigns) do
    ~H"""
    <div>
      <p><strong>Phone:</strong><%= @user.phone_number %></p>
      <p><strong>Email:</strong><%= @user.email %></p>
    </div>
    """
  end
end

We can refactor it to use slots like this:

defmodule MyAppLive.UserDetails do
  use Phoenix.Component

  def contact_info(assigns) do
    ~H"""
    <div class="contact-info">
      <%= render_slot(@inner_block) %>
    </div>
    """
  end
end

Now, we have a dynamic function component we can use to render any type of contact info:

<UserDetails.contact_info user="{@current_user}">
  <p><strong>Phone:</strong><%= @current_user.phone_number %></p>
</UserDetails.contact_info>

Or:

<UserDetails.contact_info user={@current_user}>
  <p>
    <strong>Address:</strong>
    <ul>
      <li><%= @current_user.street_address %></li>
      <li><%= @current_user.city %></li>
      <li><%= @current_user.zip_code %></li>
      <li><%= @current_user.country %></li>
    </ul>
  </p>
</UserDetails.contact_info>

The HTML markup we include between the opening and closing function component tags becomes available in the function component as the @inner_block assigns. The render_slot/2 function renders that content for us.

We can clean this up even further with the let assigns to yield a variable from the function component's template back to the calling template. Let's take a look.

First, we'll call the function component with an assigns of let set equal to a variable, user:

<UserDetails.contact_info let="{user}" user="{@current_user}">
  <!-- coming soon -->
</UserDetails.contact_info>

Then, we'll update the function component's call to render_slot/2 by invoking it with a second argument of the @user assignment, like this:

def contact_info(assigns) do
  ~H"""
  <div class="contact-info">
    <%= render_slot(@inner_block, @user) %>
  </div>
  """
end

The function component sets the variable we specified in the let assignment equal to the value of whatever is passed in as the second argument to render_slot/2. Now, the inner content between the function component's opening and closing tags in the calling template has access to the user variable. We can update our inner block to look like this:

<UserDetails.contact_info let="{user}" user="{@current_user}">
  <p><strong>Phone:</strong><%= user.phone_number %></p>
</UserDetails.contact_info>

With this basic understanding of slots in place, let's revisit the form/1 template:

~H"""
<form {@attrs}>
  <%= if @hidden_method && @hidden_method not in ~w(get post) do %>
  <input name="_method" type="hidden" value="{@hidden_method}" />
  <% end %> <%= if @csrf_token do %>
  <input name="_csrf_token" type="hidden" value="{@csrf_token}" />
  <% end %> <%= render_slot(@inner_block, @form) %>
</form>
"""

Here, its calling render_slot/2 with the @inner_block assignment and the @form assignment. Recall that we invoked form/1 like this:

<.form
    let={f}
    for={@changeset}
    id="book-form"
    phx-change="validate"
    phx-submit="save">

  <%= label f, :title %>
  <%= text_input f, :title %>
  <%= error_tag f, :title %>

  <!-- ... -->
</.form>

So, the form fields render as the @inner_block assignment. The call to <.form>
establishes a variable f, which gets set to a value of the second argument passed to render_slot/2. The second argument to render_slot/2 is the @form assignment, pointing to our Phoenix.HTML.Form struct. So, the inner content in our calling template can use the f variable to reference the form struct and build out the Phoenix form to render.

And that's it!

Wrap Up: Demystifying Phoenix LiveView's Form Function Component

While the call to the form/1 function component can seem mysterious, tracing the code under the hood isn't too daunting.

To recap: we can see that the function establishes a Phoenix form and assembles some assigns. Then, it returns a HEEx template that renders an HTML form with the assigns. The HEEx template uses Phoenix Component's slot functionality to render the inner content of our specified form fields, and yields the Phoenix form back to the calling template where we construct those fields.

I hope that this dive under the hood of the form/1 function component has not only demystified that function, but also given you a deeper understanding of how you can use function components and slots in your own live views. Now, you're ready to build out your own extensible function component that dynamically renders different inner blocks of content.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Securing Your Phoenix LiveView Apps

Sophie DeBenedetto — Tue, 01 Feb 2022 12:03:29 +0000

LiveView is a compelling choice for building modern web apps. Built on top of Elixir's OTP tooling, and leveraging WebSockets, it offers super fast real-time, interactive features alongside impressive developer productivity.

In this post, we'll show you how to secure your live view routes with function plugs and group live routes in a secure live session.

Let's dive straight in!

Using Live View to Build a Phoenix Web App

We'll be building some authentication and authorization features into a Phoenix web app built with live view.

The Arcade web app presents regular users with many online games to play. Our app has a survey feature that collects users' demographic data and game ratings. It also has an admin dashboard that should only be accessible to app admins to view survey results.

For the purpose of this post, we'll assume that logged-in users can visit the /games index and /games/:id show routes to select and play a game, along with the /survey route to fill out the user survey.

Additionally, admin users should only be able to visit the /admin-dashboard page. We'll assume that these pages and the live views that back them have already been built. Our focus is on introducing the authentication and authorization code we need to secure these live views.

Let's get started.

Protect Sensitive Routes in Your Phoenix LiveView App

This post assumes you've already built your registration and login flows, along with some function plugs for authenticating the current user and storing their token in the Phoenix session. I recommend using the Phoenix Auth generator to generate this code for free.

This generated code ensures that Phoenix will add a key of :current_user to the conn struct and a "user_token" key to the session when a user logs in.

Now, let's start in the router by putting some live routes behind authentication.

If you run the Phoenix Auth generator, you generate a module, ArcadeWeb.UserAuth, that implements a function plug require_authenticated_user/2, shown here:

def require_authenticated_user(conn, _opts) do
  if conn.assigns[:current_user] do
    conn
  else
    conn
    |> put_flash(:error, "You must log in to access this page.")
    |> maybe_store_return_to()
    |> redirect(to: Routes.user_session_path(conn, :new))
    |> halt()
  end
end

The details of this function aren't too important. Just understand that it takes in a first argument of the conn struct and checks for the presence of a :current_user key. If one is found, it returns the conn. If not, then it redirects to the login path.

When the auth generator creates the UserAuth module, it also imports into your router, like this:

# router.ex
import ArcadeWeb.UserAuth

We can create a new router scope using this function plug. Let's require that a user is logged in for access to the scope routes, like this:

scope "/", ArcadeWeb do
  pipe_through [:browser, :require_authenticated_user]
  # ...
end

Add the product index, show routes, and the /survey route that any authenticated user can currently visit:

scope "/", ArcadeWeb do
  pipe_through [:browser, :require_authenticated_user]
  live "/products", ProductLive.Index
  live "/products/:id", ProductLive.Show
  live "/survey", SurveyLive
end

Now, when a user visits /products or any other route in our new scope, Phoenix invokes the require_authenticated_user function plug. Believe it or not, that's all we have to do to restrict our live routes to logged-in users.

We can take a similar approach to authorizing admins to visit the /admin-dashboard. We'll add a new function plug to the UserAuth module, like this:

def require_admin_user(%{current_user: current_user} = conn, _opts) do
  if current_user.admin do
    conn
  else
    conn
    |> put_flash(:error, "You must log in to access this page.")
    |> maybe_store_return_to()
    |> redirect(to: Routes.page_path(conn))
    |> halt()
  end
end

def require_admin_user(conn, _opts) do
  conn
    |> put_flash(:error, "You must log in to access this page.")
    |> maybe_store_return_to()
    |> redirect(to: Routes.page_path(conn))
    |> halt()
end

If the function plug is invoked with a conn struct that does not contain the current user, we will redirect to the root path.

If the function plug is called with a conn that contains a current user, we will check if that user is an admin. If so, return the conn, otherwise, redirect. The details of our check for the admin status, current_user.admin, don't really matter here. Your app may implement admin logic differently. The main takeaway is that we now have a function plug that can authorize certain routes by enforcing that the current user is present and an admin.

Let's now use our new function plug in our router. We'll create a second scope with a pipeline that uses the require_admin_user/1 function plug:

scope "/", ArcadeWeb do
  pipe_through [:browser, :require_admin_user]
  live "/admin-dashboard", Admin.DashboardLive
end

Great! If a user points their browser at /admin-dashboard, our function plug will be invoked.

We've ensured that our live view routes are secure with nothing more than normal Phoenix auth plugs. We can implement authentication — requiring the presence of a current user — and authorization — requiring that the current user has specific permissions or roles — just like you would for regular Phoenix routes.

Now, let's look at a new LiveView feature for grouping live routes together and a security challenge that it presents.

Group Live Views in a Live Session

You'll use live sessions to group similar live routes with shared layouts and auth logic. Grouping live routes together in a live session means that we can live redirect to those routes from any other route in the same live session group.

A live redirect is a special kind of redirect that leverages the existing WebSocket connection, minimizing network traffic and keeping your live view speedy.

When you live redirect from one live view to another in the same live session, the current live view process terminates. The new live view is mounted over the current WebSocket connection without reloading the whole page.

This works great for live views that share a layout. The shared layout that frames the live view content will stay in place, and only the portion of the page that renders the current live view within that layout will change.

Let's create our first live session group now for the routes behind regular user authentication:

scope "/", ArcadeWeb do
  pipe_through [:browser, :require_authenticated_user]

  live_session :user, root_layout: {ArcadeWeb.LayoutView, "authenticated.html"} do
    live "/products", ProductLive.Index
    live "/products/:id", ProductLive.Show
    live "/survey", SurveyLive
  end
end

Here, we establish a live session block to contain all of our routes that need to be authenticated for regular users and tell them to share a common layout found in lib/arcade_web/templates/layout/authenticated.html.heex. You might notice that the live_session macro is called with a first argument, :user. We'll see how that comes into play in just a bit.

Whenever a user live redirects from the /products route to /products/:id, for example, the existing WebSocket connection will not terminate. Instead, we'll kill the current live view process, mount the new live view, and only re-render the necessary portions of the page within the shared layout.

Grouping Live Routes: The Security Problem

This approach presents a security challenge. If we re-use the existing WebSocket connection, we won't be sending a new HTTP request, and we won't go through the plug pipeline defined in our router.

So we must perform authentication and authorization in our router to prevent direct navigation to sensitive routes from the browser. We must also ensure that our live views can perform their own authentication and authorization every time they mount (whether due to a user pointing their browser directly at a live route or a live redirect between live routes in a shared live session).

Luckily for us, LiveView presents an API for performing authorization and authentication when the live view mounts, making it easy for us to apply this logic across all live routes in a shared session. Let's take a look.

Protect Live Views When They Mount

The LiveView framework allows us to hook into a callback function that will run whenever a live view mounts. The on_mount/4 lifecycle hook will fire before the live view mounts, making it the perfect place to isolate re-usable auth logic that can be shared among live views in a live session.

Start by defining a module that implements an on_mount/4 function, like this:

defmodule ArcadeWeb.UserAuthLive do
  import Phoenix.LiveView
  alias Arcade.Accounts

  def on_mount(:user, params, %{"user_token" => user_token} = _session, socket) do
    socket =
      socket
      |> assign(:current_user, Accounts.get_user_by_session_token(user_token))
    if socket.assigns.current_user do
      {:cont, socket}
    else
      {:halt, redirect(socket, to: "/login")}
    end
  end
end

This function will be called with:

a first argument of the atom that we passed to our live_session macro
a second argument of any params that were part of the incoming web request
a third argument of the session containing the "user_token" used to identify the current user
a fourth argument of the socket (remember, if you use the Phoenix Auth generator, Phoenix will add this "user_token" to the session when a user logs in.)

We perform some basic authentication here by looking up the current user and assigning the result to the socket. If the socket then contains a current user value rather than nil, we continue. Otherwise, we halt and redirect. Any on_mount/4 function must conform to this API, returning the :cont tuple or the :halt tuple.

Next up, let's tell our live session to apply this on_mount/4 callback to all of the live routes in its grouping:

scope "/", ArcadeWeb do
  pipe_through [:browser, :require_authenticated_user]

  live_session :user, on_mount: UserAuthLive, root_layout: {ArcadeWeb.LayoutView, "authenticated.html"} do
    live "/products", ProductLive.Index
    live "/products/:id", ProductLive.Show
    live "/survey", SurveyLive
  end
end

Whenever there is a live redirect to "/products" route (or any other route in that live session), the given live view will invoke ArcadeWeb.UserAuthLive.on_mount/4 with a first argument of :user and our authentication logic will execute. Furthermore, any live view within the live session will mount with the :current_user already set in its socket assigns, since we're adding it in the on_mount callback.

Let's set up a similar callback for the admin live session. Add this function to UserAuthLive:

def on_mount(:admin, params, %{"user_token" => user_token} = _session, socket) do
  socket =
    socket
    |> assign(:current_user, Accounts.get_user_by_session_token(user_token))
  if socket.assigns.current_user.admin do
    {:cont, socket}
  else
    {:halt, redirect(socket, to: "/")}
  end
end

Here, when on_mount/4 is called with a first argument of :admin, we will authorize the current user and authenticate them as an admin. Let's add this to a new live session for the admin-protected routes now:

 scope "/", ArcadeWeb do
  pipe_through [:browser, :require_admin_user]
  live_session :admin, on_mount: {UserAuthLive, :admin}, root_layout: {ArcadeWeb.LayoutView, "admin.html"} do
    live "/admin-dashboard", Admin.DashboardLive
    # more admin routes
  end
end

Here, we group admin-protected routes with a shared admin layout. And whenever any of the live views in this session block mount, the UserAuthLive.on_mount/4 function will be called with the :admin atom as a first argument. This ensures that only admin users can access those pages, even when live redirected.

Thanks to Elixir's pattern matching, we can group all of our auth-related on_mount/4 callbacks in a shared module and implement however many live_session blocks we need to organize our live views.

Wrap Up: Easily Group Live Views to Secure Your Phoenix LiveView App

In this post, we explored how LiveView allows you to group live routes in a shared session. Grouping enables live views to easily share a layout and implement shared authentication and authorization logic.

Remember, you must authenticate and authorize both your protected routes in the router and your live views when they mount. Reach for function plug pipelines to achieve the former, and live session and the on_mount/4 callback to accomplish the latter. With this combination of tools, you can bulletproof your live views, making them highly secure and capable of sophisticated authorization logic.

I hope you've found this post useful. Until next time: happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Build Interactive Phoenix LiveView UIs with Components

Sophie DeBenedetto — Tue, 18 Jan 2022 14:12:08 +0000

LiveView empowers developers to build interactive, single-page web apps with ease by providing a framework that eliminates the need for guesswork.

In this post, we'll take a look at how you can layer simple, single-purpose functional components to wrap up shared presentation logic. We'll also use more sophisticated live components to craft easy-to-maintain single-page flows that handle complex user interactions.

Along the way, you'll gain a solid understanding of working with HEEx — Phoenix and LiveView's new templating engine — and you'll see some of LiveView's out-of-the-box function components in action.

Let's dive in!

The Feature: Compose a User Survey UI for a Phoenix LiveView App

Before we dive into writing any actual code, let's talk about the feature we'll build. Imagine that you're responsible for a Phoenix web app, Arcade, that provides in-browser games to users. A user can log in, select a game to play, and even invite friends to play games with them.

In this post, we'll build out a "user survey" feature that asks the user to fill out some demographic info about themselves and then provide a rating for each of several games. We'll focus on that second part of the survey — the game rating forms.

Let's lay out what we'll build in a bit more detail. We'll begin with a parent live view that lives at the /survey route, ArcadeWeb.SurveyLive. This live view will render a child functional component, "ratings index". The ratings index component will iterate over the games and show a game rating if one by the current user exists, or a form for a new rating if not. If users haven't completed a game rating, they will see a list of forms to rate each game 1-5 stars. If they have completed some (or all) ratings, they will see those displayed and forms to submit ratings for the games they have not yet rated.

Here's a look at how it will work:

With our plan in place, we're ready to start writing code.

Define the Parent Live View

We'll build a route first, then mount and render the initial live view.

Define the Survey Route

Our first job is to establish a route. The survey will live at /survey, and it should only work for authenticated users so we can deliver a survey to single, identifiable users. We'll tie the route to the yet-to-be-written SurveyLive live view, with the :index live action, like this:

# router.ex
scope "/", ArcadeWeb do
  pipe_through [:browser, :require_authenticated_user]
    live "/survey", SurveyLive, :index
end

This code assumes you've used the Phoenix Auth generator to add authentication to your Phoenix app. The details of authentication aren't important for our purposes here. Just know that the survey route is a protected route that requires an authenticated user. This means that when a logged-in user points their browser at /survey, the SurveyLive view will mount with a session argument that contains a key of "user_token" pointing to a token we can use to identify the current user. The generator also gives us a function, Accounts.get_user_by_session_token(user_token), that we will use to fetch the user for that token.

With our route established, it's time to define the SurveyLive live view.

Mount the Survey Live View

The mount/3 function builds the initial state for SurveyLive. Let's think a bit about that initial state. We need to use the current user to build our survey's demographic and rating portions since a demographic belongs to a user and a rating belongs to a game and a user. So we want to store that user in the live view's state. This way, we can make it available to the rating form component later. Now, let's implement a mount/3 function that adds the current user to socket assigns, like this:

# lib/arcade_web/live/survey_live.ex
defmodule ArcadeWeb.SurveyLive do
  use ArcadeWeb, :live_view

  def mount(_params, %{"user_token" => user_token}, socket) do
    {:ok,
      socket
      |> assign(
          :current_user,
          Accounts.get_user_by_session_token(user_token))}
  end
end

Okay, we're ready to render a simple version of our survey live view.

Render the Survey Live View

We won't provide a render/1 function, instead we'll use a template — lib/arcade_web/live/survey_live.html.heex. Let's keep it simple for now:

<section class="row">
  <h2>Survey</h2>
</section>

Reload your browser, and you'll see the bare-bones template shown here:

We have the basic framework for our survey UI in place. Now, we're ready to build our first function component.

List Ratings in Phoenix LiveView

Before we build our ratings index function component, let's talk about what function components are and how they work. A function component takes in an assigns argument and returns a HEEx template. Function components are implemented in modules that use the Phoenix.Component behaviour, which also gives us a convenient syntax for rendering function components.

We're almost ready to define our function component. Let's take a step back and discuss HEEx.

Understanding HEEx Templates

A HEEx template is any file ending in the .heex extension that is implicitly rendered by a live view, or any markup rendered by a live view or component that is encapsulated in the ~H""", """ tags. The HEEx templating engine is an extension of EEx. Just like EEx templates, HEEx will process template replacements within your HTML code. Everything between the <%= and %> expressions is a template replacement. HEEx will evaluate the Elixir code within those tags and replace them with the result.

HEEx does more than just templating, though. It also:

provides compile-time HTML validations
gives us a convenient component rendering syntax
optimizes the amount of content sent over the wire, allowing LiveView to render only those portions of the template that need updating when state changes

HEEx is the default templating engine for Phoenix and LiveView. Any generated template files in your Phoenix app will be HEEx templates and end in the .html.heex extension. When using inline render/1 functions in your live views, or function components, you'll return HEEx templates with the ~H sigil.

With that basic understanding in place, we're ready to implement the ratings index function component.

Define the Function Component

We'll build a rating index component responsible for orchestrating the state of all the game ratings in our survey. This component will iterate over the games and render the rating details if a user rating exists, or the rating form if it doesn't. The responsibility for rendering rating details will be handled by a "rating show" function component. A live "rating form" component will handle rendering and managing a rating form. More on live components in a bit.

Meanwhile, SurveyLive will continue to be responsible for managing the overall state and appearance of the survey page. The rating index component will receive the list of game ratings to render from the parent live view. The parent live view is responsible for maintaining and updating that list.

In this way, we keep our code organized and easy to maintain because it adheres to the single responsibility principle — each component has one job to do. By layering these components within the parent SurveyLive view, we compose a series of small, manageable pieces into one interactive feature — the user survey page.

We'll begin by implementing the RatingLive.Index function component. Then, we'll move on to the rating show component, followed by the rating form component.

Create a file, lib/arcade_web/live/rating_live/index.ex, and key in the following component definition:

defmodule ArcadeWeb.RatingLive.Index do
  use Phoenix.Component
  use Phoenix.HTML
  alias ArcadeWeb.RatingLive
end

Our function component uses the Phoenix.Component behaviour which we need to render HEEx templates. Any module that implements only function components will use this behaviour. You'll use a different behaviour when building live or stateful components, which we'll do later on in this post.

We're also using the Phoenix.HTML behaviour here to bring in the Phoenix.HTML.raw/1 function that we'll use to render unicode characters — more on that in a bit. Finally, we're aliasing the name of the component module itself so it's easy to ergonomically invoke other function components defined within the module from within our main function component here.

The entry point of our module will be the games/1 function. We'll call on this function component from the parent live view to render the list of games. The function will take in an assigns argument containing the list of games passed in from the parent SurveyLive view. It will return a HEEx template that iterates over that list and renders another function component to show the game rating details if a rating exists and the rating form live component if not. Define that function now, as shown here:

def games(assigns) do
  ~H"""
  <div class="survey-component-container">
    <.heading games={@games} />
    <.list games={@games} current_user={@current_user}/>
  </div>
  """
end

We're composing our games/1 function out of two additional function components — heading/1 and list/1. We call on those function components with the .function_name assigns... /> syntax. This invokes the function component and passes in whatever assigns we provide as the assigns argument to that function.

It's also worth noting the {} interpolation syntax here, instead of the <%= %> EEx tags you might be used to. This is because HEEx, unlike EEx, isn't just responsible for evaluating and templating Elixir expressions into your HTML. It also parses and validates the HTML itself. So you can't use the traditional EEx tags inside HTML tags in a HEEx template. Instead, use curly braces to interpolate values inside HTML tags and function component calls, and use EEx tags when interpolating values in the body, or inner content, of those tags.

Let's build those additional function components now, starting with heading/1:

def heading(assigns) do
  ~H"""
    <h2>
      Ratings
      <%= if ratings_complete?(@games), do: raw "&#x2713;" %>
    </h2>
  """
end

The heading/1 function is pretty small and single-purpose. It renders an <h2> element that encapsulates some text along with a helper function that checks to see if all of the games have a rating by the current user. If so, we render the unicode to a checkmark and the user can see that all of the ratings forms have been completed.

Before we implement this helper function, let's see how we're going to render the index component with a list of games.

When we render this index component from the SurveyLive template, we'll use the SurveyLive view to query for the list of games with ratings by the current preloaded user.

Then, we'll pass that list of games down into the index component. So we can assume that each game in the @games list has its ratings list populated only with a rating from the current user. With that in mind, we can implement the ratings_complete?/1 function to iterate over the list of games and return true if there is a rating for every game. Add in your function now, like this:

 defp ratings_complete?(games) do
  Enum.all?(games, fn game ->
    length(game.ratings) == 1
  end)
end

Now if a user has completed all of the game ratings, they'll see the "Ratings" header with a nice checkmark next to it:

With the heading/1 function component out of the way, let's turn our attention to list/1. Add in this function now:

def list(assigns) do
  ~H"""
  <%= for {game, index} <- Enum.with_index(@games) do %>
    <%= if rating = List.first(game.ratings) do %>
      <h3>Show rating coming soon!</h3>
    <% else %>
      <h3>Rating form coming soon!</h3>
    <% end %>
  <% end %>
  """
end

Here, we use a for comprehension that maps over all of the games in the system, where each game's ratings list contains the single preloaded rating by the given user if one exists. Inside that comprehension, the template will render the rating details if a rating exists or a form for that rating if not. Nesting components in this manner lets the reader of the code deal with a tiny bit of complexity at a time.

We'll dig into this logic a bit more when we're ready to implement these final two components. With the index component out of the way, we are ready to weave it into our SurveyLive template.

Render the Component

The next bit of code we'll write shows how the presentation of our view can change based on the contents of the socket. The SurveyLive view will use the state of the overall survey to control what is shown to the user. This view holds the list of games, and their ratings by the current user, in state. It will pass this list into the RatingLive.games/1 function component as part of the component assigns. The contents of this list will allow the games/1 function component to determine if it should show rating details or a rating form.

Let's update SurveyLive now to query for the list of games and their ratings from the current user and add them to socket assigns, like this:

defmodule ArcadeWeb.SurveyLive do
  use ArcadeWeb, :live_view
  alias Arcade.Survey

  def mount(_params, %{"user_token" => token}, socket) do
    {:ok,
     socket
     |> assign(:current_user, Accounts.get_user_by_session_token(token))
     |> assign(:games, Catalog.list_games_with_user_rating(user))}
  end
end

Assume that the Catalog.list_games_with_user_rating/1 context function returns the list of all games, with only the rating by the given user preloaded, if any.

Now we're ready to render our rating index function component from the SurveyLive template:

<!-- lib/arcade_web/live/survey_live.html.heex -->
  <RatingLive.Index.games games={@games}
      current_user={@current_user} />

Here, we're once again using the <.function_component assigns... /> syntax to render our function component and the {} interpolation syntax for interpolating within tags in a HEEx template.

Now that we're rendering our RatingLive.Index.games/1 function component with the game list, let's build the stateless function component to show the existing rating for a game.

Show a Rating

We're getting closer to the goal of showing ratings, step by step. Remember, we'll show the existing ratings, and forms for ratings, otherwise.

Let's cover the case for ratings that exist first. We'll define a stateless component to show a rating. Then, we'll render that component from within the HEEx template returned by RatingLive.Index.games/1. Let's get started.

Build the Function Component

Create a file, lib/arcade_web/live/rating_live/show_component.ex, and key this in:

defmodule ArcadeWeb.RatingLive.Show do
  use Phoenix.Component
  use Phoenix.HTML
end

We're defining a module that uses the Phoenix.Component behaviour and the Phoenix.HTML behaviour, since we'll once again need support for the Phoenix.HTML.raw/1 function to render unicode characters.

Okay, let's move on to the entry point of our function component, the stars/1 function. We'll call this function from within the HEEx template returned by RatingLive.Index.games/1 with an assigns that includes the given game's rating by the current user.

The stars/1 function will operate on this rating and use some helper functions to construct a list of filled and unfilled unicode star characters. We'll construct that list using a simple pipeline, and then render it in a HEEx template, like this:

def stars(assigns) do
  stars =
    filled_stars(assigns.rating.stars)
    |> Enum.concat(unfilled_stars(assigns.rating.stars))
    |> Enum.join(" ")

  ~H"""
  <div>
    <h4>
      <%= @game.name %>:<br/>
      <%= raw stars %>
    </h4>
  </div>
  """
end

The filled_stars/1 and unfilled_stars/1 helper functions are interesting. Take a look at them here:

def filled_stars(stars) do
  List.duplicate("&#x2605;", stars)
end

def unfilled_stars(stars) do
  List.duplicate("&#x2606;", 5 - stars)
end

Examining our pipeline in the stars/1 function, we can see that we call on filled_stars/1 to produce a list of filled-in, or "checked", star unicode characters corresponding to the number of stars the game rating has. Then, we pipe that into a call to Enum.concat/2 with a second argument of the output from unfilled_stars/1. This second helper function produces a list of empty, or not checked, star characters for the remaining number of stars.

For example, if the number of stars in the rating is 3, our pipeline of helper functions will create a list of three checked stars and two un-checked stars. Our pipeline concatenates the two lists together and joins them into a string of HTML that we can render in the template.

We have everything we need to display a completed rating, so it's time to roll several components up together.

Render the Component

We're ready to implement the next phase of our plan. The RatingLive.Index.games/1 function component iterates over the list of games in the @games assigns. If a rating is present, we show it. Add in the call to our new Show.stars/1 component now, like this:

def list(assigns) do
  ~H"""
  <%= for {game, index} <- Enum.with_index(@games) do %>
    <%= if rating = List.first(game.ratings) do %>
      <Show.stars rating={rating} game={game} />
    <% else %>
      <h3>Rating form coming soon!</h3>
    <% end %>
  <% end %>
  """
end

It's a straight for comprehension with an if statement. If a rating exists, we render the function component by calling on it with the game and rating assigns. If not, we need to render the form. Let's build that form and render it now.

Submit a Rating

Our rating form will display the form and manage its state, validating and saving the rating. We'll need to pass a game and user for our database relationships, and the game's index in the parent LiveView's socket.assigns.games list. We'll use this index later on to update SurveyLive state efficiently.

Build the Rating Form Component

The component will be stateful since it needs to manage the state of the rating form and respond to user interactions to validate form changes and handle the form submission.

A stateful, or live, component is any module that uses the :live_component behaviour and renders a HEEx template. Such modules can implement the live component lifecycle functions, including mount/3, update/2, and render/1, and respond to user events by implementing a handle_event/3 function. Let's define our live component module now. Create a file, lib/arcade_web/live/rating_live/form.ex, and key this in:

defmodule ArcadeWeb.RatingLive.FormComponent do
  use ArcadeWeb, :live_component
  alias Arcade.Survey
  alias Arcade.Survey.Rating
end

This is simple enough, to begin with. We define our module, use the :live_component behaviour, and add in some aliases that we'll need later.

We'll use LiveView's .form/1 function (more on that in a bit) to construct the rating form. This function requires a changeset, so we'll need to store one in our component's state. Here's where the component lifecycle comes into play. When we render a live component, LiveView starts the component in the parent view's process and calls these callbacks, in order:

mount/1
: The single argument is the socket, and we use this callback to set the initial state. This callback is invoked only once, when the component is first rendered from the parent live view.
update/2
: The two arguments are the assigns argument given to live_component/3 and the socket. By default, it merges the assigns argument into the socket.assigns established in mount/1. We'll use this callback to add additional content to the socket each time live_component/3 is called.
render/1
: The one argument is socket.assigns. It works like a render in any other live view.

Stateful components will always follow this process when first mounted and rendered. Then, when the component updates in response to changes in the parent live view, only the update/2 and render/1 callbacks fire. Since these updates skip the mount/1 callback, the update/2 function is the safest place to establish the component's initial state. Let's create our component's update/2 function now, like this:

def update(assigns, socket) do
  {:ok,
    socket
    |> assign(assigns)
    |> assign_rating()
    |> assign_changeset()}
end

def assign_rating(
    %{assigns: %{current_user: user, game: game}} = socket) do
  assign(socket, :rating, %Rating{user_id: user.id, game_id: game.id})
end

def assign_changeset(%{assigns: %{rating: rating}} = socket) do
  assign(socket, :changeset, Survey.change_rating(rating))
end

These reducer functions will add the necessary keys to our socket.assigns. They'll drop in any assigns our parent sends, add a new Rating struct, and finally establish a changeset for the new rating.

There are no surprises here. One reducer builds a new rating, and the other uses the Survey context to build a changeset for that rating. Now, on to render.

With our socket established, we're ready to render. We'll choose a template to keep our markup code neatly compartmentalized. Create a file, lib/arcade_web/live/rating_live/form.html.heex. Add the game title markup followed by the game rating form shown here:

<div class="survey-component-container">
  <section class="row">
  <h4><%= @game.name %></h4>
  </section>
  <section class="row">
    <.form
      let={f}
      for={@changeset}
      phx-change="validate"
      phx-submit="save"
      phx_target={@myself}
      id={@id}>

      <%= label f, :stars%>
      <%= select f, :stars, Enum.reverse(1..5) %>
      <%= error_tag f, :stars %>

      <%= hidden_input f, :user_id%>
      <%= hidden_input f, :game_id%>

      <%= submit "Save", phx_disable_with: "Saving..." %>
    </.form>
  </section>
</div>

The form template is really just a standard Phoenix form, although the syntax for rendering the form may be new to you. The main function in the template is the form/1 function: a function component made available by LiveView under the hood. The form function component returns a rendered HEEx template containing an HTML form built with the help of Phoenix.HTML.Form.form_for/4.

If you're feeling adventurous, you can check out the source code for the form function component. For now, all you really need to know is that calling form/1 returns an HTML form for the specified changeset, with the specified LiveView bindings. Let's take a closer look at how our form is rendered.

Since the form/1 function is built on top of the form_for/4 function, it presents a similar API. Here, we're generating a form for the @changeset assignment that was put in assigns via the update/2 callback.

Then, we bind two events to the form, a phx-change to send a validate event and a phx-submit to send a save event. We target our form component to receive events by setting phx-target to @myself, and we tack on an id.

Note that we've set a dynamic HTML id of the stateful component id, stored in socket assigns as @id. This is because the game rating form will appear multiple times on the page, once for each game, and we need to ensure that each form gets a unique id. You'll see how we set the id assigns for the component when we render it in a bit.

Our form has a stars field with a label and error tag and a hidden field for each user and game relationship. We tie things up with a submit button.

We'll come back to the events a bit later. For now, let's fold our work into the RatingLive.Index.list/1 function component.

Render the Component

The RatingLive.Index.games/1 function component should render the rating form component if no rating for the given game and user exists. Let's do that now.

def list(assigns) do
  ~H"""
  <%= for {game, index} <- Enum.with_index(@games) do %>
    <%= if rating = List.first(game.ratings) do %>
      <Show.stars rating={rating} game={game} />
    <% else %>
      <.live_component module={RatingLive.Form}
                        id={"rating-form-#{game.id}"}
                        game={game}
                        game_index={index}
                        current_user={@current_user } />
    <% end %>
  <% end %>
  """
end

Here, we call on the component with the live_component/1 function, passing the user and game into the component as assigns, along with the game's index in the @games assignment. We add an :id, a requirement of all stateful components. Since we'll only have one rating per component, our id with an embedded game.id should be unique.

The live_component/1 function is a function component made available to us by the LiveView framework. It takes in an argument of some assigns and returns a HEEx template that renders the given component within the parent live view. When using live_component/1 to render a live component, you must specify an assigns of module, pointing to the name of the live component module to mount and render, and an assigns of id, which LiveView will use to keep track of the component. Also, note the {} interpolation syntax we're using — this syntax is required when interpolating within HTML or HEEx tags.

It's been a while since we've looked at things in the browser — but now, if you point your browser at /survey, you should see something like this:

Handle Component Events

We've bound events to save and validate our form, so we should teach our component how to do both. We need one handle_event/2 function head for each of the save and validate events. Let's start with validate:

def handle_event("validate", %{"rating" => rating_params}, socket) do
  {:noreply, validate_rating(socket, rating_params)}
end

We need to build the reducer next:

def validate_rating(socket, rating_params) do
  changeset =
    socket.assigns.rating
    |> Survey.change_rating(rating_params)
    |> Map.put(:action, :validate)

  assign(socket, :changeset, changeset)
end

Our validate_rating/2 reducer function validates the changeset and returns a new socket with the validated changeset (containing any errors) in socket assigns. This will cause the component to re-render the template with the updated changeset, allowing the error_tag helpers in our form_for form to render any errors.

Next up, we'll implement a handle_event/2 function that matches the save event:

def handle_event("save", %{"rating" => rating_params}, socket) do
  {:noreply, save_rating(socket, rating_params)}
end

And here's the reducer:

def save_rating(
         %{assigns: %{product_index: product_index, product: product}} = socket,
         rating_params
       ) do
  case Survey.create_rating(rating_params) do
    {:ok, rating} ->
      product = %{product | ratings: [rating]}
      send(self(), {:created_rating, product, product_index})
      socket

    {:error, %Ecto.Changeset{} = changeset} ->
      assign(socket, changeset: changeset)
  end
end

Here, we attempt to save the form. On failure, we assign a new changeset. On success, we send a message to the parent live view to do the heavy lifting for us. Then, as all handlers must do, we return the socket.

Update the Rating Index

What should happen when the game rating successfully saves? The RatingLive.Index.games/1 function should no longer render the form for that game. Instead, the survey should display the saved rating. This kind of state change is squarely the responsibility of SurveyLive. Our message will serve to notify the parent live view to change.

Here's the interesting bit. All the parent needs to do is update the socket. The RatingLive.Index.games/1 function already renders the right thing based on the content of the assigns that it receives from the parent, SurveyLive. All we need to do is implement a handler to deal with the "created rating" message.

# lib/arcade_web/live/survey_live.ex
def handle_info({:created_rating, updated_product, product_index}, socket) do
  {:noreply, handle_rating_created(socket, updated_product, product_index)}
end

We use handle_info so that the parent live view process, SurveyLive, can respond to the message sent by the child component. Now, our reducer can take the appropriate action. Notice that the message we match has a message name, an updated game, and its index in the :games list. We can use that information to update the game list without going back to the database. We'll implement the reducer below to do this work:

def handle_rating_created(
         %{assigns: %{products: products}} = socket,
         updated_product,
         product_index
       ) do

  socket
  |> put_flash(:info, "Rating submitted successfully")
  |> assign(
    :products,
    List.replace_at(products, product_index, updated_product)
  )
end

The handle_rating_created/3 reducer adds a flash message and updates the game list with its rating. This causes the template to re-render, passing this updated game list to RatingLive.Index.games/1. That function component, in turn, knows just what to do with a game containing a rating by the given user — it will render that rating's details instead of a rating form.

Notice the lovely layering. In the parent live view layer, all we need to do is manage the list of games and ratings. All of the form handling and rating or demographic details go elsewhere.

The end result of a submitted rating is an updated game list and a flash message. Submit a rating, and see what happens:

You just witnessed the power of components and LiveView.

Wrap Up: You've Built a Complex LiveView UI with Components

This post covered a lot of ground. You implemented a sophisticated UI composed from simple layers, all thanks to LiveView components. You can wrap up simple markup with function components, while live components allow you to maintain component state and respond to events. Meanwhile, HEEx templates and LiveView provide some nice semantics for rendering markup and both types of components.

LiveView is growing fast — it's responding to the community's needs and providing even more ergonomic solutions for developing complex interactive single-page apps. Function components and HEEx are just some of the latest features that make LiveView even more enjoyable to use.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

What's New in Elixir 1.13

Sophie DeBenedetto — Tue, 21 Dec 2021 12:53:01 +0000

On December 3rd, the Elixir core team announced the latest Elixir release, 1.13. In this post, we'll explore some of the more exciting changes and new features, and discuss their impact on the Elixir community.

All About Tooling

Elixir 1.13 comes packed with lots of goodies and improvements designed to make the day-to-day life of Elixir developers easier. With new additions like semantic code recompilation, better support for auto-complete in IEx, an API for custom code formatting, and more, quality of life for Elixir developers is going to greatly increase. Let's dive into some of the latest developments now.

Making IEx Better

Elixir 1.13 comes with a number of changes that enhance the IEx experience. The introduction of the new Code.Fragment module greatly improves the autocomplete story for Elixir, making your work in IEx even easier. Code.Fragment provides conveniences for analyzing fragments of textual code and returning information about those fragments. Practically speaking, this means that IEx now supports autocomplete of sigils, structs, and paths. Let's take a look at some examples.

Previously, attempting to autocomplete the ~ character didn't lead to anything too useful:

iex(1)> ~
!/1                           !=/2
!==/2                         %/2
%{}/1                         &&/2
&/1                           */2
++/2                          +/1
+/2                           --/2
-/1                           -/2
..///3                        ../2
./2                           //2
::/2                          </2
<<>>/1                        <=/2
<>/2                          =/2
==/2                          ===/2
=~/2                          >/2
>=/2                          @/1
^/1                           __CALLER__/0
__DIR__/0                     __ENV__/0
__MODULE__/0                  __STACKTRACE__/0
__aliases__/1                 __block__/1
abs/1                         alias!/1
alias/2                       and/2
apply/2                       apply/3
b/1                           binary_part/3
binding/0                     binding/1
bit_size/1                    break!/1
break!/2                      break!/3
break!/4                      breaks/0
byte_size/1                   c/1
c/2                           case/2
cd/1                          ceil/1
clear/0                       cond/1
continue/0                    def/1
def/2                         defdelegate/2
defexception/1                defguard/1
defguardp/1                   defimpl/2
defimpl/3                     defmacro/1
defmacro/2                    defmacrop/1
defmacrop/2                   defmodule/2
defo
# ... and the list goes on

Now, if you attempt the same autocomplete, you'll see this:

iex> ~
~C (sigil_C)    ~D (sigil_D)    ~N (sigil_N)    ~R (sigil_R)
~S (sigil_S)    ~T (sigil_T)    ~U (sigil_U)    ~W (sigil_W)
~c (sigil_c)    ~r (sigil_r)    ~s (sigil_s)    ~w (sigil_w)

A helpful list of sigil functions.

Similarly, in earlier Elixir versions, trying to autocomplete a struct would yield the following:

First, assume you have a struct, User, defined like this:

defmodule User do
  defstruct [:name, :age]
end

Attempt IEx autocomplete by hitting %Use<TAB>:

iex> %Use
 %User.__struct__
__struct__/0    __struct__/1

Not exactly a helpful look at the struct. Let's see what the same operation yields in Elixir 1.13:

iex> %User{
name:    age:

Here, we get a helpful look at all of the fields of our struct.

These autocomplete improvements don't just make IEx better. Your autocomplete experience in your text editor of choice will also improve. With additional changes like new compilation tracers and new Module functions for retrieving module metadata, we can expect to see new code intelligence features come out that will improve your text editor experiences with Elixir even further.

Faster Elixir Code Recompilation

With Elixir's new semantic code recompilation, your code will be recompiled less frequently. This means a faster development life cycle, especially when it comes to working with large Elixir applications.

Prior to Elixir 1.13, your entire Elixir app would be recompiled whenever any of the mix.exs, config/config.exs, src/*, or mix.lock files changed. Now, thanks to semantic code compilation, your code will recompile only when you change the compilation options in the mix.exs file or the configuration in the config/config.exs file.

Code compilation has gotten smarter in other ways too. Not only will your application be recompiled less frequently, but certain changes will now trigger recompilation of smaller parts of your app, or no recompilation at all.

For example, now if the size and digest of a file stay the same, the file will not be recompiled. This means switching or rebasing git branches will trigger far fewer recompilations. Additionally, changing compile-time config files (like config/config.exs) will only recompile any files that depend on the reconfigured files. Practically speaking, this means that, for example, bumping your Phoenix version in mix.exs will only recompile lib/my_app_web and not lib/my_app.

Semantic compilation is accompanied by some improvements to the mix xref task. This task gives you information about your files and how they depend on each other. Here's an abbreviated look at running mix xref with the graph option in a standard Phoenix app:

// ♥ mix xref graph
lib/pento.ex
lib/pento/accounts.ex
├── lib/pento/accounts/user.ex (export)
├── lib/pento/accounts/user_notifier.ex
├── lib/pento/accounts/user_token.ex (export)
└── lib/pento/repo.ex

This tells us that all of the files listed under lib/pento/accounts/ex depend on account.ex at compilation time. The task already gives us some great tooling to identify that a compile-time dependency exists, but doesn't really provide any insight into why such a dependency exists. So, if you needed to track down the source of an inappropriate compile-time dependency when that source is non-obvious (if it comes from a macro, for example), this tool used to fall short. Until now!

Elixir 1.13 adds a new option to the mix xref task - the trace option. Now, you can run mix xref trace <file name> and see a list of exactly where all of our compile time dependencies, exports, and runtime dependencies come from within that file, down to the exact line number. Let's check it out:

lib/pento/accounts.ex:26: call Pento.Repo.get_by/2 (runtime)
lib/pento/accounts.ex:43: call Pento.Repo.get_by/2 (runtime)
lib/pento/accounts.ex:44: call Pento.Accounts.User.valid_password?/2 (runtime)
lib/pento/accounts.ex:61: call Pento.Repo.get!/2 (runtime)
lib/pento/accounts.ex:78: struct Pento.Accounts.User (export)
lib/pento/accounts.ex:79: call Pento.Accounts.User.registration_changeset/2 (runtime)
lib/pento/accounts.ex:80: call Pento.Repo.insert/1 (runtime)
lib/pento/accounts.ex:92: struct Pento.Accounts.User (export)
lib/pento/accounts.ex:93: call Pento.Accounts.User.registration_changeset/3 (runtime)
lib/pento/accounts.ex:108: call Pento.Accounts.User.email_changeset/2 (runtime)
lib/pento/accounts.ex:126: call Pento.Accounts.User.email_changeset/2 (runtime)
lib/pento/accounts.ex:127: call Pento.Accounts.User.validate_current_password/2 (runtime)
lib/pento/accounts.ex:140: call Pento.Accounts.UserToken.verify_change_email_token_query/2 (runtime)
lib/pento/accounts.ex:141: struct Pento.Accounts.UserToken (export)
lib/pento/accounts.ex:141: call Pento.Repo.one/1 (runtime)
# ... I'm truncating this output for brevity, but there's more!

This gives us a powerful tool for understanding the interdependencies in our code and eliminating them where necessary.

Better Error Messages in Elixir

Elixir 1.13 also brings us improved error messages for both the SyntaxError and TokenMissing errors. Such error messages now include code snippets that show you the exact source of your error. This is just one more feature to improve your development experience and make hunting down bugs even easier. Let's try it out.

Assuming I've added the following bad code to my project:

def add_nums(num1, num2) do
  num1 + *num2
end

I should see the following error when I try to do something like run iex -S mix:

== Compilation error in file lib/pento/accounts.ex ==
** (SyntaxError) lib/pento/accounts.ex:26:9: syntax error before: '*'
    |
 26 |     num1 + *num2
    |         ^
    (elixir 1.13.0) lib/kernel/parallel_compiler.ex:346: anonymous fn/5 in Kernel.ParallelCompiler.spawn_workers/7

Extensible Formatting

The latest Elixir release extends the capabilities of the Elixir formatter so you can implement custom formatting for any embedded code.

Consider the following scenario. You have a LiveView app that renders lots of HEEx templates:

# lib/my_app_live/user_live.ex

def render(assigns)
  ~H"""
  <h2> User Profile
    </h2>
  """
end

You can see that the code in our HEEx template is formatted improperly, but since its not actually invalid, HEEx won't complain about it. When we run mix format, the code embed within the sigil_H won't be formatted.

Now, though, Elixir provides an API through which we can implement and apply custom formatters for embedded code. Here's how it works.

You'll define a custom formatter plugin module along these lines:

defmodule MixHEExFormatter do
  @behaviour Mix.Tasks.Format

  def features(_opts) do
    [sigils: [:H], extensions: [".ex",".heex"]]
  end

  def format(contents, opts) do
    # logic that formats HEEx
  end
end

Then, you'll add your plugin to your app in .formatter.exs, like this:

[
  plugins: [MixHEExFormatter],
  # Remember to update the inputs list to include the new extensions
  inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}", "lib/my_app_live/live/*.{ex,.heex}"]
]

With this new functionality, we can expect to see some upcoming development of custom formatters for common types of embedded code. The Zigler project already has an issue open for a custom ~Z formatter, and I hope to see some development for a ~H soon.

AST-Based Tooling

Another exciting new piece of functionality comes by way of two new functions added to the Code module. Code.quoted_to_algebra/2 and `Code.string_to_quoted_with_comments/2 will enable you to fetch the Elixir AST with original comments and convert it into formatted code.

This capability will empower developers to create tools for custom formatting of Elixir source code.

Better Testing in Elixir

Elixir 1.13 ships with a few new ExUnit niceties for debugging tests. ExUnit will now show a hint when comparing two different but equivalent strings, making debugging minor discrepancies between strings in your ExUnit expectations much less of a headache.

ExUnit also provides two new functions:

ExUnit.CaptureIO.with_io/3 allows you to apply a function to captured IO output
ExUnit.CaptureLog.with_log/2 invokes the given function and returns the result and a captured log.

Here's the new example from the docs:

elixir{result, log} =with_log(fn ->Logger.error("log msg")2 + 2end)`

assert result == 4
assert log =~ "log msg"
`

With this, we can easily test the result of function invocations and validate assertions against any log statements made during those function invocations.

Lastly, mix supports a new option, --profile-require=time - so that you can run mix test --profile-require=time to debug test suite load times.

New Elixir Language Features

In addition to this impressive list of new tooling capabilities, Elixir 1.13 also ships with some exciting new pieces of Elixir functionality. We'll take a look at just of few of these changes here.

The Enum.slide/3 function takes in arguments of an enumerable, index or range of indices, and the insertion index. It "slides" the index or range of indices to the target insertion index, like this:

`elixir

#Slide a single element
Enum.slide([:a, :b, :c, :d, :e, :f, :g], 5, 1)
#=> [:a, :f, :b, :c, :d, :e, :g]

Here, we "slid" the element living at index 5, :f, and inserted it instead at index 1.

The List.keyfind/4 function receives a list of tuples and returns the first tuple where the element at the position in the tuple matches the given key. For example:

elixirList.keyfind([a: 1, b: 2], :a, 0)# => {:a, 1}

The Keyword.validate/2 function lets you validate a keyword list to ensure that it contains only the provided set of values.

You can now also filter a keyword list with the Keyword.filter/2 function and map over a keyword list with the Keyword.map/2 function.

Wrap Up

This was a quick overview of the highlights from Elixir 1.13, but there's lots more to see, and you can check out the full changelog. This latest Elixir release means that quality of life for Elixir developers is getting even better.

New autocomplete and associated features make writing Elixir code faster, semantic code recompilation means faster development cycles, and new testing conveniences and Elixir language capabilities give us even more tools to get the job done.

We're excited to see what the community builds on top of all of this functionality.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

How to Do Live Uploads in Phoenix LiveView

Sophie DeBenedetto — Tue, 19 Oct 2021 14:19:21 +0000

The LiveView framework supports all of the most common features that Single-Page Apps must offer their users, including multipart uploads. In fact, LiveView can give us highly interactive file uploads, right out of the box.

In this post, we'll add a file upload feature to an existing Phoenix LiveView application. Along the way, you'll learn how to use LiveView to display upload progress and feedback while editing and saving uploaded files.

Setting Up the Image Upload Feature

The live upload examples that we'll be looking at in this post are drawn from the "Forms and Changesets" chapter in my book, Programming LiveView, co-authored with Bruce Tate. Check it out for an even deeper dive into LiveView forms and so much more.

In this example, we have an online game store — Arcade — that allows users to browse and review products. Admins can access a product management interface to create, edit, and delete the products we offer our users. We'll give admins the ability to upload a product image that is then stored in the database along with the given product. Let's plan out our new image upload feature before we start writing any code.

We'll begin in our application core by adding an image_upload field to the table and schema for products. Then, we'll create a LiveView form component that supports file uploads. Finally, we'll teach our component to report on upload progress and other bits of upload feedback.

This post will focus on adding the live upload functionality to an existing LiveView app that already implements:

A live view for displaying products.
A LiveView component that contains a form for creating/editing products.

We'll zero in on the code required to add the upload functionality to this form.

Check out my earlier post for a basic introduction to working with forms in LiveView.

Now we're ready to write some code.

Persist Product Images in Phoenix LiveView

Assuming our Phoenix LiveView app already has a products table and Product schema, we'll now update both to store an image_upload attribute. This attribute will point to the location of the uploaded file. Once we have our backend wired up, we'll be able to update the existing live view form to accommodate file uploads for a product.

We'll start at the database layer by generating a migration to add a field, :image_upload, to the products table.

First, generate your migration file:

[arcade] ➔ mix ecto.gen.migration add_image_to_products
* creating priv/repo/migrations/20201231152152_add_image_to_products.exs

This creates a migration file for us, priv/repo/migrations/20201231152152_add_image_to_products.exs. Open up that file and key in the contents to the change function:

defmodule Arcade.Repo.Migrations.AddImageToProducts do
  use Ecto.Migration

  def change do
    alter table(:products) do
      add :image_upload, :string
    end
  end
end

This code will add the new database field when we run the migration. Let's do that now:

[arcade] ➔ mix ecto.migrate

[info]  == Running 20201231152152 \

Arcade.Repo.Migrations.AddImageToProducts.change/0 forward
10:22:24.034 [info]  alter table products

10:22:24.041 [info]  == Migrated 20201231152152 in 0.0s

This migration added a new column — :image_upload — of type :string to the products table, but our schema still needs attention.

Update the corresponding Product schema by adding the new :image_upload field to the schema function, which should look like this:

defmodule Arcade.Catalog.Product do
  use Ecto.Schema
  import Ecto.Changeset

  schema "products" do
    field :description, :string
    field :name, :string
    field :sku, :integer
    field :unit_price, :float
    field :image_upload, :string
    timestamps()
  end

  @doc false
  def changeset(product, attrs) do
    product
    |> cast(attrs, [:name, :description, :unit_price, :sku, :image_upload])
    |> validate_required([:name, :description, :unit_price, :sku])
    |> validate_number(:unit_price, greater_than: 0)
    |> unique_constraint(:sku)
  end
end

Remember, the changeset cast/4 function must explicitly whitelist new fields, so make sure you add the :image_upload attribute there, as shown above.

Now that the changeset has an :image_upload attribute, we can save product records that know their image upload location. With that in place, we can make an image upload field available in the ProductLive.FormComponent's form. We're one step closer to giving users the ability to save products with images.

Now, let's turn our attention to the component.

How to Allow Live Uploads

We'll see our product changeset in action in a bit. First, we need to give the product form the ability to support file uploads. In our Phoenix application, both the "new product" and "edit product" pages use the ProductLive.FormComponent. This provides one centralized place to maintain our product form. Changes to this component will enable users to upload an image for a new product and a product they are editing.

To enable uploads for our component, or any live view, we need to call the allow_upload/3 function with a first argument of the socket. This will put the data into socket assigns that the LiveView framework will then use to perform file uploads. So, for a component, we'll call allow_upload/3 when the component first starts up and establishes its initial state in the update/2 function. For a live view, we'd call allow_upload/3 in the mount/3 function.

The allow_upload/3 function is a reducer that takes in an argument of the socket, the upload name, and the upload options and returns an annotated socket. Supported options include file types, file size, number of files per upload name, and more. Let's see it in action:

defmodule ArcadeWeb.ProductLive.FormComponent do
  use ArcadeWeb, :live_component
  alias Arcade.Product

  @impl true
  def update(%{product: product} = assigns, socket) do
    changeset = Product.changeset(product, %{})
    {:ok, socket
      |> assign(assigns)
      |> assign(:changeset, changeset)
      |> allow_upload(:image, accept: ~w(.jpg .jpeg .png), max_entries: 1)}
  end

In allow_upload/3, we pipe in a socket and specify a name for our upload: :image. We also provide some options — the maximum number of permitted files and the accepted file formats.

Let's take a look at what our socket assigns looks like after allow_upload/3 is invoked:

%{
  # ...
  uploads: %{
    __phoenix_refs_to_names__: %{"phx-FlZ_j-hPIdCQuQGG" => :image},
    image: #Phoenix.LiveView.UploadConfig<
      accept: ".jpg,.jpeg,.png",
      entries: [],
      errors: [],
      max_entries: 1,
      max_file_size: 8000000,
      name: :image,
      progress_event: #Function<1.71870957/3 ...>,
      ref: "phx-FlZ_j-hPIdCQuQGG",
      ...
    >
  }
}

The socket now contains an :uploads map that specifies the configuration for each upload field your live view allows. We allowed uploads for an upload called :image. So our map contains a key of :image pointing to a value of the configuration constructed using the options we gave allow_upload/3. This means that we can add a file upload field called :image to our form, and LiveView will track the progress of files uploaded via the field within socket.assigns.uploads.image.

You can call allow_upload/3 multiple times with different upload names, thus allowing any number of file uploads in a given live view or component. For example, you could have a form that allows a user to upload a main image, a thumbnail image, a hero image, and more.

Now that we've set up our uploads state, let's take a closer look at the :image upload configuration.

Upload Configurations in Phoenix LiveView

The :image upload config looks something like this:

#Phoenix.LiveView.UploadConfig<
  accept: ".jpg,.jpeg,.png",
  entries: [],
  errors: [],
  max_entries: 1,
  max_file_size: 8000000,
  name: :image,
  progress_event: #Function<1.71870957/3 ...>,
  ref: "phx-FlZ_j-hPIdCQuQGG",
  ...
>

Notice that it contains the configuration options we passed to allow_upload/3: the accepted file types list and file formats.

It also has an attribute called :entries, which points to an empty list. When a user uploads a file for the :image form field, LiveView will automatically update this list with the file upload entry as it completes.

Similarly, the :errors list starts out empty and is automatically populated by LiveView with any errors from an invalid file upload entry.

In this way, the LiveView framework does the work of performing the file upload and tracking its state for you. We'll see both of these attributes in action in a bit.

Now that we've allowed uploads in our component, we're ready to update the template with the file upload form field.

Render the File Upload Field

You'll use the Phoenix.LiveView.Helpers.live_file_input/2 function to generate the HTML for a file upload form field. Here's a look at our form component template:

<%= f = form_for @changeset, "#",
          id: "product-form",
          phx_target: @myself,
          phx_change: "validate",
          phx_submit: "save" %>

  <%= label f, :name %>
  <%= text_input f, :name %>
  <%= error_tag f, :name %>

  <%= label f, :description %>
  <%= text_input f, :description %>
  <%= error_tag f, :description %>

  <%= label f, :unit_price %>
  <%= number_input f, :unit_price, step: "any" %>
  <%= error_tag f, :unit_price %>

  <%= label f, :sku %>
  <%= number_input f, :sku %>
  <%= error_tag f, :sku %>

  <% # File upload fields here: %>
  <%= label f, :image %>
  <%= live_file_input @uploads.image %>

  <%= submit "Save", phx_disable_with: "Saving..." %>
</form>

Notice the use of live_file_input/2 with an argument of @uploads.image. Remember, socket.assigns has a map of uploads. Here, we provide @uploads.image to live_file_input/2 to create a form field with the right configuration and tie that form field to the correct part of socket state. This means that LiveView will update socket.assigns.uploads.image with any new entries or errors that occur when a user uploads a file via this form input.

The live view can present upload progress by displaying data from @uploads.image.entries and @uploads.image.errors. LiveView will handle all of the details of uploading the file and updating socket assigns @uploads.image entries and errors for us. All we have to do is render the data that is stored in the socket. We'll take that on soon.

Now, we should be able to see the file upload field displayed in the browser like this:

And if you inspect the element, you'll see that the live_file_input/2 function generated the appropriate HTML:

You can see that the generated HTML has the accept=".jpg,.jpeg,.png" attribute set, thanks to the options we passed to allow_upload/3.

LiveView also supports drag-and-drop file uploads. All we have to do is add an element to the page with the phx-drop-target attribute, like this:

<%= f = form_for @changeset, "#",
          id: "product-form",
          phx_target: @myself,
          phx_change: "validate",
          phx_submit: "save" %>

  # ...

  <div phx-drop-target="<%= @uploads.image.ref %>">
    <%= live_file_input @uploads.image %>
  </div

  # ...
</form>

We give the attribute a value of @uploads.image.ref. This socket assignment is the ID that LiveView JavaScript uses to identify the file upload form field and tie it to the correct key in socket.assigns.uploads. So now, if a user clicks the "choose file" button or drags-and-drops a file into the area of this div, LiveView will store the file info in the socket.assigns.uploads assignment, under the name of the specified upload, in that upload's :entries list.

As with other form interactions, LiveView automatically handles the client/server communication. When the user submits the form, LiveView's JavaScript will first upload the file(s) and then invoke the handle_event/3 callback for the form's phx-submit event. To process the file upload, this event handler will need to consume the file upload stored in socket.assigns.uploads.image.entries. Let's do that now.

Consume Uploaded Entries

Our handle_event/3 function for the phx_submit: "save" form event will use LiveView's consume_uploaded_entry/3 function to process the uploaded file. For now, we'll have our function write the uploaded file to our app's static assets in priv/static/images. This is so that we can display it on the product show template later on.

Here's our code:

defmodule ArcadeWeb.ProductLive.FormComponent do
# ...
  def handle_event("save", product_params, socket) do
    file_path =
      consume_uploaded_entry(socket, :image, fn %{path: path}, _entry ->
        dest = Path.join("priv/static/uploads", Path.basename(path))
        File.cp!(path, dest)
        Routes.static_path(socket, "/uploads/#{Path.basename(dest)}")
      end)

    product = save_product(Map.put(product_params, :image_upload, file_path)

    {:noreply,
      socket
      |> push_redirect(to: Routes.product_show_path(socket, :show, product))}
  end
end

We save the image to static assets and return the file path from consume_uploaded_entry/3. Then, we call a helper function — save_product/1 (not pictured here) — to update the product with the given form params, including the new :image_upload attribute set to our new file path. Finally, we redirect to the Product Show page.

To see our code in action, let's add some markup to the product show template to display image uploads. Then, we'll try out our feature.

Display Image Uploads

Open up lib/arcade_web/live/product_live/show.html.leex and add the following markup to display the uploaded image or a fallback:

<article class="column">
   <img
      alt="product image" width="200" height="200"
      src="<%=Routes.static_path(
              @socket,
              @product.image_upload || "/images/default-thumbnail.jpg")%>">
  </article>
<!-- product details... -->

Perfect. Now, we can test drive it. Visit /products/1/edit and upload a file:

Once you submit the form, you'll see the show page render the newly uploaded image, like this:

We did it! The LiveView framework handled all of the client/server communication details that make the page interactive. LiveView performed the file upload for you and made responding to upload events easy and customizable. You only needed to tell the live view which uploads to track and what to do with uploaded files when the form is submitted. Then you added the file upload form field to the page with the view helper and LiveView handled the rest!

There's one last thing to do. Earlier, I promised reactive file uploads that share feedback with the user. Let's take a look at this now.

Display Upload Feedback in Phoenix LiveView Forms

We know that LiveView automatically updates the :entries and :errors lists in the uploads config portion of socket.assigns once the upload begins. Let's display this information in the template to give the user real-time progress tracking. The code is amazingly simple. We'll iterate over @uploads.image.entries to display the progress for each entry:

<%= for entry <- @uploads.image.entries do %>
  <p>
    <progress value={entry.progress} max="100"> <%= entry.progress %>% </progress>
  </p>
<% end %>

Here, we use the HTML progress tag to create a simple progress bar that is populated with the progress of our file upload in real-time. As LiveView's JavaScript is uploading the file for you, LiveView is updating the value of the entry's progress in socket assigns. This causes the relevant portion of the template to re-render, thereby showing the updated progress bar in real-time. LiveView handles the work of tracking the changes to the image entry's progress. All we have to do is display it.

You can use a similar approach to iterate over and display any errors stored in @uploads.image.errors. You won't have to do any work to validate files and populate errors. LiveView handles those details. All you need to do is display any errors based on the needs of your user interface. Here's a look at the code:

<%= for err <- upload_errors(@uploads.image, entry) do %>
  <p class="alert alert-danger"><%= friendly_error(err) %></p>
<% end %>

Here, we use the Phoenix.LiveView.Helpers.upload_errors/2 function to return the errors for the specified upload.

The error messages aren't very user-friendly, though. So, we'll implement a helper function, friendly_error/1, in our LiveView component that looks like this:

defmodule ArcadeWeb.ProductLive.FormComponent do
  # ...

  def error_to_string(:too_large), do: "Image too large"
  def error_to_string(:too_many_files), do: "Too many files"
  def error_to_string(:not_accepted), do: "Unacceptable file type"
end

There's more that LiveView file uploads can do. LiveView makes it easy to:

cancel an upload
upload multiple files for a given upload config
upload files directly from the client to a cloud provider

and more.

Check out the LiveView file upload documentation for details.

Wrap-up: Build Complex Forms Easily with Phoenix LiveView

LiveView enables reactive file uploads right out of the box. Without writing any JavaScript, or even any custom HTML, you can build interactive file upload forms directly into your live view.

LiveView handles the details of client/server communication and upload state management, leaving you on the hook to write a very small amount of custom code specifying how your uploads should behave and how uploaded files should be saved.

This is a pattern you'll see again and again in LiveView — the framework handles the communication and state management details of our SPA, and we can focus on writing application-specific code to support our features.

Now that you've had a glimpse of what LiveView can do with form uploads, you're ready to build complex, interactive forms that support real-time uploads in the wild. Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Sophie is a Senior Engineer at GitHub, co-author of Programming Phoenix LiveView, and co-host of the BeamRad.io podcast. She has a passion for coding education. Historically, she is a cat person but will admit to owning a dog. You can find her on Twitter or check out her blog.

Real-Time Form Validation with Phoenix LiveView

Sophie DeBenedetto — Tue, 05 Oct 2021 11:44:01 +0000

LiveView is a compelling choice for building modern web apps. Built on top of Elixir's OTP tooling and leveraging WebSockets, it offers super-fast real-time, interactive features alongside impressive developer productivity.

LiveView keeps the developer's mind firmly rooted on the server-side, even when testing and debugging. This can empower you to deliver interactive features in single-page apps faster than ever before. Some of the most common interactions on the web are form validation and submission. These days, users expect to see form feedback presented to them in real-time, and LiveView offers first-class support for exactly that.

In this post, I'll show you how to build LiveView forms that validate changes and provide feedback to the user in real-time. Along the way, you'll learn how to model change in your Phoenix application with schemaless changesets and compose LiveView code to handle that change.

The Feature: Adding a Form to a Phoenix LiveView App

The form examples we'll be looking at in this post are inspired by the "Forms and Changesets" chapter in my book, Programming LiveView, co-authored with Bruce Tate. Check it out for an even deeper dive into LiveView testing and so much more.

We'll focus on that last piece of functionality. A user can invite a friend to play a game with them by filling out a form with the friend's email address. This will email a link to the recipient's email address so they can join the game.

Our form will need to implement some validation — namely, to ensure that a valid email address is provided. It should show any validation errors, as well as the results of a successful form submission in real-time. We won't worry too much about the exact details of sending emails for now. Instead, we'll keep our focus on the form validations in LiveView.

We'll begin where you'll almost always want to start when building a new feature in a Phoenix application — in the application core. We'll model game invitations in their module. You'll build a boundary layer in a Phoenix context module that we'll call on in LiveView later to validate form input and send invitation emails.

Let's get started!

Model Change in Phoenix with Ecto Changesets

We're almost ready to start writing code. But first, let's think about the role that forms and changesets play in our Phoenix application.

Consider Ecto changesets. Changesets are policies for changing data, and they:

Cast unstructured user data into a known, structured form — most commonly, an Ecto database schema, ensuring data safety.
Capture differences between safe, consistent data and a proposed change, allowing for efficiency.
Validate data using known consistent rules, ensuring data consistency.
Provide a contract for communicating error states and valid states, ensuring a common interface for change.

def changeset(game, attrs) do
  game
  |> cast(attrs, [:name, :description, :unit_price, :sku])
  |> validate_required([:name, :description, :unit_price, :sku])
  |> unique_constraint(:sku)
end

The changeset/2 function captures differences between the structured game and the unstructured attrs.

Then, with cast/4, the changeset trims the attributes to a known field list and converts them to the correct types. This ensures safety by guaranteeing that you won't let any unknown or invalid attributes into your database.

Finally, the validate/2 and unique_constraint/2 functions validate the inbound data, ensuring consistency.

The result is a data structure with known states and error message formats, ensuring interface compatibility.

Consequently, any forms that use this changeset know exactly how to behave — validating form input and presenting errors in accordance with the changeset's rules.

In this post, we're going to shift off of this well-known path of generated changesets. You'll see just how versatile changesets can be when it comes to modeling changes to data, with or without a database. You'll build a custom, schemaless changeset for data that isn't backed by a database table, and you'll use that changeset in a form within a live view.

Schemaless Changesets in the Phoenix Application Core

You've likely used changesets to model changes to data that is persisted in your database. Our game invitation feature doesn't require database persistence, however. A user will provide the email address of the friend they'd like to invite, and our application will simply send out the email with the link to the game. We don't need to store that invitee's data at this point.

Luckily, we can use schemaless changesets to model data that doesn't get saved to the database. A schemaless changeset is based on a simple Elixir map or struct, rather than an Ecto schema-backed struct. The only difference is that, when working with a plain Elixir struct, we need to provide the changeset with the type of information that Elixir would normally handle. We'll see this in action in a moment.

First, let's define the core module that we'll use to model a game Recipient. Create a new file, arcade/lib/game/Recipient.ex, and define the module like this:

defmodule Arcade.Invite.Recipient do
  defstruct [:name, :email]
end

Our module is simple so far. It implements a struct with two keys, :name and :email.

Next up, we need to give our module awareness of the types that will be considered valid by any changeset we create. Let's use a module attribute to store this map of types so that we can access it later:

defmodule Arcade.Invite.Recipient do
  defstruct [:name, :email]
  @types %{game_name: :string, email: :string}
end

Now, we'll alias the module and import Ecto.Changeset so that we can use the changeset functions:

defmodule Arcade.Invite.Recipient do
  defstruct [:name, :email]
  @types %{game_name: :string, email: :string}

  alias Arcade.Invite.Recipient
  import Ecto.Changeset
end

Finally, we're ready to define the changeset/2 function that will be responsible for casting recipient data into a changeset and validating it:

defmodule Arcade.Invite.Recipient do
  # ...
  def changeset(%Recipient{} = invitation, attrs) do
    {invitation, @types}
    |> cast(attrs, Map.keys(@types))
    |> validate_required([:game_name, :email])
    |> validate_format(:email, ~r/@/)
  end
end

We validate the presence of the :game_name and :email attributes, and then validate the format of :email.

Now, we can create recipient changesets like this:

iex> alias Arcade.Invite.Recipient
iex> i = %Recipient
iex> Recipient.changeset(r, %{email: "juniper@email.com", game_name: "Chess"})
#Ecto.Changeset<...valid?: true>

Let's see what happens if we try to create a changeset with an attribute of an invalid type:

iex> Recipient.changeset(r, %{email: "juniper@email.com", game_name: 1234})
#Ecto.Changeset<errors: [game_name: {"is invalid", ...]}],valid?: false>

Ecto.Changeset.cast/4 relies on @types to identify the invalid type and provide a descriptive error.

Next, try a changeset that breaks one of the custom validation rules:

iex> Recipient.changeset(r, %{email: "juniper's email", game_name: "Chess"})
#Ecto.Changeset<changes: %{email: "juniper's email", ...},
  errors: [email: {"has invalid format", ...}],valid?: false>

This function successfully captures our change policy in code, and the returned changeset tells the user exactly what is wrong.

Now that our changeset is up and running, let's quickly build out an Invite context that will present the interface for interacting with the changeset.

The Boundary Layer in Elixir

Create a file, lib/arcade/invite.ex and add in the following:

defmodule Arcade.Invite do
  alias Arcade.Invite.Recipient

  def change_invitation(%Recipient{} = recipient, attrs \\ %{}) do
    Recipient.changeset(recipient, attrs)
  end

  def send_invite(recipient, attrs) do
    # send email to promo recipient
  end
end

This context is a beautifully concise boundary for our service. The change_invitation/2 function returns a recipient changeset, and send_invite/2 is a placeholder for sending a game invite email.

Other than the internal tweaks we made inside Recipient.changeset/2, building a Phoenix context module with a schemaless changeset looks identical to building an Ecto-backed one. When all is said and done, in the view layer, schemaless changesets and schema-backed ones will look identical.

Let's turn our attention to the view layer now and build out our live view.

Build and Define the Live View

This live view will have the feel of a typical live view with a form. First, we'll create a simple route and wire it to a live view. Next, we'll use our Invite context to produce a schemaless changeset, and add it to the socket within a mount/3 function. We'll render a form with this changeset and apply changes to the changeset by handling events from the form.

Let's get going!

Create a file, lib/arcade_web/live/invite_live.ex and fill in the following:

defmodule ArcadeWeb.InviteLive do
  use ArcadeWeb, :live_view
  alias Arcade.Invite
  alias Arcade.Invite.Recipient

  def mount(_params, _session, socket) do
    {:ok, socket}
  end
end

We pull in the LiveView behavior, alias our modules for later use and implement a simple mount/3 function.

Let's use an implicit render/1. Create a template file in lib/arcade_web/live/invite_live.html.leex, starting with some simple markup:

<h2>Invite a Friend to Play!</h2>
<h4>
  Enter the name of a game and your friend's email below and we'll send them an
  invite to join you in playing a game!
</h4>

Now, let's define a live route and fire up the server. In the router, add the following route behind authentication:

scope "/", ArcadeWeb do
  pipe_through [:browser, :require_authenticated_user]
  live "/invite", InviteLive
end

With that, you should be able to fire up the Phoenix server, point your browser at /invite and see your live view render the invitation template:

As the live view is up and running, we're ready to build out the form for an invitation recipient.

Render the Phoenix LiveView Form

We'll use mount/3 to store a recipient struct and a changeset in the socket:

 def mount(_params, _session, socket) do
  {:ok,
    socket
    |> assign_recipient()
    |> assign_changeset()}
end

def assign_recipient(socket) do
  socket
  |> assign(:recipient, %Recipient{})
end

def assign_changeset(%{assigns: %{recipient: recipient}} = socket) do
  socket
  |> assign(:changeset, Invite.change_recipient(recipient))
end

The mount/3 function uses two helper functions, assign_recipient/1 and assign_changeset/1, to add a recipient struct and a changeset for that recipient to socket assigns. These pure, single-purpose reducer functions are reusable building blocks for managing the live view's state.

Remarkably, the schemaless changeset can be used in our form exactly like database-backed ones. We'll use socket.assigns.changeset in the template's form, like this:

<%= f = form_for @changeset, "#",
  id: "invite-form",
  phx_change: "validate",
  phx_submit: "save" %>

  <%= label f, :game_name %>
  <%= text_input f, :game_name  %>
  <%= error_tag f, :game_name %>

  <%= label f, :email %>
  <%= text_input f, :email%>
  <%= error_tag f, :email %>

  <%= submit "Send Invite"%>
</form>

Now, if you point your browser at /invite, you should see this:

Our form implements two LiveView bindings, phx-change, and phx-submit. Let's dig into these events now.

Handle Form Events in LiveView

We'll start with a look at the phx-change event. LiveView will send a "validate" event each time the form changes and include the form params in the event metadata.

So, we'll implement a handle_event/3 function for this event that builds a new changeset from the params and adds it to the socket:

# lib/arcade_web/live/invite_live.ex
def handle_event(
      "validate",
      %{"recipient" => recipient_params},
      %{assigns: %{recipient: recipient}} = socket) do
  changeset =
    recipient
    |> Invite.change_recipient(recipient_params)
    |> Map.put(:action, :validate)

    {:noreply,
    socket
    |> assign(:changeset, changeset)}
end

Let's break this down. The Invite.change_recipient/2 context function creates a new changeset using the recipient from socket state and the params from the form change event.

Then, we use Map.put(:action, :validate) to add the validate action to the changeset, a signal that instructs Phoenix to display errors. Phoenix will not display the changeset's errors otherwise.

When you think about it, this approach makes sense. Not all invalid changesets should show errors on the page. For example, the empty form for the new changeset shouldn't show any errors because the user hasn't provided any input yet. So, the Phoenix form_for function needs to be told when to display a changeset's errors. If the changeset's action is empty, then no errors are set on the form object — even if the changeset is invalid and has a non-empty :errors value.

Finally, assigns/2 adds the new changeset to the socket, triggering render/1 and displaying any errors.

Let's take a look at the form tag that displays those errors on the page. Typically, each field has a label, an input control, and an error tag, like this:

<%= label f, :email %>
<%= text_input f, :email%>
<%= error_tag f, :email %>

The error_tag/2 Phoenix view helper function displays the form's errors for a given field on a changeset, when the changeset's action is :validate.

Now, point your browser at /invite and fill out the form with a game name and an invalid email. As you can see in this image, the UI updates to display the validation errors:

That was surprisingly easy! We built a simple and powerful live view with a reactive form that displays any errors in real-time.

The live view calls on the context to create a changeset, renders it in a form, validates it on form change, and then re-renders the template after each form event. We get reactive form validation for free without writing any JavaScript or HTML. We let Ecto changesets handle the data validation rules, and we let the LiveView framework handle the client/server communication for triggering validation events and displaying the results.

There's just one thing about our form validation that needs some improvement. You'll notice that as soon as you start typing into the email form, an error appears because our validation event fires whenever the form field changes. This doesn't present our users with the best experience — we're telling them there is an error with their input before they get a chance to finish typing their full email.

Instead, we want the validation event to fire when a user clicks away or blurs from the email input. Luckily for us, LiveView makes it easy to implement this functionality with the help of the phx-debounce binding. Update your email form field to look like this:

<%= text_input f, :email, phx_debounce: "blur" %>

Now the "validate" event will only fire when a user blurs away from the email input field, and we won't show any premature validation error messages.

Learn more about LiveView's support for debouncing and other rate-limiting options.

As you might imagine, the phx-submit event works very similarly to phx-change. The "save" event fires when the user submits the form. To respond to this event, we can implement a handle_event/3 function that uses the (currently empty) context function, Invite.send_invite/2. The context function should create and validate a changeset.

If the changeset is, in fact, valid, we can pipe it to some helper function or service that handles the details of sending invitation emails.

If the changeset is not valid, we can return an error tuple. Then we can update the UI with a success or failure message accordingly. I'll leave you to practice what you're learning by implementing this behavior on your own.

Wrap-up: LiveView — A Great Choice to Build and Validate Forms in Elixir

Now you've seen that Ecto changesets are not tightly coupled to the database. Schemaless changesets let you tie backend services to Phoenix forms any time you require validation and security, whether or not your application needs to access a full relational database.

LiveView supports custom integration of forms to backend code with these schemaless changesets. To do so, you need only provide the Changeset.cast/4 function with the first argument of a two-tuple holding both data and type information. This type of code is ideal for implementing form scenarios requiring validation but without the typical database backend.

Whether you're working with schema-backed or schemaless changesets, LiveView provides real-time form validation and feedback, with very little hand-rolled code. We can use LiveView event bindings to handle form validation and submission in real-time with a few simple event handlers that call out to our nice, clean Phoenix context code.

With that, you have everything you need to build basic forms in LiveView. To dig deeper into LiveView's rich forms offerings, check out the docs. Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

DEV Community: Sophie DeBenedetto

What's New in Elixir 1.16

Better Diagnostics for Compiler Errors in Elixir

Syntax Errors

Mismatched Delimiter Errors

Other Errors

ExDoc Content Matures with Elixir

Anti-Patterns

ExDoc Cheatsheets In the Official Elixir Docs

More Enhancements

The String.replace_invalid/2 Function

New Functionality for Task.yield_many/2

The Logger.levels/0 Function

Wrap Up

Observe Your Phoenix App with Structured Logging

What is Structured Logging?

Logger Configuration in Phoenix

Using Logfmt in Phoenix Development

Using Logfmt with the AppSignal Logger in Production

Framework Vs. Application Logs

Build A Custom Logger Backend

Correlating Log Events and Other Telemetry with AppSignal

Monitoring Your Logs Using AppSignal

Logging in Phoenix: A Few Final Tips

Wrap Up

Sanitize Strings in Elixir with Pattern Matching and Recursion

The Problem: Strings with Unicode Specials

The Solution: Sanitize Your String with Pattern Matching and Recursion in Elixir

Step 1: Break Down the Problem

Step 2: Define the Module in Elixir

Step 3: Define the unicode_only/2 Function in Elixir

Wrap Up

Phoenix LiveView 0.18: New Special HTML Attributes

What are Special HTML Attributes in LiveView 0.18?

Cleaner LiveView Templates with :if and :for

Dynamic Function Components with :let

Wrap Up

Fix Process Bottlenecks with Elixir 1.14's Partition Supervisor

The Problem: Dynamic Supervisor Bottlenecks in Your Elixir App

The Slow Worker and Its Dynamic Supervisor

Demo: The Bottleneck

The Fix: Supervise a Fleet of Dynamic Supervisors with a Partition Supervisor

Using PartitionSupervisor in the Dynamic Supervisor Module

Demo: No More Bottleneck

PartitionSupervisor in Elixir 1.14: Under the Hood

Partitioning Dynamic Supervisors

Routing Requests to Dynamic Supervisors

Wrap Up

Elixir 1.14: Better Debugging with dbg/2 and More

Debugging with Kernel.dbg/2 in Elixir

Debugging the Current Binding

Debugging Provided Code

Debugging with dbg/2 and IEx in Elixir

More dbg/2 Goodies

Debugging Pipelines with dbg/2

Configuring dbg/2

Inspect Improvements

Better Binary Evaluation Errors

Wrap Up

Phoenix LiveView Under The Hood: The Form Function Component

What Are Function Components in Phoenix LiveView?

Calling the form/1 Function Component

The form/1 Component Under the Hood

Creating the Phoenix.Form

Rendering the HEEx Template

Render the Inner Block with Phoenix Component Slots

Wrap Up: Demystifying Phoenix LiveView's Form Function Component

Securing Your Phoenix LiveView Apps

Using Live View to Build a Phoenix Web App

Protect Sensitive Routes in Your Phoenix LiveView App

Group Live Views in a Live Session

Grouping Live Routes: The Security Problem

Protect Live Views When They Mount

Wrap Up: Easily Group Live Views to Secure Your Phoenix LiveView App

Build Interactive Phoenix LiveView UIs with Components

The Feature: Compose a User Survey UI for a Phoenix LiveView App

Define the Parent Live View

Define the Survey Route

Mount the Survey Live View

Render the Survey Live View

The `String.replace_invalid/2` Function

New Functionality for `Task.yield_many/2`

The `Logger.levels/0` Function

Step 3: Define the `unicode_only/2` Function in Elixir

Cleaner LiveView Templates with `:if` and `:for`

Dynamic Function Components with `:let`

Using `PartitionSupervisor` in the Dynamic Supervisor Module

`PartitionSupervisor` in Elixir 1.14: Under the Hood

Debugging with `Kernel.dbg/2` in Elixir

Debugging with `dbg/2` and IEx in Elixir

More `dbg/2` Goodies

Debugging Pipelines with `dbg/2`

Configuring `dbg/2`

Calling the `form/1` Function Component

The `form/1` Component Under the Hood

Creating the `Phoenix.Form`