DEV Community: arathunku

FIT parsing and code generation in Elixir

arathunku — Mon, 19 Feb 2024 13:00:00 +0000

Flexible and Interoperable Data Transfer (FIT) developed by Garmin is a format used to store activity/workout files. Garmin provides SDKs in many languages, and there are even more various libraries that are based on its specification, some open source, some not. The specification consists of this FIT Protocol page and a huge spreadsheet in downloadable SDK describing types and messages the FIT file may contain.

At the end of last year, I’ve started implementing a FIT parser in pure Elixir. Excellent Erlang binary pattern matching made it very pleasant to do and at this moment, the decoder can parse many FIT files. Some encoders don’t fully follow the specification to the dot. I’ve scavenged various FIT files from issue reports, examples in SDKs and my own years of running. I wanted to fully support anything the file may contain - components, dev fields, subfields and it does! Mostly matching what official fit2csv tool is outputting.

This post is about a specific aspect of being able to even start parsing a FIT file, “Profile”. “Profile” is based on the spreadsheet in SDK. It describes all the various types of fields, records, their type, scales, offsets, bit offsets and more. This spreadsheet is also used to generate all encoders and decoders in official SDK for C, C++, Java, Python, JavaScript and more. Such profile in official SDK in JavaScript is whooping 25k (yes, k) lines of code, similar amount in Python.

I also explored fitparser in Rust (almost 30k lines), and fit in Go(only 1.5k lines!). In my extremely naive attempt in Elixir so far, my profile is around 45k lines, that’s a lot of lines.

Taking shortcuts with `inspect/2`

The very basic and simplified input for codegen looks somewhat like this:

types = [
 ["activity", "enum", [["manual", 0], ["auto_multi_sport", 1]]],
 [
 "activity_class",
 "enum",
 [
 ["level_max", 100],
 ["level", 127],
 ["athlete", 128, "0 to 100"]
 ]
 ],
 ["workout_hr", "uint32", [["bpm_offset", 100]]],
 ["weight", "uint16", [["calculating", 65534]]]
 # ... 2000 more types
]

In reality, it’s nowhere like this and it’s coming from xlsx file, but it should be enough to imagine what we’re working with.

Each element of the list is a field with its type and values. Example, for the first element, activity field is always 0 or 1 and each values corresponds to specific label after parsing - manaul or auto_multi_sport in this case. In FIT file there actually won’t be any activity string but something like 51 which is described by another field… It’s fun and efficient binary format! FIT Protocol doc.

In my naive attempt to get the job done, I’ve used Elixir’s Inspect protocol that’s implemented for all the data types. It can be used to dump data structs into strings and read them back later, matching the original data structure. And as you’ll see below, it mostly works.

iex> field = %{type: "uint32", name: "distance"}
%{name: "distance", type: "uint32"}
iex> inspect(field) |> Code.eval_string()
{%{name: "distance", type: "uint32"}, []}
iex> IO.puts(inspect(field))
%{name: "distance", type: "uint32"}

If you try to do that for a very large data structure (like the one in FIT’s spreadsheet), inspect will truncate some data, this is easily fixed by adding limit: :infinity to inspect/1 call.

We also need to be on a lookout for array of integers which may look awfully a lot like a charlist, or maybe… a charlist looks like an array of integers… all the same. Let’s ensure it stays consistent, with the charlists option.

iex> inspect([?h, ?e, ?l, ?l, ?o, 112])
"~c\"hellop\""
iex> inspect([?h, ?e, ?l, ?l, ?o, 255])
"[104, 101, 108, 108, 111, 255]"
iex> inspect([?h, ?e, ?l, ?l, ?o], charlists: :as_lists)
"[104, 101, 108, 108, 111]"

It’s of course, all the same.

iex> 'hello' == [?h, ?e, ?l, ?l, ?o]
true

This approach with dumping data structs into profile.ex worked! My parser uses successfully the module with all the types and messages from the spreadsheet, but…

It’s really annoying to depend on inspect/2. I cannot, without additional options, define custom inspect for many structs, which makes debugging harder when everything is extremely verbose and structs contain many fields.

Another big disadvantage is that while it may work for data structs, I cannot easily dump references to functions or call other modules.

iex> f = & &1
#Function<42.3316493/1 in :erl_eval.expr/6>
iex> inspect(f)
"#Function<42.3316493/1 in :erl_eval.expr/6>"
iex> f.(1)
1
iex> inspect(f) |> Code.eval_string()
{nil, []}

I needed a better approach, and I didn’t want to step back to dumping elixir-like strings into profile.ex while deeply traversing the data structure. I wanted to use the power of Elixir, macros! After all, they can be used to generate other Elixir code.

Switch to macros

There’s a lot to Elixir macros, I still don’t fully grasp all the intricate details about them, but maybe it’s for the best. They’re very powerful. Official docs, and more importantly, Understanding Elixir Macros series by Saša Jurić, were extremely helpful in making my prototype work. They can be used to capture AST for any Elixir code and then dump it into a file.

How do we switch from inspect to macros? Easy!

#### instead of this
iex> field = %{type: "uint32", name: "distance"}
#### do
iex> field = quote(do: %{type: "uint32", name: "distance"})
{:%{}, [], [type: "uint32", name: "distance"]}
iex> field |> Macro.to_string()
"%{type: \"uint32\", name: \"distance\"}"
#### or
iex> field = %{type: "uint32", name: "distance"}
iex> Macro.escape(field) |> Macro.to_string()
"%{name: \"distance\", type: \"uint32\"}"

This works for creating modules too! Elixir’s macros is still a proper AST. It works with syntax highlighting and autocomplete and any other Elixir code, except for comments, they’re not part of AST.

field = quote(do: %{type: "uint32", name: "distance"})
quote do
 defmodule Exgen.Profile.ExampleField do
 @field unquote(field)

 def field, do: @field
 end
end
|> Macro.to_string()
|> IO.iodata_to_binary()
|> IO.puts()

# results in

defmodule Exgen.Profile.ExampleField do
 @field %{name: "distance", type: "uint32"}
 def field do
 @field
 end
end

This brings us closer to proper code generation for profile. Macros also allow us to properly generate method calls!

iex> field = %{type: quote(do: Exgen.Types.by_name("uint32")), name: "distance"}
iex> Macro.escape(field) |> Macro.to_string() |> IO.puts()
%{
 name: "distance",
 type: {
 {:., [], [
 {: __aliases__ , [alias: false], [:Exgen, :Types]},
 :by_name
 ]}, [], ["uint32"]
 }
}

…or not? That’s not right. Our type quote with the function call gets escaped twice. To fix this, in this specific case:

iex> Macro.escape(%{field | type: {:unquote, [], [field.type]}}, unquote: true)
iex> v |> Macro.to_string() |> IO.puts()
%{name: "distance", type: Exgen.Types.by_name("uint32")}

That’s much better! In the case of profile generation, it’s pretty straightforward to know what needs to be marked with :unquote for proper code generation. For a more general approach, it should be possible to deeply walk the data structure and mark all quote with :unquote if needed.

Formatting

To tighten it all up, the whole code generation, there’s one last thing to make it pretty - formatting. We don’t want to have one massive unreadable line. Fear not, Elixir has a built-in formatter, mix format , and it’s not only available in CLI but also as a method call - Code.format_string/2. Before the final dump into a file, it can be used to nicely format all the generated code in any way you want.

Conclusion

The approach with macros is much more readable, testable and less error-prone. If something is wrong with syntax, it’s more obvious in macro than in a combination of string with inspect, defmodule Profile do #{inspect(types)} end.

One last thing I’m still missing is how to attach comments. Elixir macros don’t have comments in AST, formatter handles them with Code.string_to_quoted_with_comments/2. I don’t see any easy way to attach them to AST from macro, but in the end, it may not be needed. I want to experiment with a switch to methods with @doc annotations and creating proper named structs for each type, instead of generic one with name: attribute.

The generated profile is now a lot of smaller and it’s still in progress due to above fiddle with @doc. At this moment rest of the parser is still in a rough shape, missing some niceties like CRC calculation or speed (1st make it work!). When this is resolved, I plan to release it as open source library and maybe get some help with adding encoder for it. It would be awesome to be able to decode, edit and encode back FIT files.

My favorite Elixir macro is dbg/2, or maybe def/2… It’s all macros! 😱

Shell scripting with Elixir

arathunku — Sun, 11 Feb 2024 20:00:00 +0000

When simple bash scripts start to become unwieldy, you may consider reaching out for something “higher” level, like Perl, Ruby or Python. I’m reaching out for Elixir. Maybe the startup times are not perfect for every use case, but Elixir is extremely versatile. It’s easy to add dependencies, debug, iterate and even write tests, all in a single file! I believe, due to LiveBook, it really hits its stride in recent years; the ecosystem leans heavily into ergonomic developer experience and great out-of-the-box defaults (Req is amazing!). In few lines of code, you can connect to Postgres, send HTTP requests or start HTTP server.

For my own use cases I’ve written scripts for displaying weather(focus only on rain, v. important for dog walks when your dog hates rain) in i3status-rust, transforming various CSV files from bank exports into beancount format, or creating Telegram bots.

A simple script may be just:

#!/usr/bin/env elixir
IO.puts("Hello world")

Well, if it were that simple, I’d just do echo "Hello world" and skip Elixir, but then there’d be no point in writing this blog post, right? So, I have another, a bit more involved and what may seem like over-complicated template. It’s a starting point for more complex scripts, and I can either remove some of the parts I don’t need or start extending it.

The template

#!/usr/bin/env -S ERL_FLAGS=+B elixir
Mix.install([])

if System.get_env("DEPS_ONLY") == "true" do
 System.halt(0)
 Process.sleep(:infinity)
end

defmodule Hello do
 @moduledoc """
 <!-- TODO -->

 ## Usage

 $ bin/hello --help

 """

 @args [help: :boolean]
 def main(args) do
 {parsed, args} = OptionParser.parse!(args, strict: @args)
 cmd(parsed, args)
 end

 defp cmd([help: true], _), do: IO.puts(@moduledoc)
 defp cmd(_parsed, _args) do
 IO.puts(@moduledoc)
 System.stop(1)
 end
end

Hello.main(System.argv())

First of all, the shebang is not the usual thing you’d expect. It configures an additional flag for erl. This flag, per docs, “De-activates the break handler for (SIGINT) ^C and ^\ “. I’ll expand on it in Signals section.

Now, we’re executing, and there’s a ready-to-go Mix.install/2 statement where we can add additional dependencies. To make HTTP requests, we can add battries-included HTTP client like {:req, "~> 0.4"}. Processing JSON? {:jason, "~> 1.4"}. We can search for packages with mix hex.search [package-name] or lookup the latest version mix hex.info [package-name]

$ mix hex.info req
Req is a batteries-included HTTP client for Elixir.

Config: {:req, "~> 0.4.8"}
Releases: 0.4.8, 0.4.7, 0.4.6, 0.4.5, 0.4.4, 0.4.3, 0.4.2, 0.4.1, ...

Licenses: Apache-2.0
Links:
 Changelog: https://hexdocs.pm/req/changelog.html
 GitHub: https://github.com/wojtekmach/req

Another unusual block is if System.get_env("DEPS_ONLY") do ... end. If the script doesn’t have any dependencies, I’d just delete it or let it be. It’s useful in cases when the script has dependencies. We can use this block to cache dependencies and compilation of the script, skipping the execution of the rest of the script. This is handy for CI setups or when building container images. For CI, I’d also define a directory where the dependencies should be cached. for GitLab CI, a job may looks like this:

print-hello:
 # https://hub.docker.com/r/hexpm/elixir/tags?page=1
 image: hexpm/elixir:1.16.1-erlang-26.2.2-debian-bookworm-20240130-slim
 variables:
   MIX_INSTALL_DIR: "$CI_PROJECT_DIR/.cache/mix"
 cache:
   - key: elixir-cache
 paths:
   - .cache
 script:
   - DEPS_ONLY=true bin/hello
   - bin/hello

After that, there’s a module where we’ll be documenting what the script is about, defining options for parsing arguments and failing gracefully when invalid options are passed, ensuring proper error exit code. That’s also where we’d extend the script, before last cmd/2. CLI argument parsing is done with built-in OptionParser.

This structure may seem a bit verbose and may look like a lot of boilerplate, but again, if it were simple, we’d have just stayed with the bash in the first place. Here, this structure can easily grow with the script.

I used to define inline functions like print_help = fn -> ... end or process_args = fn (args) -> .. end but in the end, working within a module is cleaner, and no need to look if given function is anonymous function (.() call) or module’s function.

With the template in place, we’re ready to add some logic to it. Elixir can do quite a bit just with the standard library, but there are also some gotchas. Let’s go through some common needs.

Output

IO will probably be the most often used module. It can be used to write stdout with functions like IO.write/1, IO.puts/1, or to stderr with their equivalent 2-argument calls like IO.puts(:stderr, "Error"). We can also read inputs with IO.read/2. Any writing or reading can be also handled as a stream with IO.stream/2.

IO.puts("Hello")
IO.puts(:stderr, "Invalid argument")

IO.stream(:stdin, :line) # that's the default
|> Enum.map(&String.trim_trailing(&1, "\n"))
|> Enum.map(&String.reverse/1)
|> Enum.map(&IO.puts/1)

Colors

There’s no need to define color codes manually. With IO.ANSI, we can add text and background colors easily.

iex(1)> IO.ANSI.blue_background()
"\e[44m"
iex(2)> v <> "Example" <> IO.ANSI.reset()
"\e[44mExample\e[0m"
iex(3)> v |> IO.puts()
Example
:ok

Exit code

Another quick one, there’s System.stop(exit_code) to gently shutdown VM, what may not be obvious is that it’s async process. Make sure to call Process.sleep(:infinity) after it to block the execution. This ensures that all the applications are taken down gently. There’s alternative of System.halt/1 and it forces immediate shutdown of Erlang runtime system.

Subprocesses

For one-off commands, System.cmd/3 is enough.

iex(1)> {output, 0} = System.cmd("git", ["rev-parse", "--show-toplevel"])
{"/home/arathunku/code/github.com/arathunku/elixir-cli-template-example\n", 0}

With pattern matching we ensure immediate exit if there’s any other exit code than the successful one, and we can process the output.

It gets more tricky if you want to create another BEAM process while continuing with the rest of the execution. If something goes wrong or Erlang system crashes, OS process might get left behind. In these cases, instead of reinveting a wheel of managing OS processes, it’s good occasion to make a use of this Mix.install/2 at the beginning and add MuonTrap. It will ensure the processes are, as described in README, well-behaved.

Mix.install([{:muontrap, "~> 1.0"}])

defmodule Hello do
  ...
  defp cmd([], []) do
    _pid = spawn_link(fn ->
      MuonTrap.cmd("ping", ["-i", "5", "localhost"], into: IO.stream(:stdio, :line))
      System.stop()
    end)
    Process.sleep(:infinity)
  end
end

Signals, some are not like the others

It’s tricky and it will probably not behave as you’d expect based on your experience in other languages. Some signals are handled by default by Erlang system, for more details check nice documentation in PR. For scripting… usually signals don’t matter that much, at least in my case. If you spawn a GenServer, it’ll receive terminate/2 for cleanup, assuming gentle shutdown.

We can still skip BREAK menu(^C) and exit immediately if we start with ERL_FLAGS=+B elixir. This is why it’s in the template at the beginning. Some other signals can be cough by swapping default erl_signal_server, but not all of them*. In the example above we’ll do just that, handle what we’re interested it and defer rest to the default handler.

*At this moment, INT cannot be trapped, see this issue

defmodule Signals do
 @behaviour :gen_event
 def init(_), do: {:ok, nil}

 def handle_event(:sigusr1, state) do
 IO.puts("USR1, continue...")
 {:ok, state}
 end

 def handle_event(:sigterm, _state) do
   IO.puts("Ok, ok, let me take a moment and exit...")
   Process.sleep(3)
   System.stop()
 end

 def handle_event(signal, state) do
   :erl_signal_handler.handle_event(signal, state)
 end

 def handle_call(_, state), do: {:ok, :ok, state}

 def terminate(reason, _state) do
   IO.puts("Goodbye! #{inspect(reason)}")
 end
end

with :ok <- :gen_event.swap_handler(
 :erl_signal_server, {:erl_signal_handler, []}, { __MODULE__ , []}
) do
  IO.puts("I'll wait for signals!")
  Process.sleep(:infinity)
else
 err ->
   IO.warn("Something went wrong. err=#{inspect(err)}")
end

Testing

In Rust, we can write tests next to the code with #[cfg(test)], and these will run when cargo test is executed. Did you know you can do kind of a similar thing in Elixir scripts? There’s no magic here, we need to create a test module and trigger ExUnit

if System.get_env("MIX_ENV") == "test" do
 ExUnit.start()

 defmodule HelloTest do
 use ExUnit.Case, async: true
 import ExUnit.CaptureIO

 test "prints a message when no arguments are passed" do
 assert capture_io(fn -> Tests.main([]) end) == "Hello World\n"
 end

 test "prints help for unknown arguments" do
 assert capture_io(fn -> Tests.main(["--help"]) end) =~ "Example of adding ExUnit"
 end
 end
else
 Hello.main(System.argv())
end

Going beyond scripts

OptionParser is good. Maybe it doesn’t do all the stuff that something like Rust clap does but it’s absolutely getting the job done.

You can go beyond simple CLI scripts and build full TUI apps. Progress bars? Charts? Text editor? Here, Ratatouille comes to the rescue. Simple TUI counter example or more advanced ones - more advanced examples.

If you’d like to see even more examples with Mix.install/2, make sure to check mix_install_examples! There’re examples of HTTP servers, CSV parsing, web scraping, machine learning or full Phoenix LiveView file uploader(!!!), but at this point you may consider just using mix new ... and setting up a proper Mix project. After that, you can use burrito to ship a single Elixir CLI binary for end-users.

Is Elixir more complicated than Ruby od Python as a bash replacement? It depends, of course it depends. If you already know Python or Ruby well, you’ll probably prefer them but Elixir can absolutely be used too! It’s a bit on the slow side to start up, may not be a best choice to implement PS1, but if the startup speed doesn’t matter that much and you want go have very ergonomic language for scripting at your hand - it’s great.

Closing note

Writing this article was kind of a strange experience. You may think it’s a bit light on examples and details, and it is done on purpose. There’s strong focus on documentation within the community and I didn’t want to repeat what’s already out there in official docs and libraries. Just look at this beautiful System.cmd/3 documentation or IO.ANSI docs, and it’s all available in iex with h/1.

When writing the article, I’ve dumped all my scripts and tests into this repository. Thanks for reading!

Seamless navigation between SPA and LiveView

arathunku — Mon, 05 Feb 2024 04:40:00 +0000

tl;dr

Navigating between SPA and LiveView mostly works!

Recently, many websites have become single-page applications (SPAs), written in React/Vue/Svelte/ w/e else is trendy now, sometimes needlessly, but what’s done is done. I was wondering if it would be possible to break out from this path, where everything is rendered by SPA, and extend it with Phoenix LiveView, without radically affecting the user experience or requiring a months-long rewrite.

I believe the approach and problems presented in the post should work for different tools working similarly to LiveView, like LiveWire, or Turbo, but I’ve not tested that. Similarly, here in this blog post, I’ll use React, but I’ll refer to the frontend app as SPA because it’s mostly independent of the library.

The goal of the exercise is simple, can we replace a page with LiveView?

The setup

At the very beginning, the out-of-the-box setup for many SPAs configs is that they have their own dev server, with its own index.html that’s rendered on /*. Later on deployment, if there’s no server-side rendering, everything is served as static files.

Our server is nowhere to be found on the request’s path. That’s a bummer. We need to move to the good old way, where the app’s server is doing the routing and rendering HTML on the server. Starting with index.html. Otherwise, there’s no place for our Phoenix app.

This will be different for every SPA setup (there are dozens, or maybe hundreds, ?!). For brevity and for the purposes of the exercise, I created two apps. The first one is the most typical and boring Phoenix app. The second is the Vite React template for SPA. They’re living in their own worlds, without knowing about one another, at least at this stage.

$ mix phx.new sample
$ mix phx.server
[info] Migrations already up
[info] Running SampleWeb.Endpoint with cowboy 2.10.0 at 127.0.0.1:4000 (http)
[info] Access SampleWeb.Endpoint at http://localhost:4000
[watch] build finished, watching for changes...

Rebuilding...
Done in 550ms.


$ npm create vite@latest sample -- --template react
$ cd sample
$ npm install
$ npm run dev

Extending SPA with LiveView

Vite docs include a guide for integration with backend and we’ll follow it. It’s mostly correct; it’s just a matter of putting the things in the right place with additional LiveView specific glue.

We start with defining LiveView page where we can dump all requests.

 scope "/", SampleWeb do
 pipe_through :browser

+ live "/app/*path", SpaLive
 end

SpaLive is going to have a single responsibility - show SPA. This means the SPA in DOM will be rendered inside LiveView.

defmodule SampleWeb.SpaLive do
 use SampleWeb, :live_view

 def mount(params, session, socket) do
 {:ok, socket, layout: {SampleWeb.Layouts, :spa}}
 end

 # When URLs changes, do nothing, SPA is handling the routing
 def handle_params(uri, _, socket) do
 {:noreply, socket}
 end

 def render(assigns) do
 # It's critical that div where the SPA is going to be rendered
 # is ignored with phx-update
 ~H"""
 <div id="root" phx-update="ignore" phx-hook="LoadSPA"></div>
 """
 end
end

Starting from the top, we have mount/3 with a small change, we set a layout dedicated to SPA. It simply delegates all the content to inner content and keeps a place for flash messages from server.

<main>
 <.flash_group flash={@flash} />
 <%= @inner_content %>
</main>

We do it in such a way as to leave a place for any future HTML that may be part of the layout for the page. This is useful in case we’d like to move more of the layout to server, some boring stuff like a footer, a navigation bar or a cookie prompt. We don’t change anything in root.html.heex.

Next, we have handle_params/3. It’s a no-op method because SPA is handling the routing, we don’t need to do anything here. It’s important for links using navigate, in cases where they point to SPA and only this method will be called, not mount.

Finally, render/1. We cannot simply put <script> tags in ~H block, as instructed by Vite’s backend integration guide. This would work on the initial render, but if there’s navigation between different LiveView controlled pages, these scripts won’t be executed! LiveView under the hood is using morphdom and DOM is set via innerHTML.

HTML specifies that a <script> tag inserted with innerHTML should not execute.

This is a reason why we need to take a different approach, phx-hook. We’ll use # JavaScript interoperability to load SPA.

// required by Vite
// error handling of import is left as another exercise
const setupReactPatching = (callback) => {
 if (window.$RefreshReg$) {
 callback()
 return;
 }

 import("http://localhost:5173/@react-refresh").then(RefreshRuntime => {
 RefreshRuntime.default.injectIntoGlobalHook(window)
 window.$RefreshReg$ = () => {}
 window.$RefreshSig$ = () => (type) => type
 window. __vite_plugin_react_preamble_installed__ = true
 callback()
 })
}

const LoadSPA = {
 mounted() {
 setupReactPatching(() => {
 import("http://localhost:5173/@vite/client")
 import("http://localhost:5173/src/main.jsx").then(mount => {
 this.unmount = mount.default(this.el)
 })
 })
 },

 destroyed() {
 if (this.unmount) {
 this.unmount()
 }
 }
}

export default LoadSPA;

In this hook, we follow Vite’s guide as before for backend integration and we load JS files from its own dev server.

We don’t set up any additional communication between LiveView and SPA, frontend will still send requests to server as before (if it did). If you’re interested in communication between SPA and LiveView, make sure to read React in LiveView: How and Why? blog post.

Back to LiveView hook. This isn’t fully working yet, notice how import of main.jsx exported a function to render SPA, we need to modify the most typical SPA set up to wait with rendering until we actually need it! Not on initial load.

// sample/src/main.jsx
const mount = (element) => {
 const root = ReactDOM.createRoot(element);
 root.render(
 <React.StrictMode>
 <App />
 </React.StrictMode>,
 );

 return () => {
 root.unmount();
 };
};

export default mount;

…and that’s it, or is it? What about links?

We must be able to create links handled by LiveView on the client side. In a typical Phoenix heex template we’d use something like <.link navigate={~p"/about"}>About</.link>. LiveView then captures clicks (or fallbacks gracefully) and does client-side routing to seamlessly switch between pages. We need to do this for links on frontend. This can be done by adding phx annotations, and exactly we mimic the behavior of this LiveView component.

Sample component wrapping React Router link.

const LiveLink = ({ navigate, href, patch, replace, ...props }) => {
 const phxDataProps = {}

 if (navigate) {
 phxDataProps["data-phx-link"] = "redirect";
 phxDataProps["data-phx-link-state"] = replace ? "replace" : "push";
 } else if (patch) {
 phxDataProps["data-phx-link"] = "patch";
 phxDataProps["data-phx-link-state"] = replace ? "replace" : "push";
 }

 return <Link {...props} {...phxDataProps} to={navigate || patch || href || "#"} />;
};

For example, /app/login is a LiveView page and everything else is still on SPA.

Phoenix routing:

 live "/app/login", LoginLive
 live "/app/*path", SpaLive

A sample menu on SPA then may look like this:

// Full client rounting. LiveView won't see it at all
<Link to="/app/about">About</Link>

// LiveView captures click, handle_params/3 is called but SPA still renderd the page
<LiveLink patch="/app/contact">Contact</LiveLink>

// LiveView captures the click and navigates to a different LiveView dedicated to login page
<LiveLink navigate="/app/login">Login</LiveLink>

When it comes to navigation to SPA views, on backend it works as with any other link. Sample menu:

~H"""
 <.link navigate={~p"/app/about"}>About</.link>
 <.link navigate={~p"/app/contact"}>Contact</.link>
 <.link navigate={~p"/app/login"}>Login</.link>
"""

That’s all there is to navigate between SPA and LiveView, for a demo.

It wouldn’t work on deployment, for deployment, we need to load files from the manifest instead of running dev server, so the LiveView hook presented earlier would be a bit more complex.

Side note: if you’re using react-router, ensure you initialize the routes when SPA is mounting, not outside of it on initial load! Otherwise, when LiveView navigates browser’s history, react-router state will be out of date and it won’t render the correct page!

- const router = createBrowserRouter([...]);

const mount = (element) => {
 const root = ReactDOM.createRoot(element);
+ const router = createBrowserRouter([...]);
 root.render(
 <React.StrictMode>
 <RouterProvider router={router} />
 </React.StrictMode>,
 );
 ....

Conclusion

I actually thought it would be much more difficult to connect both worlds and was pleasantly surprised how little glue was needed in very specific places.

Such integration might be a practical option to explore for older applications, where LiveView looks very interesting to the team, but at the same time we need to maintain and develop our frontend too. We can still write SPA as before, we can create backend views like there’s no SPA and LiveView navigates between the two separate worlds.

And this separation is concerning because there are some areas where the two will overlap.

UI Components

It’s unlikely that client UI toolkit uses Web Components. This means that we either have to re-implement UI components, using the same styling on backend, or migrate needed frontend components to web components.

Currently on LiveView side, while basic web components would work, slots and styling for shadow DOM needs special consideration, see this thread on elixirforum.

Switch between two modes

If you follow the demo at the beginning closely, there’s noticable rerender when page switches between the two modes. This is because almost the whole DOM is replaced when JS/LiveView takes over.

This is very noticeable on elements that would be shared and must look the same in both SPA and LiveView. If it’s make-or-break, it might be worth considering moving these elements to server side and letting SPA just render the “inside” of the page, not the whole layout.

There’s more

There is more that needs to be done for great UX before you go ahead and deploy it for production. All invalid links now point to SPA, always. Maybe previously any user with the page left open, didn’t keep the WebSocket connection open and everything was cached. Now you need to handle this load on server side. And many more things, unique to each setup. If you would like to check the full code from the demo, see this repository. Thanks for reading!

Taming Slack notifications and channels

arathunku — Sun, 26 Feb 2023 14:00:00 +0000

I’ve heard this at least a dozen of times how Slack is overwhelming, distracting, and not actually helping in organizing any knowledge. I wholeheartedly agree! I’m a fan of async-first approach, with a call here and there - when needed, with clear goal. Slack is instant messenger first, and by default facilitates very synchronous communication. Too synchronous for my taste, but it can be made not so instant! My settings focus on this, and I’d like to share some of my tweaks.

The 1st offender is typing a message prompt! For me, that’s 100% noise that only wastes time, you may be waiting for the message that may never come! Noise, noise, noise.

I never try to stay on top of all messages by being bombarded with sounds about every single one of them. I’ve muted all sounds. If something is absolutely critical, there’s on-call rotation and in the worst case, a phone number to call.

Did you know you can also receive a summary by email(bottom of the screenshot above)? Useful for catching up after holidays. Notice, how Slack encourages here more instant communication. It’s not like on a Discourse forum where you may select “summary” from a day/week/month, it’s ONCE AN HOUR , and only when you’re not active.

Sounds are muted - perfect. Now time for desktop notifications too, they’re enabled for keywords and I use different settings for other channels. This doesn’t mean I receive desktop notifications all the time.

Notifications schedule:

I get them when I start work and not earlier, not later. Anyone who’s messaging me gets information that my notifications are muted, but they’re welcome to click “Send now…” and bypass this.

Status: away, I always keep it that way and disable Slack’s prompt to toggle it back on. In the past, I’ve noticed people waiting for the dot to go green to send me a message instead of doing it whenever they wanted/needed to. Green dot, online presence, is a sign of synchronous communication. We don’t have presence tracking on email! (fortunately)

Inbox Zero fans love this setting, me included! Only unread channels are visible, keeps channels list tidy. You’re welcome to join even 1000 channels, but your sidebar won’t grow as much unless they’re all very active, then maybe reconsider few things… They’re only visible when there’s something new in them, and even in that case - it’s easy to tell if it’s a mention/keyword or just a normal message and catch up later, or never.

You can always jump quickly to any channel via C-k shortcut, sidebar is not needed for that.

And the last one, https://nohello.net!

DEV Community: arathunku

FIT parsing and code generation in Elixir

Taking shortcuts with inspect/2

Switch to macros

Formatting

Conclusion

Shell scripting with Elixir

The template

Output

Colors

Exit code

Subprocesses

Signals, some are not like the others

Testing

Going beyond scripts

Closing note

Seamless navigation between SPA and LiveView

tl;dr

The setup

Extending SPA with LiveView

Conclusion

UI Components

Switch between two modes

There’s more

Taming Slack notifications and channels

Taking shortcuts with `inspect/2`