DEV Community: Masatoshi Nishiguchi

Bringing Open JTalk to Elixir/Nerves: Make Your Pi Speak Japanese 🇯🇵

Masatoshi Nishiguchi — Mon, 15 Sep 2025 10:52:11 +0000

日本語版

Introduction

One day my friend kurokouji asked, innocently opening a rabbit hole:

Can my Nerves-powered Raspberry Pi talk in Japanese using Open JTalk?

Challenge accepted.

I figured the answer was yes—I just didn’t know how yet. As Antonio Inoki once said:

元氣が有れば何でもできる (If you have spirit, you can do anything)

So there was no reason not to try. Worst case, I’d learn something. Best case, the Pi speaks Japanese. Thanks to modern AI tooling, we can now explore rabbit holes without getting totally lost.

The result is open_jtalk_elixir—a portable Elixir wrapper for Open JTalk, Japan’s classic text-to-speech engine.

This library:

Builds a native Open JTalk CLI during compilation
Bundles the required dictionary + voice assets by default
Exposes a clean Elixir API like this:

Even if the destination was clear, the path was full of lessons. This post documents the journey—the dead ends, the design trade-offs, and how the pieces came together.

What Is Open JTalk?

Open JTalk is a well-established Japanese text-to-speech (TTS) engine widely used in research, embedded systems, and hobby projects.

It’s built from these components:

MeCab 0.996 — A morphological analyzer that parses Japanese text into words with readings and grammatical features.
HTS Engine API 1.10 — A statistical parametric speech synthesis engine that generates the actual waveform from features.
Open JTalk Dictionary 1.11 — The UTF-8 dictionary package (open_jtalk_dic_utf_8-1.11) distributed with Open JTalk. It provides the lexical data required by MeCab for morphological analysis and is maintained as part of the official Open JTalk project.
HTS Voice “Mei” (MMDAgent Example 1.8) — A widely used Japanese voice model, clear and natural sounding.
Open JTalk CLI 1.11 — A C-based command-line tool that ties all the components together and outputs audio.

Despite its age, Open JTalk remains relevant—especially when you want offline, no-internet Japanese TTS in embedded environments like Raspberry Pi or Nerves.

Challenges in embedded environments

Making Open JTalk work inside a Nerves-powered Elixir firmware image isn’t trivial.

Here’s why:

1. Native Build Complexity

Open JTalk uses autotools to compile its C stack: MeCab, HTS Engine, and Open JTalk itself. These aren't built for cross-compilation out of the box.

2. Required Assets Are Large

Open JTalk doesn’t work without its dictionary and voice model files. You need to ship:

Dictionary: ~100 MB
Mei voice: ~2 MB
CLI binary: ~1 MB

If you skip bundling them, your app might compile just fine... but silently fail to synthesize speech unless the user manually downloads those assets post-install.

That’s not the kind of "developer experience" we want.

The approach that worked: Vendor + Bundle + Shell Out

After exploring various approaches, I found this simple one is effective:

Vendor and compile all C dependencies at Elixir build time (mix compile)
Bundle voice + dictionary files into the priv/ directory
Use System.cmd/3 to shell out from Elixir to the native CLI

That means you get a fully working setup out-of-the-box, even on embedded targets like Raspberry Pi running Nerves.

If you're concerned about firmware size—such as for minimal Nerves deployments—you can disable bundled assets like this:

OPENJTALK_BUNDLE_ASSETS=0 \
OPENJTALK_DIC_DIR=/data/jdic \
OPENJTALK_VOICE=/data/mei.htsvoice \
mix compile

You can even skip the build entirely (e.g. if prebuilt in CI):

OPENJTALK_BIN=./bin/open_jtalk mix compile

All we’re really doing is letting Elixir orchestrate the CLI, while the heavy lifting stays in native code.It might not be fancy, but it’s reliable, portable, and easy to debug.

Ideas That Didn’t Work

Before settling on the current approach, I tried several others. Here's why they failed:

Custom Nerves System

The first idea was to build a custom Nerves system image with Open JTalk baked in. It sounds powerful—everything precompiled and baked into the firmware.

But in practice, it's heavy for iteration and slow for development. Any time you want to tweak or test something, you're rebuilding the world.

On top of that, even with Open JTalk included, you’d still need to wrap the CLI in Elixir and handle asset loading.

I ruled this out early—it wasn’t worth the complexity for what I was trying to build.

Elixir Native Implemented Functions (NIFs)

NIFs let you call C functions directly from Elixir—fast, no shelling out.

But NIFs come with risk: if anything goes wrong, they can crash the entire BEAM VM. Portability across different CPU/OS combinations is also tricky.

So I skipped this route too. Safer for me—and for your runtime.

Elixir Port

I’ve used Ports before—for example, in my sgp40 sensor library. It works great when input/output is simple.

Open JTalk isn’t that simple. Encoding issues, temp files, complex args... Port was more pain than gain.

Build System Details

The native stack is built via a Makefile + shell scripts. The flow:

Fetch Sources Downloads and unpacks:

mecab
mecab-ipadic-utf8
hts_engine_API
open_jtalk

Configure & Patch
Applies platform-specific patches (e.g., macOS fixes, updated config.guess).
Compile C Stack
Each lib is compiled using autotools and linked statically when possible.
Bundle CLI + Assets
Places everything into priv/, ready for Elixir to use.

Cross-Platform Portability

Getting it to run on:

🐧 Linux (x86_64 & ARM)
🍎 macOS (Apple Silicon)
🍓 Raspberry Pi via Nerves

...was an adventure.

Autotools Pain

Autotools can’t detect modern platforms without help:

configure: error: cannot guess build type

So I inject fresh config.sub and config.guess before each build.

macOS Quirks

BSD install doesn’t support -D → shim script added.
No static linking → disabled on Darwin.

Nerves Cross-Compiling

Nerves handles toolchains if you respect MIX_TARGET and pass the right env:

export MIX_TARGET=rpi4
mix deps.get
mix compile
mix firmware

Worked like a charm!

How to Use in Elixir

After mix compile, usage is dead simple:

# Speak out loud using system audio
OpenJTalk.say("元氣が有れば、何でもできる")

# Synthesize to WAV
OpenJTalk.to_wav("こんにちは", out: "/tmp/test.wav")

# Get WAV as binary
{:ok, binary} = OpenJTalk.to_binary("おはようございます")

Options available

All functions accept the same set of options, which map to Open JTalk’s CLI flags but with friendlier names:

Option	Meaning	Default
`:pitch_shift`	Semitone shift	`0`
`:rate`	Speaking speed multiplier	`1.0`
`:timbre`	Voice quality adjustment	`0.0`
`:gain`	Output gain (in dB)	`0`
`:voice`	Path to `.htsvoice` file	(bundled)
`:dictionary`	Path to dictionary folder	(bundled)
`:timeout`	Max runtime in milliseconds	`20000`

Asset Resolution

The library automatically finds the required pieces (binary, dictionary, voice) in this order:

Environment variables (OPENJTALK_CLI, OPENJTALK_DIC_DIR, OPENJTALK_VOICE)
Bundled assets: files placed in priv/ at compile time
System locations: /usr/share, /usr/local/share, Homebrew, etc.

You can inspect asset resolution:

OpenJTalk.info()

If you ever swap files or tweak environment variables at runtime, you can reset the cache with:

OpenJTalk.Assets.reset_cache()

Lessons Learned

Bundle assets by default — Users expect things to “just work.” Opt-out is fine, but opt-in creates surprises and frustration.
Respect Nerves conventions — Cross-compilation works smoothly if you let Nerves set the toolchain. Fighting it only adds pain.
Autotools lacks modern platform support — Legacy config.guess and config.sub frequently fail; replacing them is essential for portability.
macOS quirks matter — Even if you don’t use macOS, someone else will. Handle BSD tool differences and static linking limits early.
Elixir + shelling out is a solid choice — For complex native tools, it’s simpler and safer than managing Ports..
Keep Makefiles boring — Avoid turning Makefiles into a programming language. Offload logic to scripts for clarity and maintainability.
Test on ARM early — x86 isn’t the default world anymore. Raspberry Pi, Apple Silicon, and cloud ARM runners should be first-class citizens.

Final Thoughts

I started with a goal: get my Raspberry Pi to talk Japanese using Elixir.

It turned into a full-blown cross-platform build system for a decades-old C library—with clean UX and out-of-the-box Nerves support.

But the end result?

That’s all we wanted.

Give open_jtalk_elixir a try, and let your Pi speak too.

Getting Started with AtomVM: Setup with Prebuilt Firmware (ESP32-S3, September 2025)

Masatoshi Nishiguchi — Tue, 02 Sep 2025 01:22:31 +0000

I previously tried using AtomVM, a lightweight virtual machine that lets you run Elixir (and Erlang) applications directly on microcontrollers like the ESP32-S3, but honestly, I rushed through the process without fully understanding the underlying details — especially around firmware setup.

This time around, I wanted to go back and properly understand how to set things up, focusing specifically on getting started with AtomVM using a prebuilt firmware image. The goal is to simplify the onboarding experience without building anything from source.

This post doesn’t dig into advanced features. Instead, it’s a hands-on guide to get your ESP32-S3 board up and running with AtomVM, so you can start running Elixir code on real hardware as quickly as possible.

Originally published in Japanese: AtomVM 入門: ビルド済みイメージを活用した環境構築 (2025年9月)

Target Environment and Devices

Here's what I used for this setup.

Target Board

Seeed Studio XIAO ESP32-S3
- Onboard LED on GPIO 21 (active-low)

Host Machine

Linux (Debian-based — LMDE6)
Elixir 1.17 with Erlang/OTP 27
AtomVM v0.6.6 (prebuilt firmware)
esptool 5.0 — official flashing tool for ESP32
picocom 3.1 — for serial monitoring
Python 3.13 — required by esptool

Notes

esptool is a Python tool and requires Python 3.x.
Since we're using a prebuilt AtomVM image, you do not need to install ESP-IDF.
For compatibility info (e.g. which Elixir & OTP versions are supported by AtomVM), refer to the official release notes.

Installing esptool

esptool is the official command-line utility for flashing firmware to ESP32 boards.

We'll install it using pip, which makes it easy to manage and update.

pip install "esptool>=5,<6"

Version 5.x introduces hyphenated options instead of older underscore syntax.
Make sure Python 3.x is installed on your system.

Checking the Serial Port

When you connect your ESP32-S3 via USB, Linux will create a new serial device (e.g., /dev/ttyACM0). You’ll need this port name when flashing firmware or uploading .avm files.

On Linux, you can run:

dmesg | grep tty

Output might look like:

[12345.678901] cdc_acm 1-1.4:1.0: ttyACM0: USB ACM device

In this case, use /dev/ttyACM0.

You’re typically looking for something like /dev/ttyACM0 or /dev/ttyUSB0, depending on your board and USB-to-serial chip.

Verifying `esptool` and Board Connection

Once esptool is installed, you can check that:

It's correctly installed
It can communicate with your ESP32-S3 board

# Check version
esptool.py version

# Query flash chip information
esptool.py -p /dev/ttyACM0 flash-id

If this succeeds, you're ready to start flashing firmware.

Downloading and Flashing AtomVM Firmware

We’ll be using a prebuilt binary from the AtomVM GitHub Releases, so there’s no need to build anything from source.

We’ll use the official prebuilt AtomVM image (v0.6.6) for ESP32-S3.

Download the image

# Destination folder is arbitrary
mkdir -p $HOME/Projects/atomvm && cd $_

curl -LO https://github.com/atomvm/AtomVM/releases/download/v0.6.6/AtomVM-esp32s3-elixir-v0.6.6.img

Erase flash (recommended)

esptool.py --chip auto --port /dev/ttyACM0 --baud 921600 erase-flash

Write AtomVM firmware

esptool.py --chip auto --port /dev/ttyACM0 --baud 921600   write_flash 0x0 $HOME/Projects/atomvm/AtomVM-esp32s3-elixir-v0.6.6.img

According to the AtomVM docs, the bootloader offset for ESP32-S3 is 0x0.

Building and Flashing an Elixir Application

With AtomVM running on your ESP32-S3, you can now deploy an actual Elixir application to it!

In this section, we'll use the classic Blinky sample to flash an .avm (AtomVM bytecode) file that blinks the onboard LED.

Get the sample project

Clone the official atomvm_examples repo:

cd ~/Projects/atomvm  # Or any directory you like

git clone https://github.com/atomvm/atomvm_examples.git
cd atomvm_examples/elixir/Blinky

Install dependencies (mainly exatomvm):

mix deps.get

Adjust GPIO pin for XIAO ESP32-S3

By default, the sample blinks GPIO 2 — but XIAO’s onboard LED is wired to GPIO 21 (active low). Update lib/blinky.ex accordingly:

defmodule Blinky do
  @pin 21

  def start() do
    :gpio.set_pin_mode(@pin, :output)
    loop(@pin, :low)  # LOW = LED ON (active-low)
  end

  defp loop(pin, level) do
    :io.format(~c"Setting pin ~p ~p~n", [pin, level])
    :gpio.digital_write(pin, level)
    Process.sleep(1000)
    loop(pin, toggle(level))
  end

  defp toggle(:high), do: :low
  defp toggle(:low),  do: :high
end

Configure `mix.exs` with flash offset

You must set the .avm flash offset to 0x250000. Without this, the ESP32-S3 won’t recognize your app at boot.

def atomvm do
  [
    start: SampleApp,
    flash_offset: 0x250000  # important!
  ]
end

Package the app

Build the .avm files using:

mix atomvm.packbeam

This generates .avm files such as:

Blinky.avm
deps.avm
priv.avm

Flash the app to ESP32-S3

Finally, push your app:

mix atomvm.esp32.flash --port /dev/ttyACM0

This will write the .avm files to the right spot, without touching the AtomVM firmware.

You should now see the LED start blinking 🎉

Viewing Logs over Serial

Your Blinky app doesn't just blink the LED — it also prints log messages via serial.

To view these logs in real time, you'll need a serial monitor. We'll use picocom — a simple, lightweight tool that works great on Linux.

To start monitoring logs from your ESP32-S3 board:

picocom /dev/ttyACM0 --baud 115200

This opens the serial console. You should immediately start seeing output like:

These messages come from your Elixir code (:io.format), showing the pin toggling every second.

To exit picocom, press: Ctrl+A then Ctrl+X.

Wrapping Up

In this post, we walked through the full process of setting up an ESP32-S3 board to run Elixir code using AtomVM — no custom firmware builds required.

This setup makes it easy to start tinkering with embedded Elixir, even without diving deep into ESP-IDF or AtomVM internals.

Next steps might include integrating with sensors, displaying data, or connecting to the cloud via MQTT.

Hopefully, this gave you a smooth start with AtomVM on ESP32-S3!

https://github.com/mnishiguchi/hello_atomvm_tft_spi

Building a Local AI-Friendly Code Viewer Inspired by Uithub

Masatoshi Nishiguchi — Fri, 28 Mar 2025 04:52:29 +0000

I've been using AI tools daily at work—thanks to my employer’s support for AI-assisted development. At the same time, I like keeping my text editor simple and sometimes prefer writing code without real-time AI suggestions.

Recently, I found Uithub, a helpful web service introduced by a coworker. It renders Git repository contents (directory trees and file contents) in a format that works well for AI conversations.

Here’s a related blog post from a teammate:

Qiita: Uithubを使ってAIとコードを会話する

I liked the idea so much that I wanted something similar locally—without needing to upload code anywhere.

ニホンゴ版

First Attempt: Shell Script

My first attempt was a Bash script that mimicked Uithub’s output. I wrote about it here:

Local version of Uithub using shell script

While it worked, it was hard to maintain or extend. So I decided to rewrite it in a more suitable language.

Rewriting in Go

Around the same time, I noticed that asdf was rewritten in Go. I had also been practicing Go recently, so I rewrote my local Uithub-inspired tool as a Go CLI app.

For the CLI framework, I picked urfave/cli/v2, which is also used by asdf. It’s lightweight and works well for small tools.

I named it uit, short for Uithub-Inspired Tool

Here’s the GitHub repo: https://github.com/mnishiguchi/uit

You can download prebuilt binaries for macOS and Linux from the Releases page.

Wrapping up

Just a small tool I built for myself, but if it helps others too — even better.

Boost Your Terminal Navigation with the y z Workflow

Masatoshi Nishiguchi — Tue, 18 Mar 2025 23:36:30 +0000

Navigating directories efficiently in the terminal is essential for
productivity. While cd has been the standard for decades, there are smarter
and faster alternatives.

Meet the y → z workflow—a powerful way to jump between directories and
manage files effortlessly.

What is `y` → `z`

The letters y and z don’t have an inherent meaning, but their alphabetical
order mirrors c and d—making them easy to remember as a cd alternative.

This workflow leverages:

zoxide – A smarter cd replacement that remembers frequently visited directories.
yazi – A blazing-fast terminal file manager with zoxide integration.
fzf – A fuzzy finder that enhances interactive navigation.

The Power of `z` (zoxide)

zoxide enhances terminal navigation by remembering frequently visited
directories. The z command allows you to quickly jump to a known directory
based on name matching. For even more flexibility, zi integrates zoxide
with fzf, enabling an interactive fuzzy search through your directory
history, letting you select a destination effortlessly.

Basic Commands

z project    # Jump to a frequently visited "project" directory 
zi           # Use fzf to interactively search recent directories

The Power of `y` (yazi)

yazi is a powerful
terminal-based file manager designed for fast and efficient navigation. One of
its standout features is its seamless integration with zoxide. While using
yazi, you can press z to instantly jump to any frequently visited directory
tracked by zoxide. This allows you to combine fuzzy directory switching with
visual file management, creating a smooth and efficient workflow for handling
files and navigating your system.

Basic Commands

y            # Open yazi

Once inside yazi, press z to instantly jump to a directory tracked by
zoxide.

However, when a full TUI file explorer isn't necessary, you might prefer using
zi directly. Instead of opening yazi, the zi command allows you to
instantly select and jump to a directory using fuzzy search—making it the
fastest option for quick directory switching.

Conclusion

If you're looking for a faster way to move around your file system, the y →
z workflow is a game-changer.

Elixir/Nerves: blinking LED

Masatoshi Nishiguchi — Mon, 24 Jun 2024 01:48:42 +0000

Introduction

When we first learn Nerves, one of the things we come up with is blinking LEDs on a breadboard.

Today I'd like to try one implementation using GenServer.

This is actually my outcome at this event I participated in last weekend "Shop for parts in Akihabara and get started on Nerves right away!". It was so much fun hanging out in Akihabara, Tokyo with Nerves enthusiasts.

Thank you very much to the organizers myasu and nako_sleep_9h!

myasu is very knowledgeable about electric and electronic stuff. I learned tons of pro tips from him, which I might write up as my future posts. He is the author of this book about Nerves.

nako_sleep_9h is awesome at organizing and mc'ing fun-driven engineering events across Japan. She wrote a report on Day 1 for this event.

めでたい、光った光った〜#piyopiyoex pic.twitter.com/5WBqnFftQU
— nako@9時間睡眠 (@nako_sleep_9h) June 22, 2024

Nerves Livebook

There are more than a few ways to get started with Nerves IoT projects but one of the easiest is to use prebuild Nerves Livebook firmware. Nerves Livebook allows us to control our target devices such as Raspberry Pi, doing Elixir coding in a web browser.

https://github.com/nerves-livebook/nerves_livebook/blob/main/README.md

Blink using Circuits GPIO

Nerves Livebook has a nice tutorial notebook about how to use the circuits_gpio package to control a GPIO as an output, and blink an LED. It shows us what electric parts we need minimally and how we wire them before getting down to business.

https://github.com/nerves-livebook/nerves_livebook/blob/main/priv/samples/basics/blink.livemd

Wrapping our LED blinking logic in GenServer

When we want to repeat the blinking forever but want to stop at some stage without restarting the device, it is nice to prepare a GenServer that manages the LED blinking. There can be many ways to achieve similar effect but here is one implementation using GenServer.

First, let's write a simple module that wraps GPIO-related logic within so that we can forget about how exactly we need to communicate with our LEDs.

defmodule LedBlink do
  def open(led_pin) do
    Circuits.GPIO.open(led_pin, :output)
  end

  def close(led_pin) do
    Circuits.GPIO.close(led_pin)
  end

  def toggle(gpio, 1), do: off(gpio)
  def toggle(gpio, 0), do: on(gpio)
  def toggle(gpio, _), do: on(gpio)

  def on(gpio) do
    :ok = Circuits.GPIO.write(gpio, 1)

    {:ok, 1}
  end

  def off(gpio) do
    :ok = Circuits.GPIO.write(gpio, 0)

    {:ok, 0}
  end
end

Then next, we make a GenServer that lets an LED blink forever.

defmodule BlinkServer do
  use GenServer, restart: :temporary

  require Logger

  @run_interval_ms 1000

  ## Client

  @doc """
  Start a BlinkServer for the provided GPIO pin. It lets the LED blink forever.

  ### Examples

      iex>  BlinkServer.start_link(led_pin: "GPIO17")

  """
  def start_link(opts) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

  def stop() do
    GenServer.stop(__MODULE__)
  end

  ## Callbacks

  @impl true
  def init(opts) do
    led_pin = Access.fetch!(opts, :led_pin)

    initial_state = %{
      gpio_ref: nil,
      led_state: 0,
      led_pin: led_pin
    }

    {:ok, initial_state, {:continue, :init_gpio}}
  end

  @impl true
  def handle_continue(:init_gpio, state) do
    case LedBlink.open(state.led_pin) do
      {:ok, gpio_ref} ->
        new_state = %{state | gpio_ref: gpio_ref}

        send(self(), :toggle_led_state)

        {:noreply, new_state}

      {:error, error} ->
        {:stop, error}
    end
  end

  @impl true
  def handle_info(:toggle_led_state, state) do
    {:ok, new_led_state} = LedBlink.toggle(state.gpio_ref, state.led_state)

    new_state = %{state | led_state: new_led_state}
    Logger.debug("toggled LED: #{new_state.led_state}")

    Process.send_after(self(), :toggle_led_state, @run_interval_ms)

    {:noreply, new_state}
  end

  @impl true
  def terminate(reason, state) do
    LedBlink.close(state.gpio_ref)
    Logger.debug("terminated: #{reason}")

    :ok
  end
end

Here is how to use our BlinkServer.

When starting BlinkServer, we specify the GPIO pin we use for our LED.

When we are done experimenting, we can stop our running BlinkServer worker.

Logger.configure(level: :debug)

BlinkServer.start_link(led_pin: "GPIO17")

Process.sleep(10_000)

BlinkServer.stop()

This implementation of BlinkServer is a singleton GenServer, meaning we are not allowed to start more than one BlinkServer workers. So if we change our mind on the GPIO pin we use, for example, we would need to stop the worker and start a new worker with different options.

circuits_gpio package

circuits_gpio is the one that lets us control GPIO in our Elixir code.

Wrapping up

We wrote a simple GenServer that lets our LED blink.

Once you get a feel of it, I would like to see your LED blinking logic. Please share!

今日は初Nerves！
8月末のSWESTに向けた知識を付けていこう～ https://t.co/SrGEUJjzkS
— FRICK (@TewiEwi_no96) June 21, 2024

Nerves勉強会2日目！
開始〜 https://t.co/zgMKDEGrH8 pic.twitter.com/rT7QiHslAa
— FRICK (@TewiEwi_no96) June 23, 2024

#piyopiyoex
【オフライン】秋葉原でパーツお買い物＆そのままNerves入門！
はじまたー pic.twitter.com/BPiPBxZMTq
— ysaito (@ysaito8015) June 23, 2024

神田明神きたよ#piyopiyoex pic.twitter.com/xuxuiWcl2F
— nako@9時間睡眠 (@nako_sleep_9h) June 22, 2024

今日はこちらへ参加！ https://t.co/L28N3O6nIF
— myasu (@etcinitd) June 22, 2024

今日はDay2!!https://t.co/hgJwwqGO9W
— myasu (@etcinitd) June 23, 2024

Where Nerves-related Mix tasks are defined?

Masatoshi Nishiguchi — Mon, 15 Jan 2024 20:03:19 +0000

Nerves comes with a number of powerful and convenient Mix tasks.

https://nerves-project.org/

Since I have not used many of those Mix tasks, first I want to learn what Mix tasks are available for my Nerves projects.

The Nerves project is made up of separate per-scope sub-projects. So Mix tasks are maintained in different repositories. It is great for maintainability, but finding source code can be a bit hard without prior knowledge. For that reason, it is nice to familiarize ourselves with Nerves sub-projects by casually visiting them.

Mix

In case, somebody is new to Mix, here is some info.

Mix is a build tool that provides tasks such as creating, compiling, testing, and managing dependencies for Elixir projects.

https://hexdocs.pm/mix/Mix.html

https://elixirschool.com/en/lessons/basics/mix

Enumerating Mix tasks in Elixir project

First of let's try and enumerate Mix tasks available for your Nerves project. We can achieve that just by running mix help.

As an example, let's try it under the nerves_livebook project.

cd path/to/nerves_livebook

mix help

At this point, the results include non-Nerves Mix tasks. As I counted the number of lines, it was a total of 65. It might vary depending on projects.

mix help | wc -l

Filtering down to only Nerves-related Mix tasks

I simply ran grep and was able to narrow it down to only Nerves related Mix tasks!

mix help | grep 'mix' | grep -iE 'nerves|firmware'

The results are as follows.

mix burn                    # Write a firmware image to an SDCard
mix compile.nerves_package  # Nerves Package Compiler
mix firmware                # Build a firmware bundle
mix firmware.burn           # Build a firmware bundle and write it to an SDCard
mix firmware.gen.gdb        # Generates a helper shell script for using gdb to analyze core dumps
mix firmware.gen.script     # Generates a shell script for pushing firmware updates
mix firmware.image          # Create a firmware image file
mix firmware.metadata       # Print out metadata for the current firmware
mix firmware.patch          # Build a firmware patch
mix firmware.unpack         # Unpack a firmware bundle for inspection
mix local.nerves            # Checks for updates to nerves_bootstrap
mix nerves.artifact         # Creates system and toolchain artifacts for Nerves
mix nerves.artifact.details # Prints Nerves artifact details
mix nerves.clean            # Cleans dependencies and build artifacts
mix nerves.info             # Prints Nerves information
mix nerves.new              # Creates a new Nerves application
mix nerves.system.shell     # Enter a shell to configure a custom system
mix nerves_key.device       # Simulate NervesKey device key creation
mix nerves_key.signer       # Manages NervesKey signing keys
mix upload                  # Uploads firmware to a Nerves device over SSH

In case we want to learn the innerworkings of the commands later, it is nice to know where their souce code is located.

Where Nerves-related Mix tasks are defined

This was more difficult than I had imagined. Nerves projects are packaged by concern. Therefore, various features are managed in different repositories.

The nerves package's README.md explains what each repository is responsible for with a comprehensive listing.

Our project is spread over many repositories in order to focus on a limited scope per repository.

This repository (nerves-project/nerves) is an entrance to Nerves and provides the core tooling and documentation.

Here are the packages that define Nerves-related Mix tasks.

nerves package

https://hexdocs.pm/nerves

mix burn                    # Write a firmware image to an SDCard
mix compile.nerves_package  # Nerves Package Compiler
mix firmware                # Build a firmware bundle
mix firmware.burn           # Build a firmware bundle and write it to an SDCard
mix firmware.gen.gdb        # Generates a helper shell script for using gdb to analyze core dumps
mix firmware.image          # Create a firmware image file
mix firmware.metadata       # Print out metadata for the current firmware
mix firmware.patch          # Build a firmware patch
mix firmware.unpack         # Unpack a firmware bundle for inspection
mix nerves.artifact         # Creates system and toolchain artifacts for Nerves
mix nerves.artifact.details # Prints Nerves artifact details
mix nerves.clean            # Cleans dependencies and build artifacts
mix nerves.info             # Prints Nerves information
mix nerves.system.shell     # Enter a shell to configure a custom system

nerves_bootstrap package

https://hexdocs.pm/nerves_bootstrap

mix local.nerves            # Checks for updates to nerves_bootstrap
mix nerves.new              # Creates a new Nerves application

nerves_key package

https://hexdocs.pm/nerves_key

mix nerves_key.device       # Simulate NervesKey device key creation
mix nerves_key.signer       # Manages NervesKey signing keys

ssh_subsystem_fwup package

https://hexdocs.pm/ssh_subsystem_fwup

mix firmware.gen.script     # Generates a shell script for pushing firmware updates
mix upload                  # Uploads firmware to a Nerves device over SSH

🎉🎉🎉

Wrapping up

To be honest, I have never used most of the Mix tasks I saw today yet. If you have any cool techniques using them, please share!

Build Phoenix Docker Compose development environment easily

Masatoshi Nishiguchi — Tue, 28 Nov 2023 23:25:12 +0000

I wrote a script called phx-docker-compose-new that can be used instead of mix phx.new to generate a new Phoenix application.

Building the development environment for a Phoenix application using Docker Compose is convenient because you can set up not only the application but also the PostgreSQL database, Livebook, etc. all at once.

However, the reality is that it is not always easy and requires some know-how.

https://qiita.com/koyo-miyamura/items/a609de2e9fadaf198243

https://zenn.dev/koga1020/articles/d260bc1bde8267

https://qiita.com/mnishiguchi/items/e367743bca3520e2a387

I just thought it would be nice to have a script that allows us to easily build a Phoenix development environment using Docker Compose. Of course, just thinking about it won't change anything, so I got down to business straight away.

日本語版

Getting started

Make sure that Git, Docker, and Docker Compose are installed in your system.

git version
docker --version
docker compose version

Download the source code for the phx-docker-compose-new command.

git clone https://github.com/mnishiguchi/phx-docker-compose-new.git ~/.phx-docker-compose-new

Define an alias so you can use the phx-docker-compose-new command in your terminal.

alias phx-docker-compose-new=~/.phx-docker-compose-new/phx-docker-compose-new.sh

Generate a Phoenix sample app using the phx-docker-compose-new command. For available options, refer to Phoenix official documentation.

phx-docker-compose-new sample_phx_app --no-assets --no-gettext --no-mailer

Go into the generated app directory and launch the app.

cd sample_phx_app

bin/start

You can access the URLs below and start developing your Phoenix app right now!

You can look at the log with the command below. Press Ctrl + C to close the log.

bin/logs --follow

Elixir's interactive console (IEx) can be started with the following command.

bin/console

Since IEx is open, we might as well do something. Let's display a list of processes.

IEx.configure inspect: [limit: :infinity]

for pid <- Process.list, do: {pid, Process.info(pid, :registered_name) |> elem(1)}

To stop the app, use the following command.

bin/stop

🎉🎉🎉

Wrapping up

Congratulations! Now you can build a Phoenix application development environment easily any time.

Elixirでリストの最初の要素を取得

Masatoshi Nishiguchi — Thu, 17 Aug 2023 00:24:08 +0000

ElixirでListの最初の要素を取得する方法についていくつか考えてみました。一見初歩中の初歩のように見えますが、実は複数のやり方があり、その使い分けがエンジニアとしての腕の見せどころにもなりえます。

これからElixirを始める方にはこのサイトがおすすめです。

https://elixir-lang.info/

Elixirとコミュニティの雰囲気をゆるく味わいたい方は「先端ピアちゃん」さんの動画が超オススメです。

https://www.youtube.com/@piacerex

ElixirのList

Listは値の集合です。複数の異なるタイプを含むことができます。一意でない値を含むこともできます。

ひとつ注意点は、ElixirのListは連結リストとして実装されていることです。今回の実験では特に問題がないですが、他の主要プログラミング言語の配列とは異なるので思い込みは禁物です。

https://elixirschool.com/ja/lessons/basics/collections

https://qiita.com/search?q=Elixir+List

実験の準備

実験用List

要素数の異なるパターンのリストを用意します。

lists = [
  [],
  [nil],
  [1],
  [1, 2],
  [1, 2, 3],
]

実験を実行する関数

前項で作成したリストを何度も使用するので、共通部分を切り出し関数化します。
万一、例外が発生した場合にもテストを止めず、エラー内容を値として処理します。

test_fun = fn do_test ->
  for list <- lists do
    try do
      do_test.(list)
    rescue
      e -> inspect(e)
    end
  end
end

Kernel.hd/1 (hd/1)

hd/1はListが空の時にはエラーになります。

test_fun.(fn list ->
  hd(list)
end)

["%ArgumentError{message: \"errors were found at the given arguments:\\n\\n  * 1st argument: not a nonempty list\\n\"}",
 nil,
 1,
 1,
 1]

https://hexdocs.pm/elixir/Kernel.html#hd/1

List.first/1

List.first/2のユニークなところは、空リストの最初の要素はnilと解釈する点です。この特性が便利な場面があります。

test_fun.(fn list ->
  List.first(list)
end)

[nil,
 nil,
 1,
 1,
 1]

https://hexdocs.pm/elixir/List.html#first/2

Enum.at/2

@torifukukaiou さんからお便りをいただき、これを見落としていたのに気がつきました。ありがとうございます。

Enum.at/2はList.first/2と同様に空リストの最初の要素をnilとします。

test_fun.(fn list ->
  Enum.at(list, 0)
end)

[nil,
 nil,
 1,
 1,
 1]

Enum.fetch/2

Enum.fetch/2は空リストの最初の要素を探すことを異常とみなしますが、プログラマーが例外処理しやすいように{:ok, any} | :errorを出力してくれます。

test_fun.(fn list ->
  Enum.fetch(list, 0)
end)

[:error,
 {:ok, nil},
 {:ok, 1},
 {:ok, 1},
 {:ok, 1}]

https://hexdocs.pm/elixir/Enum.html#fetch/2

Enum.fetch!/2

Enum.fetch!/2は空リストの最初の要素を探すことを異常とみなし、エラーを起こします。hd/1の時と結果が同じです。エラーメッセージはhd/1の方が具体的でわかりやすような気がしますが、こっちのエラーの方がスッキリ簡潔です。ここは好みが別れるところかもしれません。

test_fun.(fn list ->
  Enum.fetch!(list, 0)
end)

["%Enum.OutOfBoundsError{message: \"out of bounds error\"}",
 nil
 1,
 1,
 1]

https://hexdocs.pm/elixir/Enum.html#fetch!/2

[first | _rest]でパターンマッチ

このパターンは最初の要素と残り全部の2つに分割されます。Listが空の時はエラーになります。要素が一個の時は大丈夫のようです。

test_fun.(fn list ->
  [first | _rest] = list
  first
end)

["%MatchError{term: []}",
 nil,
 1,
 1,
 1]

https://elixirschool.com/ja/lessons/basics/pattern_matching

[first, _, _]でパターンマッチ

このパターンの場合は、要素数がピッタリ一致する場合のみヨシとされます。

test_fun.(fn list ->
  [first, _, _] = list
  first
end)

["%MatchError{term: []}",
 "%MatchError{term: [nil]}",
 "%MatchError{term: [1]}",
 "%MatchError{term: [1, 2]}",
 1]

ElixirのEnum技

List等の集合を操作する時に使うEnumには無数の興味深い関数やテクニックがありますが、ここではいくつか特にイーやつをご紹介させて頂こうと思います。

https://qiita.com/torifukukaiou/items/e07ed758d1259d14a2b7

https://qiita.com/torifukukaiou/items/3b65e5c04fa8c55f526e

https://qiita.com/torifukukaiou/items/4481f7884a20ab4b1bea

Elixir Phoenix 1.7 breadcrumb component

Masatoshi Nishiguchi — Sat, 12 Aug 2023 19:39:15 +0000

I like the breadcrumb navigation in a web app so I wanted to make one what works in Phoenix LiveView apps.

what breadcrumb navigation is

I think I first learned about the breadcrumb from Bootstrap a long time ago, but I do not know the exact definition of it. Let's wikipedia.

A breadcrumb or breadcrumb trail is a graphical control element used as a navigational aid in user interfaces and on web pages. It allows users to keep track and maintain awareness of their locations within programs, documents, or websites.

https://en.wikipedia.org/wiki/Breadcrumb_navigation

my custom breadcrumb component

I do not need anything smart. Simple is best. My breadcrumb component simply accepts a list of navigation items. It might be used like below.

<.breadcrumb items={[
  %{text: "Home", navigate: ~p"/"},
  %{text: "Examples", navigate: ~p"/examples"},
  %{text: "Light"}
]} />

By the way, the LiveView pages in the screen recording above are from Pragmatic Studio Phoenix LiveView Course. The online course is amazing. I added my custom breadcrumb component onto them for my extra learning.

HTML and Tailwind classes

Phoenix 1.7 comes with Tailwind CSS by default. Tailwind CSS just works without any configuration changes.

First of all, I did the Internet search for an example HTML snippet with Tailwind CSS classes because my primary focus is to enjoy programming with Elixir and Phoenix, not HTML, not Tailwind CSS.

I started by using this example snippet from Flowbite.

https://flowbite.com/docs/components/breadcrumb

Phoenix.Component.link/1

For links, I took advangate of Phoenix.Component.link/1 and its :navaigate attribute so I can navigate between LiveView pages smoothely.

https://hexdocs.pm/phoenix_live_view/Phoenix.Component.html#link/1

icons

Phoenix 1.7 ships with SVG icons from heroicons and those SVG icons can be used easily via MyAppWeb.CoreComponents.icon component.

Since all the functions in MyAppWeb.CoreComponents module are imported by default, the module name can be omitted when we use the icon component.

Thanks to the icon component, we no longer need to wrap our head around in terms of setting up icons, as long as we are happy with what heroicons library provides. It is very nice.

building breadcrumb component

I created a new module named MyAppWeb.Breadcrumb that is dedicated for the breadcrumb function component. Alternatively it could have been in MyAppWeb.CoreComponents along with the icon component but I chose to create a new module so that I can focus on the breadcrumb concern. I might change my mind in the future.

Here is the entirety of the module I created.

defmodule MyAppWeb.Breadcrumb do
  use Phoenix.Component
  import MyAppWeb.CoreComponents

  attr :items, :list, required: true

  def breadcrumb(assigns) do
    assigns = assign(assigns, :size, length(assigns.items))

    ~H"""
    <nav class="flex" aria-label="breadcrumb">
      <ol class="inline-flex items-center space-x-1 md:space-x-3">
        <.breadcrumb_item
          :for={{item, index} <- Enum.with_index(@items)}
          type={index_to_item_type(index, @size)}
          navigate={item[:navigate]}
          text={item[:text]}
        />
      </ol>
    </nav>
    """
  end

  defp index_to_item_type(0, _size), do: "first"
  defp index_to_item_type(index, size) when index == size - 1, do: "last"
  defp index_to_item_type(_index, _size), do: "middle"

  attr :type, :string, default: "middle"
  attr :navigate, :string, default: "/"
  attr :text, :string, required: true

  defp breadcrumb_item(assigns) when assigns.type == "first" do
    ~H"""
    <li class="inline-flex items-center">
      <.link navigate={@navigate} class="inline-flex items-center text-sm font-medium">
        <.icon name="hero-home" class="h-4 w-4" />
      </.link>
    </li>
    """
  end

  defp breadcrumb_item(assigns) when assigns.type == "last" do
    ~H"""
    <li aria-current="page">
      <div class="flex items-center">
        <.icon name="hero-chevron-right" class="h-4 w-4" />
        <span class="ml-1 text-sm font-medium md:ml-2">
          <%= @text %>
        </span>
      </div>
    </li>
    """
  end

  defp breadcrumb_item(assigns) do
    ~H"""
    <li>
      <div class="flex items-center">
        <.icon name="hero-chevron-right" class="h-4 w-4" />
        <.link navigate={@navigate} class="ml-1 text-sm font-medium md:ml-2 ">
          <%= @text %>
        </.link>
      </div>
    </li>
    """
  end
end

only one public function

The module has only one public function called breadcrumb, which is a function component. For readability, I factored out a sub-component breadcrumb_item that maps provided info to the breadcrumb item markup according to the breadcrumb item type.

three types of breadcrumb items

Typically the breadcrumb navigation has three types of items and they have different looks and behaviors.

the leftmost
- the home
- link enabled
- can look special with home icon
the rightmost
- the current
- link disabled
the in-between
- all other paths between the home and the current
- link enabled

For each type, the sub-component breadcrumb_item renders differently. We can easily determine the appropriate breadcrumb item type by the index and length of the list. The private function index_to_item_type does the job.

We need one preparation before rendering the component. The length of the list varies so we need to find out the list length beforehand.

Enum.with_index/2 gives an index to each element of the list. With that index and the pre-calculated list length, we can determine the item type.

tmux のコピペとクリップボード

Masatoshi Nishiguchi — Sun, 28 May 2023 12:56:06 +0000

はじめに

古い macbook に Linux Mint 21 をインストールして復活させました。普段は仕事で macOS を使っているので、Linux Mint に行ったりきたりしやすいように工夫を試みています。その一環として可能な限り OS を問わずに共通の設定で neovim と tmux を使えるように取り組んでいます。今回は、「tmux のコピペとクリップボード」について調べてわかったことをメモします。

世の中にはいろんな OS が存在しますが、ここでは macOS と Linux Mint しか登場しません。Linux Mint の中身は Ubuntu らしいので、Linux Mint は多分 Ubuntu と読み替えてもいいと思います。しらんけど。

クリップボード

クリップボード（英: clipboard）は、コンピュータ上で、一時的にデータを保存できる共有のメモリ領域のことである。複数の異なるプログラムからアクセス可能であり、単一のアプリケーションだけでなく異なったアプリケーション間のデータの受け渡しにも使用される。

https://ja.wikipedia.org/wiki/クリップボード

日頃何も考えずに使っているクリップボードですが、実はこれはクリップボードを操作するためのプログラムとアプリとがうまく連携していないとスムーズなコピペができないようです。

コピペのショートカット

キーボードでコピペする技です。macOS でも Linux Mint でもアルファベットのキーは同じですが、一緒に押すキーが異なります。キーを変更することもできるのでしょうが、慣れた方が早い気がします。

macOS

やりたいこと	ショートカット
切取	`⌘` + `X`
複写	`⌘` + `C`
貼付	`⌘` + `V`

https://support.apple.com/ja-jp/HT209651

Linux Mint

Linux Mint のターミナルでは Shift を押さないといけないので要注意。最初は戸惑いますが、慣れれば大丈夫。郷に入れば郷に従います。

やりたいこと	ショートカット
切取	`Ctrl` + `X`
複写	`Ctrl` + `C`
貼付	`Ctrl` + `V`
ターミナルで切取	`Ctrl` + `Shift` + `X`
ターミナルで複写	`Ctrl` + `Shift` + `C`
ターミナルで貼付	`Ctrl` + `Shift` + `V`

https://qiita.com/mnishiguchi/items/94d3ab813fca051a0e54

コピペのコマンド

プログラムからクリップボードを操作するためのコマンドが存在します。いい感じにまとまっている記事がありました。ありがとうございます。OS によって使用可能なプログラムが異なります。

https://qiita.com/sasaplus1/items/137a70e8f51f97a6636f

macOS

macOS ではコマンドが備えついているので特に悩むことはありません。

コマンド	機能
`pbcopy`	標準出力を受け取ってクリップボードにコピー
`pbpaste`	クリップボードのデータをターミナルに出力

echo "闘魂" | pbcopy

pbpaste

Linux Mint

Linux Mint にはクリップボードを操作するコマンドを自分でインストールする必要があります。また、いくつか選択肢があります。中でも xsel と xclip がよく利用されている様子です。ネット検索をしていて、「両方入れた方がいいよ！」との話があったので、深く考えず両方インストールしました。アプリにより、相性があるのではないかと想像しています。

sudo apt install xclip xsel

両方とも機能は似ています。

コピー先

ここで注意が必要なのがコピー先です。macOS のコマンドと異なり３種類あるので、クリップボードを操作するときにはコピー先がクリップボードと明示しておかないとどこにコピーしたのかわからなくなります。省略できるオプションもありますが、後々わかりやすいように明示しておくのが得策かもしれません。

clipboard
primary
secondary

https://wiki.archlinux.jp/index.php/クリップボード

xsel でコピペ

echo "闘魂" | xsel --clipboard --input

xsel --clipboard --output

詳しくはドキュメントをご参照ください。

man xsel

xclip でコピペ

echo "闘魂" | xclip -selection clipboard -in

xclip -selection clipboard -out

詳しくはドキュメントをご参照ください。

man xclip

neovim でクリップボードにヤンク（コピー）

vim 用語で copy のことを yank と言います。neovim でコピペするためにはいくつか条件を満たす必要があるようです。以下の記事にヒントがあります。

https://qiita.com/gotchane/items/0e7e6e0d5c7fa9f55c1a

                                  *clipboard-tool*
The presence of a working clipboard tool implicitly enables the '+' and '*'
registers. Nvim looks for these clipboard tools, in order of priority:

  - |g:clipboard|
  - pbcopy, pbpaste (macOS)
  - wl-copy, wl-paste (if $WAYLAND_DISPLAY is set)
  - xclip (if $DISPLAY is set)
  - xsel (if $DISPLAY is set)
  - lemonade (for SSH) https://github.com/pocke/lemonade
  - doitclient (for SSH) http://www.chiark.greenend.org.uk/~sgtatham/doit/
  - win32yank (Windows)
  - tmux (if $TMUX is set)

neovim はリストの上から順にクリップボード用のプログラムを探すようです。環境変数に依存する部分があるのでそこも注意が必要です。

tmux のコピーモード

tmux にはターミナル内で vim 風の操作をできるようにするコピーモードがあります。コピペできるだけでなくターミナル内を高速に移動できるようになり便利です。

https://github.com/tmux/tmux/wiki/Getting-Started#copy-and-paste

しかしながら、クリップボードとの連携に若干の癖がありました。しっかり準備をしないとイゴきません。

いくつか参考になる記事がありましたが、当時の tmux から設定方法が一部変わっているので注意が必要です。アイデアをありがたく頂きつつ、最後は原典に当たるのが一番です。

man tmux

現時点（2023 年 5 月）で最新は tmux 3.3a です。tmux 3.2 から copy-command オプションが導入されました。他にも書き方が変わっている部分がいくつかあります。

https://qiita.com/shimmer22/items/67ba93060ae456aadd1b

https://dev.to/iggredible/the-easy-way-to-copy-text-in-tmux-319g

https://github.com/tmux/tmux/wiki/Clipboard

色々試行錯誤した結果、~/tmux.conf の設定は以下の通りに落ち着きました。関連する部分を抜粋します。Linux Mint と macOS とでクリップボードのコマンドが違うので、条件分岐します。OS の種類の確認が厳密ではありませんが、自分用の設定なので問題ありません。

Linux 用クリップボードのプログラムとして xsel を使います。xclip より xsel の方が tmux との相性がいいようです。もちろん xsel コマンドがインストールされていることが前提です。

set-clipboard と copy-pipe が競合する場合があるそうです。ドキュメントの指示に従い set-clipboard を無効にします。

tmux のコピーモードの mode-keys は emacs か vi か選択できます。ここでは vi を使います。

コピーモード（vi）の初期設定ではコピーは Enter キーで実行するようになっています。Vimmer にとっては y でヤンクする方が自然なのでそれができるようにカスタマイズします。

# マウス有効化
set -g mouse on

# コピーモード（vi）を有効化
set-window-option -g mode-keys vi

# OS が Linux の時は xsel を使う
if-shell -b '[ "$(uname)" = "Linux" ]' {
  set -s copy-command "xsel --clipboard --input"
  display "using xsel as copy-command"
}

# OS が Darwin の時は pbcopy を使う
if-shell -b '[ "$(uname)" = "Darwin" ]' {
  set -s copy-command "pbcopy"
  display "using pbcopy as copy-command"
}

# copy-pipe と競合する場合があるので無効化
set -s set-clipboard off

# コピーモード中に Vim 風に v で選択範囲を定める
bind -Tcopy-mode-vi v send -X begin-selection

# コピーモード中に Vim 風に y で選択範囲をヤンクしてコピーモードを終了する
bind -Tcopy-mode-vi y send -X copy-pipe-and-cancel

# マウスをドラッグして選択範囲を定め、それをヤンクしてコピーモードを終了する
bind -Tcopy-mode-vi MouseDragEnd1Pane send -X copy-pipe-and-cancel

copy-command などの tmux の設定値を確認したい時は　tmux show が便利です。

tmux show copy-command

コピーモードのショートカットの初期設定は man tmux を打てば出てきます。

The following commands are supported in copy mode:

           Command                                      vi              emacs
           append-selection
           append-selection-and-cancel                  A
           back-to-indentation                          ^               M-m
           begin-selection                              Space           C-Space
           bottom-line                                  L
           cancel                                       q               Escape
           clear-selection                              Escape          C-g
           copy-end-of-line [<prefix>]
           copy-end-of-line-and-cancel [<prefix>]
           copy-pipe-end-of-line [<command>] [<prefix>]
           copy-pipe-end-of-line-and-cancel [<command>] [<prefix>]
                                                        D               C-k
           copy-line [<prefix>]
           copy-line-and-cancel [<prefix>]
           copy-pipe-line [<command>] [<prefix>]
           copy-pipe-line-and-cancel [<command>] [<prefix>]

           copy-pipe [<command>] [<prefix>]
           copy-pipe-no-clear [<command>] [<prefix>]
           copy-pipe-and-cancel [<command>] [<prefix>]
           copy-selection [<prefix>]
           copy-selection-no-clear [<prefix>]
           copy-selection-and-cancel [<prefix>]         Enter           M-w
           cursor-down                                  j               Down
           cursor-down-and-cancel
           cursor-left                                  h               Left
           cursor-right                                 l               Right
           cursor-up                                    k               Up
           end-of-line                                  $               C-e
           goto-line <line>                             :               g
           halfpage-down                                C-d             M-Down
           halfpage-down-and-cancel
           halfpage-up                                  C-u             M-Up
           history-bottom                               G               M->
           history-top                                  g               M-<
           jump-again                                   ;               ;
           jump-backward <to>                           F               F
           jump-forward <to>                            f               f
           jump-reverse                                 ,               ,
           jump-to-backward <to>                        T
           jump-to-forward <to>                         t
           jump-to-mark                                 M-x             M-x
           middle-line                                  M               M-r
           next-matching-bracket                        %               M-C-f
           next-paragraph                               }               M-}
           next-space                                   W
           next-space-end                               E
           next-word                                    w
           next-word-end                                e               M-f
           other-end                                    o
           page-down                                    C-f             PageDown
           page-down-and-cancel
           page-up                                      C-b             PageUp
           pipe [<command>] [<prefix>]
           pipe-no-clear [<command>] [<prefix>]
           pipe-and-cancel [<command>] [<prefix>]
           previous-matching-bracket                                    M-C-b
           previous-paragraph                           {               M-{
           previous-space                               B
           previous-word                                b               M-b
           rectangle-on
           rectangle-off
           rectangle-toggle                             v               R
           refresh-from-pane                            r               r
           scroll-down                                  C-e             C-Down
           scroll-down-and-cancel
           scroll-up                                    C-y             C-Up
           search-again                                 n               n
           search-backward <for>                        ?
           search-backward-incremental <for>                            C-r
           search-backward-text <for>
           search-forward <for>                         /
           search-forward-incremental <for>                             C-s
           search-forward-text <for>
           search-reverse                               N               N
           select-line                                  V
           select-word
           set-mark                                     X               X
           start-of-line                                0               C-a
           stop-selection
           toggle-position                              P               P
           top-line                                     H               M-R

さいごに

最終的にコンパクトにまとまって満足しています。Linux Mint がどんどん好きになってきました。

モクモク會

本記事は以下のモクモク會での成果です。みなさんから刺激と元氣をいただき、ありがとうございました。

https://youtu.be/c0LP23SM7BU

https://okazakirin-beam.connpass.com/

https://autoracex.connpass.com

もしご興味のある方はお氣輕にご參加ください。

https://qiita.com/piacerex/items/09876caa1e17169ec5e1

https://speakerdeck.com/elijo/elixirkomiyunitei-falsebu-kifang-guo-nei-onrainbian

https://qiita.com/torifukukaiou/items/57a40119c9eefd056cae

https://qiita.com/piacerex/items/e0b6e46b1325bb931122

https://qiita.com/torifukukaiou/items/1edb3e961acf002478fd

https://qiita.com/piacerex/items/e5590fa287d3c89eeebf

https://qiita.com/torifukukaiou/items/4481f7884a20ab4b1bea

https://note.com/awesomey/n/n4d8c355bc8f7

Elixirで進捗表示ダウンロード

Masatoshi Nishiguchi — Mon, 22 May 2023 02:08:47 +0000

Elixirで進捗状況を表示しながらダウンロードする方法について検討します。

やりたいこと

Bumblebee を使っているときにファイルをダウンロードするとこういうダウンロード進捗表示がでます。これをやってみたいです。コードは自分で書きます。

Bumblebeeのコード

プライベートの Bumblebee.Utils.HTTP モジュールにダウンロード関連のコードがありました。

Erlang の httpc モジュールと ProgressBar パッケージを使って実装されています。

ちなみに httpc の使い方はElixir Forum にまとめられています。

https://elixirforum.com/t/httpc-cheatsheet/50337

同じように httpc モジュールを使って実装しても良いのですが、個人的に日頃よく利用する Req を使って１から実装してみようと思います。

まずは、 Req をつかって簡単なGETリクエストする方法から始めます。ここではElixir のロゴの画像データをダウンロードの対象とします。

source_url = "https://elixir-lang.org/images/logo/logo.png"

Reqをつかって進捗表示なしにダウンロード

まずは、 Req をつかって進捗表示なしにダウンロードしてみます。

　

# データとしてダウンロード
<<_::binary>> = Req.get!(source_url).body

ローカルファイルとして保存したい場合は :output オプションで保存先を指定します。

destination_path = Path.join(System.tmp_dir!(), "elixir_logo.png")

# ダウンロードしてファイルに保存
Req.get!(source_url, output: destination_path)

# ちゃんと読み込めるか検証
File.read!(destination_path)

進捗表示を追加するにはどうしたら良いのでしょうか。Bumblebee のコードから ProgressBar パッケージを利用できることはすでにわかっています。それをどのように Req と連携させるかを調べます。

Req の構成要素（3 つ）

Req は 3 つの主要部分で構成されています。

Req - 高階層のAPI
Req.Request - 低階層のAPIとリクエスト構造体
Req.Steps - ひとつひとつの処理

カスタマイズは比較的容易にできそうです。

Req.Steps.run_finch/1

Req.Steps.run_finch/1 に手を加えることにより、リクエストのロジックを変更できることがわかりました。ドキュメントにわかりにくい部分がありますが、サンプルコードを読んでみて高階層のAPIに :finch_request オプションに関数を注入して Req.Steps.run_finch/1 ステップを入れ替えることができるようです。

Finch とは初期設定の Req が依存するHTTPクライアントだそうです。さらに Finch は Mint と NimblePool を使って性能を意識して実装されているそうです。

余談ですが、Elixirの関数に「闘魂」を注入する方法については以下の@torifukukaiouさんの記事がおすすめです。

https://qiita.com/torifukukaiou/items/c414310cde9b7099df55

Reqをつかって進捗表示付きダウンロードしてみる

このような形になりました。ポイントをいくつかあげます。

Req.get/2 に :finch_request オプションとしてリクエストを処理するカスタムロジック（関数）を注入します。
Finch.stream/5 でリクエストの多重化が可能です。ストリームという概念に疎いので「WEB+DB PRESS Vol.１２３」を読み返しました。「イーチ、ニィー、サン、ぁッダー！！！」
ストリームからは3パターンのメッセージが返ってくるようです。
- {:status, status} - the status of the http response
- {:headers, headers} - the headers of the http response
- {:data, data} - a streaming section of the http body
進捗表示に必要な情報はふたつ。
- データ全体のバイト数
- 受信完了したバイト数
進捗状況は記憶しておく必要があるので、Req.Response の :private フィールドに格納し、データを受信するたびに更新します。

defmodule MNishiguchi.Utils.HTTP do
  def download(source_url, req_options \\ []) do
    case Req.get(source_url, [finch_request: &finch_request/4] ++ req_options) do
      {:ok, response} -> {:ok, response.body}
      {:error, exception} -> {:error, exception}
    end
  end

  def download!(source_url, req_options \\ []) do
    Req.get!(source_url, [finch_request: &finch_request/4] ++ req_options).body
  end

  defp finch_request(req_request, finch_request, finch_name, finch_options) do
    acc = Req.Response.new()

    case Finch.stream(finch_request, finch_name, acc, &handle_message/2, finch_options) do
      {:ok, response} -> {req_request, response}
      {:error, exception} -> {req_request, exception}
    end
  end

  defp handle_message({:status, status}, response), do: %{response | status: status}

  defp handle_message({:headers, headers}, response) do
    total_size =
      Enum.find_value(headers, fn
        {"content-length", v} -> String.to_integer(v)
        {_k, _v} -> nil
      end)

    response
    |> Map.put(:headers, headers)
    |> Map.put(:private, %{total_size: total_size, downloaded_size: 0})
  end

  defp handle_message({:data, data}, response) do
    new_downloaded_size = response.private.downloaded_size + byte_size(data)
    ProgressBar.render(new_downloaded_size, response.private.total_size, suffix: :bytes)

    response
    |> Map.update!(:body, &(&1 <> data))
    |> Map.update!(:private, &%{&1 | downloaded_size: new_downloaded_size})
  end
end

以上のコードを IEx でランしてみます。

iex(5)> MNishiguchi.Utils.HTTP.download!(source_url)
|===                                                                               |   4% (1.36/34.95 KB)
|=======                                                                           |   8% (2.73/34.95 KB)
|==========                                                                        |  12% (4.10/34.95 KB)
|=============                                                                     |  16% (5.47/34.95 KB)
|================                                                                  |  20% (6.84/34.95 KB)
|===================                                                               |  23% (8.20/34.95 KB)
|======================                                                            |  27% (9.57/34.95 KB)
|=========================                                                        |  31% (10.94/34.95 KB)
|============================                                                     |  35% (12.31/34.95 KB)
|===================================                                              |  43% (15.04/34.95 KB)
|======================================                                           |  47% (16.41/34.95 KB)
|=========================================                                        |  51% (17.78/34.95 KB)
|=============================================                                    |  55% (19.15/34.95 KB)
|================================================                                 |  59% (20.52/34.95 KB)
|===================================================                              |  63% (21.88/34.95 KB)
|======================================================                           |  67% (23.25/34.95 KB)
|=========================================================                        |  70% (24.62/34.95 KB)
|============================================================                     |  74% (25.99/34.95 KB)
|===============================================================                  |  78% (27.36/34.95 KB)
|==================================================================               |  82% (28.72/34.95 KB)
|======================================================================           |  86% (30.09/34.95 KB)
|=========================================================================        |  90% (31.46/34.95 KB)
|============================================================================     |  94% (32.83/34.95 KB)
|=======================================================================================| 100% (34.95 KB)
:ok

Livebook でやるともっといい感じに進捗状況が更新されるはずです。

Bumblebee.Utils.HTTP.download/2

Bumblebee を使っているのであれば、Bumblebee.Utils.HTTP.download/2 で同じようなことができます。ドキュメントには載ってませんが利用可能です。

Bumblebee.Utils.HTTP.download(source_url, destination_path)

Nerves Livebook

せっかくいい感じのコードが書けたので Nerves Livebook に寄贈いたしました。

https://github.com/livebook-dev/nerves_livebook/blob/9515bd61b4da6b30c6165b33f9a0ae56880ddc44/priv/samples/tflite.livemd

Elixirコミュニティ

本記事は以下のモクモク會での成果です。みなさんから刺激と元氣をいただき、ありがとうございました。

https://youtu.be/c0LP23SM7BU

https://okazakirin-beam.connpass.com/

https://autoracex.connpass.com

もしご興味のある方はお氣輕にご參加ください。

https://qiita.com/piacerex/items/09876caa1e17169ec5e1

https://speakerdeck.com/elijo/elixirkomiyunitei-falsebu-kifang-guo-nei-onrainbian

https://qiita.com/torifukukaiou/items/57a40119c9eefd056cae

https://qiita.com/piacerex/items/e0b6e46b1325bb931122

https://qiita.com/torifukukaiou/items/1edb3e961acf002478fd

https://qiita.com/piacerex/items/e5590fa287d3c89eeebf

https://qiita.com/torifukukaiou/items/4481f7884a20ab4b1bea

https://note.com/awesomey/n/n4d8c355bc8f7

Elixir OpenCV: StbImageとNx.Tensor と Evision.Mat で相互変換

Masatoshi Nishiguchi — Thu, 04 May 2023 02:23:10 +0000

Elixir で OpenCV を使うときにはどのような素材をどこから取ってきてどう加工するかによって求められる関数が異なり、場合により複数のパッケージが連携する必要があるかと思います。

@ryowakabayashi さんのElixir Image と Nx と evision で相互変換が大変参考になりました。

ここでは elixir-image/image よりシンプルな elixir-nx/stb_image と cocoa-xu/evision で相互変換をしてみようと思います。

https://qiita.com/RyoWakabayashi/items/34de2207bcf8d745a01a

セットアップ

Mix.install([
  {:nx, "~> 0.5.0"},
  {:evision, "0.1.31"},
  {:stb_image, "~> 0.6.0"},

  # 画像を Web からダウンロードするため
  {:req, "~> 0.3.0"},

  # Livebook上で画像を表示するため
  # {:kino, "~> 0.9.0"}
])

cocoa-xu/evision はパッチバージョンでも破壊的な変更があるようのでバージョンをロックしておいた方が無難そうです。

https://qiita.com/RyoWakabayashi/items/2846133bc7014319d172

cocoa-xu/evision の作者は好んで Cv エイリアスを使用しているのでそれに則ります。

https://github.com/cocoa-xu/evision/tree/main/examples

alias Evision, as: Cv

画像加工の一例

一例として画像をこのように加工することができます。ここで StbImage と Nx.Tensor と Evision.Mat で相互変換がされています。

img_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/6/65/Antonio_Inoki_IMG_0398-2_20121224.JPG/330px-Antonio_Inoki_IMG_0398-2_20121224.JPG"

{x, y, w, h} = {40, 30, 250, 250}
rect_start_point = {x, y}
rect_end_point = {x + w, y + h}
rect_color = {0, 255, 0}
rect_options = [thickness: 5]

Req.get!(img_url)
|> then(& &1.body)
|> StbImage.read_binary!()
|> StbImage.to_nx()
|> Cv.Mat.from_nx_2d()
|> Cv.cvtColor(Cv.Constant.cv_COLOR_BGR2RGB())
|> Cv.rectangle(rect_start_point, rect_end_point, rect_color, rect_options)

こういう書き方もできそうです。

img_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/6/65/Antonio_Inoki_IMG_0398-2_20121224.JPG/330px-Antonio_Inoki_IMG_0398-2_20121224.JPG"
save_as = Path.join(System.tmp_dir!(), URI.encode_www_form(img_url))

{x, y, w, h} = {40, 30, 250, 250}
rect_start_point = {x, y}
rect_end_point = {x + w, y + h}
rect_color = {0, 255, 0}
rect_options = [thickness: 5]

Req.get!(img_url, output: save_as)
|> then(fn _ -> Cv.imread(save_as) end)
|> Cv.rectangle(rect_start_point, rect_end_point, rect_color, rect_options)

image creadit: https://ja.wikipedia.org/wiki/アントニオ猪木

画像の読込

画像はバイナリとして読み込む場合もあれば、ファイルを読み込む場合もあると思います。

Req.get!/2 を用いて Web から画像バイナリをダウンロードすることができます。

img_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/6/65/Antonio_Inoki_IMG_0398-2_20121224.JPG/330px-Antonio_Inoki_IMG_0398-2_20121224.JPG"

img_data =  Req.get!(img_url) |> then(& &1.body)

また、画像をファイルとしてローカルに保存したい場合があるかもしれません。

download = fn url ->
  save_as = Path.join(System.tmp_dir!(), URI.encode_www_form(url))
  Req.get!(url, output: save_as)
  save_as
end

img_file = download.(img_url)

Binary -> StbImage

画像バイナリの読み込みには StbImage.read_binary!/1 が手軽で便利です。StbImage 構造体が帰ります。

img_stb = StbImage.read_binary!(img_data)

File -> Evision.Mat

ローカルに保存された画像ファイルを読み込む場合は Cv.imread/1 が使えます。

img_mat = Cv.imread(img_file)

StbImage <-> Nx.Tensor

StbImage は簡単に Nx.Tensor へ変換できます。

img_nx = StbImage.to_nx(img_stb)

逆も簡単です。

img_stb = StbImage.from_nx(img_nx)

Nx.Tensor <-> Evision.Mat

Nx.Tensor から Evision.Mat への変換も基本的には同様にできるのですが、ここでは注意が必要です。2点あります。

RGB vs BGR

OpenCVは色チャネルをB（青）・G（緑）・R（赤）の順に保持しています。ですので、適切な場所で B と R 入れ替える必要があります。

Evision.Mat.from_nx vs Evision.Mat.from_nx_2d

これは正直いうとよくわかりませんが、2Dの画像を取り扱う場合は from_nx_2d を使わないとうまく処理ができないようです。OpenCV の関数を使って画像加工しても何も起こらない場合はここを見落としている可能性があります。エラーがでないので厄介です。

img_nx =
  img_mat
  |> Cv.cvtColor(Cv.Constant.cv_COLOR_BGR2RGB())
  |> Cv.Mat.to_nx()

img_mat =
  img_nx
  |> Cv.Mat.from_nx_2d()
  |> Cv.cvtColor(Cv.Constant.cv_COLOR_BGR2RGB())

Evision.Mat <-> StbImage

Evision.Mat と StbImage との間では直接の変換はできませんが、Nx.Tensor を介在させることにより実現可能です。

img_mat
|> Cv.cvtColor(Cv.Constant.cv_COLOR_BGR2RGB())
|> Cv.Mat.to_nx()
|> StbImage.from_nx()

img_stb
|> StbImage.to_nx()
|> Cv.Mat.from_nx_2d()
|> Cv.cvtColor(Cv.Constant.cv_COLOR_BGR2RGB())

アルファチャネルを取り除く変換

場合によりアルファチャネルを取り除きたい場合があると思いますが、現時点では Evision.Backend はスライス構文に対応していないようです。

Cv.Mat.to_nx(img_mat)[[..,..,0..2]]

# ** (RuntimeError) operation slice is not yet supported on Evision.Backend.
# Please use another backend like Nx.BinaryBackend or Torchx.Backend.
#   To use Torchx.Backend, :torchx should be added to your app's deps.
#   Please see <https://github.com/elixir-nx/nx/tree/main/torchx> for more information on how to install and use it.
# To convert the tensor to another backend, please use Evision.Mat.to_nx(tensor, Backend.ModuleName)
#   for example, Evision.Mat.to_nx(tensor, Nx.BinaryBackend) or Evision.Mat.to_nx(tensor, Torchx.Backend).

Evison.Mat を Nx.Tensor に変換するときにバックエンドをしていするとうまくいきます。

Cv.Mat.to_nx(img_mat, Nx.BinaryBackend)[[.., .., 0..2]]

DEV Community: Masatoshi Nishiguchi

Bringing Open JTalk to Elixir/Nerves: Make Your Pi Speak Japanese 🇯🇵

Introduction

What Is Open JTalk?

Challenges in embedded environments

1. Native Build Complexity

2. Required Assets Are Large

The approach that worked: Vendor + Bundle + Shell Out

Ideas That Didn’t Work

Custom Nerves System

Elixir Native Implemented Functions (NIFs)

Elixir Port

Build System Details

Cross-Platform Portability

Autotools Pain

macOS Quirks

Nerves Cross-Compiling

How to Use in Elixir

Options available

Asset Resolution

Lessons Learned

Final Thoughts

Getting Started with AtomVM: Setup with Prebuilt Firmware (ESP32-S3, September 2025)

Target Environment and Devices

Target Board

Host Machine

Notes

Installing esptool

Checking the Serial Port

Verifying esptool and Board Connection

Downloading and Flashing AtomVM Firmware

Download the image

Erase flash (recommended)

Write AtomVM firmware

Building and Flashing an Elixir Application

Get the sample project

Adjust GPIO pin for XIAO ESP32-S3

Configure mix.exs with flash offset

Package the app

Flash the app to ESP32-S3

Viewing Logs over Serial

Wrapping Up

Links

Building a Local AI-Friendly Code Viewer Inspired by Uithub

First Attempt: Shell Script

Rewriting in Go

Wrapping up

Boost Your Terminal Navigation with the y z Workflow

What is y → z

The Power of z (zoxide)

Basic Commands

The Power of y (yazi)

Basic Commands

Conclusion

Elixir/Nerves: blinking LED

Introduction

Nerves Livebook

Blink using Circuits GPIO

Wrapping our LED blinking logic in GenServer

circuits_gpio package

Wrapping up

Where Nerves-related Mix tasks are defined?

Mix

Enumerating Mix tasks in Elixir project

Filtering down to only Nerves-related Mix tasks

Where Nerves-related Mix tasks are defined

nerves package

nerves_bootstrap package

nerves_key package

ssh_subsystem_fwup package

Wrapping up

Build Phoenix Docker Compose development environment easily

Getting started

Wrapping up

Elixirでリストの最初の要素を取得

ElixirのList

実験の準備

実験用List

実験を実行する関数

Kernel.hd/1 (hd/1)

Verifying `esptool` and Board Connection

Configure `mix.exs` with flash offset

What is `y` → `z`

The Power of `z` (zoxide)

The Power of `y` (yazi)

tmux のコピペとクリップボード