How do I improve performance in Elixir

https://dev.to/tokoyax/a-little-story-about-the-no-command-1am

Continuation of the above article

Execution speed problem

Command no is slower than theyes command like this.

> yes | pv -r > /dev/null
[31.0 MiB/s]

>./no | pv -r > /dev/null
[85.6 KiB/s]

I got the following comment.

you've better to use buffer for writing. Actually, yes command does not call write (2) for each lines.

It seems to be caused by a large number of I/O, so it is better to use a buffer.

Try to improve.

Using IO List

https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1-building-output-efficiently

According to the above article, when passing strings to IO.puts, Instead of passing only one combined string, storing the part of the string to the list has better performance.

If the same string is stored in the list, it's efficiently because the same string refers to the same address.

I fixed 'no.exs' to use a buffer.

no.exs

defmodule No do
  @moduledoc """
  Documentation for No.
  """

  def main(args) do
    args
    |> parse_args()
    |> run()
    |> wait()
  end

  def say(args) do
    {_, _, _, debug} = args
    message = buffer(args)
    case debug do
      true -> IO.inspect({:debug, message, self()})
      _    -> IO.puts(message)
    end
    say(args)
  end

  defp buffer(args) do
    {argv, process_num, _, debug} = args

    message = case debug do
      true -> "#{expletive(argv)} from process no.#{process_num}"
      _    -> expletive(argv)
    end

    buffer(args, message, [])
  end

  defp buffer(args, message, list) do
    # faster than append last
    # https://hexdocs.pm/elixir/List.html
    list = [message | list]
    cond do
      IO.iodata_length(list) >= 4096 -> list
      true -> buffer(args, message, list)
    end
  end

  defp expletive(_ = ''), do: "n\n"
  defp expletive(value), do: value

  defp parse_args(args) do
    {opts, argv, _} = OptionParser.parse(
      args,
      swithces: options_list(),
      aliases: aliases_list()
    )
    {argv, opts}
  end

  defp options_list() do
    [
      processes: :integer,
      debug: :boolean,
    ]
  end

  defp aliases_list() do
    [
      p: :processes,
      d: :debug,
    ]
  end

  defp run(args) do
    {argv, opts} = args

    num = case opts[:processes] do
      nil -> 1
      _   -> String.to_integer(opts[:processes])
    end

    Enum.each((1..num), fn(n) ->
      spawn_link(No, :say, [{argv, n, self(), opts[:debug]}]) end)
  end

  defp wait(args) do
    wait(args)
  end
end

Measure

before

> ./no | pv -r > /dev/null
[85.6 KiB/s]

after

> ./no | pv -r > /dev/null
[802KiB /s]

SUCCESS :D

Try parallel execution

> ./ no - p 4 | pv - r > /dev/null
[1.33 MiB / s]

Okay, let's run in parallel more.

> ./ no - p 10000 | pv - r > /dev/null
[5.00 MiB /s]
[3.16 MiB /s]
[1.84 MiB /s]
[458 KiB /s]
[466 KiB /s]
[569 KiB /s]
[724 KiB /s]
[805 KiB /s]
[1.27 MiB /s]
[6.17 MiB /s]
[15.3 KiB /s]
[1.13 MiB /s]

Although the result which speeded up was obtained, the output result was not constant.

I don't know why.

Summary

• I got knowledge of speed improvement such as IO List.
• It is simply fun to make faster.
• The yes command is still faster.

Github: [https://github.com/tokoyax/no:title]

(reference)

• https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1-building-output-efficiently
• https://hexdocs.pm/elixir/List.html