DEV Community: niku

How to write bitstring as read such as 001 1101 on Elixir

niku — Fri, 03 May 2019 15:05:40 +0000

UPDATE: Added binary handling with using 0b and :binary.encode_unsigned/1. Thanks for Elixir Forum.

My motivation

I want to write bits like 010 1100 000 0010 (from protocol buffer's encoding sample) on Elixir particular for testing.

Please note, most of cases we have to handle on erlang or elixir is binary which means bitstring to be divisible by 8. If you wanted to write binary as you read, you may use integer literal starting 0b and :binary.encode_unsigned/1 such as:

:binary.encode_unsigned(0b1010_1100_1111_1111)
# => <<172, 255>>

However, I want to handle bitstring which is not divisible by 8, for example 3bits, 7bits, 17bits and so on. binary.encode_unsigned/1 aligns to binary.

bit_size(:binary.encode_unsigned(0b101))
# => 8

In above case, I wanted to get 3 as bit size but 8. So I can't use 0b and :binary.encode_unsigned/1 for "writing what I read".

I know writing bitstring as I read has already be able to do following like:

<<0::1, 1::1, 0::1, 1::1, 1::1, 0::1, 0::1, 0::1, 0::1, 0::1, 0::1, 0::1, 1::1, 0::1>>
# => <<88, 2::size(6)>>

It is correct. Because, in Integer, a representation of 8bits 010 1100 0 is <<88>> (you can check Integer.digits(88, 2) # => [1, 0, 1, 1, 0, 0, 0], please note heading 0s are dropped) and the rest 00 0010 is <<2::size(6)>>.

However, I feel the former <<0::1, 1::1, ...>> is quite verbose and the latter <<88, 2::size(6)>> is not intuitive. Don't you feel so?

I want to write bits as read such as:

010 1100 000 0010 
# => <<88, 2::size(6)>>

My solution

sigil is

one of the mechanisms provided by the language for working with textual representations.

That's I want.

First time, I implement as function following like:

# You can check the code on iex
defmodule BitsSigils do
  require Logger

  def sigil_b(term, _modifiers) do
    for <<c <- term>>,
      into: <<>> do
        case c do
          ^c when c in [?0, ?1] ->
            # It's a valid character as an element.
            <<c::1>>
          ^c when c in [?_, ?\s] ->
            # It's a valid character as a just placeholder.
            <<>>
          _ ->
            Logger.warn("Given unexpected sigil_b character, just ignored. The character was \"#{:binary.encode_unsigned(c)}\", its integer value was #{c}.")
            <<>>
        end
    end
  end
end

import BitsSigils
~b(_010 1100 _000 0010)
# => <<88, 2::size(6)>>

Yeah, It seems to work fine. But it doesn't work as a left side on a pattern matching.

case <<88, 2::size(6)>> do
  ~b(_010 1100 _000 0010) ->
    true
  _ ->
    false
end

# ** (CompileError) iex:16: cannot invoke remote function BitsSigils.sigil_b/2 inside a match
#     (stdlib) lists.erl:1354: :lists.mapfoldl/3
#     (stdlib) lists.erl:1354: :lists.mapfoldl/3

So finally, I implement sigil_b as a macro following like:

# You can check the code on iex
defmodule BitsSigils do
  require Logger

  defmacro sigil_b({:<<>>, meta, [arg]}, modifiers) do
    result = do_sigil_b(arg, modifiers)
    {:<<>>, meta, [Macro.escape(result)]}
  end

  def do_sigil_b(term, _modifiers) do
    for <<c <- term>>,
      into: <<>> do
        case c do
          ^c when c in [?0, ?1] ->
            # It's a valid character as an element.
            <<c::1>>
          ^c when c in [?_, ?\s] ->
            # It's a valid character as a just placeholder.
            <<>>
          _ ->
            Logger.warn("Given unexpected sigil_b character, just ignored. The character was \"#{:binary.encode_unsigned(c)}\", its integer value was #{c}.")
            <<>>
        end
    end
  end
end

import BitsSigils
case <<88, 2::size(6)>> do
  ~b(_010 1100 _000 0010) ->
    true
  _ ->
    false
end
# => true

Conclusion

Most of cases you have to handle is binary which is special case of bitstring. In such cases, You may use 0b and :binary.encode_unsigned/1
If you want to get something textual representation, you could use sigil.
If you want to use sigil at left hand side on an evaluation, you had to use macro instead of function.

Parameterized testing with ExUnit

niku — Sun, 23 Dec 2018 13:16:17 +0000

UPDATE: hauleth mentions me a better solution in ElixirForum. So, I updated this article.

I found a way which is parameterized testing with ExUnit.

ExUnit.start()

defmodule ParameterizedTest do
  use ExUnit.Case, async: true

  for {lhs, rhs} <- [{"one", 1}, {"two", 2}, {"three", 3}] do
    test "#{lhs} convert to #{rhs}" do
      assert unquote(lhs) === unquote(rhs)
    end
  end
end

  1) test one convert to 1 (ParameterizedTest)
     parameterized_test.exs:7
     Assertion with === failed
     code:  assert "one" === 1
     left:  "one"
     right: 1
     stacktrace:
       parameterized_test.exs:8: (test)



  2) test three convert to 3 (ParameterizedTest)
     parameterized_test.exs:7
     Assertion with === failed
     code:  assert "three" === 3
     left:  "three"
     right: 3
     stacktrace:
       parameterized_test.exs:8: (test)



  3) test two convert to 2 (ParameterizedTest)
     parameterized_test.exs:7
     Assertion with === failed
     code:  assert "two" === 2
     left:  "two"
     right: 2
     stacktrace:
       parameterized_test.exs:8: (test)



Finished in 0.03 seconds (0.03s on load, 0.00s on tests)
3 tests, 3 failures

Randomized with seed 169786

The Probrem

We can't pass variables from outside into the block of tests directly.

ExUnit.start()

defmodule ParameterizedTest do
  use ExUnit.Case, async: true

  for {lhs, rhs} <- [{"one", 1}, {"two", 2}, {"three", 3}] do
    test "#{lhs} convert to #{rhs}" do
      assert lhs === rhs # We can't use their variables here.
    end
  end
end

** (CompileError) parameterized_test.exs:8: undefined function lhs/0
    (stdlib) lists.erl:1338: :lists.foreach/2

Other solutions

Use assertions instead of tests

Assertions and tests work well. But, ExUnit shows only one message per test. So, If you desire showing outline of tests. It does not match well.

ExUnit.start()

defmodule ParameterizedTest do
  use ExUnit.Case, async: true

  test "convert" do
    for {lhs, rhs} <- [{"one", 1}, {"two", 2}, {"three", 3}] do
      assert lhs === rhs
    end
  end
end

  1) test convert (ParameterizedTest)
     parameterized_test.exs:6
     Assertion with === failed
     code:  assert lhs === rhs
     left:  "one"
     right: 1
     stacktrace:
       parameterized_test.exs:8: anonymous fn/2 in ParameterizedTest."test convert"/1
       (elixir) lib/enum.ex:1925: Enum."-reduce/3-lists^foldl/2-0-"/3
       parameterized_test.exs:7: (test)



Finished in 0.03 seconds (0.03s on load, 0.00s on tests)
1 test, 1 failure

Randomized with seed 303559

Use module attributes

Tests work well. But, module attributes breaks scope. So, you have to keep module attribute in mind after using it.

ExUnit.start()

defmodule ParameterizedTest do
  use ExUnit.Case, async: true

  for {lhs, rhs} <- [{"one", 1}, {"two", 2}, {"three", 3}] do
    @pair {lhs, rhs}

    test "#{lhs} convert to #{rhs}" do
      {l, r} = @pair
      assert l === r
    end
  end

  test "pair should not have any value" do
    assert nil === @pair # We don't expect getting any values
  end
end

  1) test pair should not have any value (ParameterizedTest)
     parameterized_test.exs:15
     Assertion with === failed
     code:  assert nil === @pair
     left:  nil
     right: {"three", 3}
     stacktrace:
       parameterized_test.exs:16: (test)



  2) test three convert to 3 (ParameterizedTest)
     parameterized_test.exs:9
     Assertion with === failed
     code:  assert l === r
     left:  "three"
     right: 3
     stacktrace:
       parameterized_test.exs:11: (test)



  3) test two convert to 2 (ParameterizedTest)
     parameterized_test.exs:9
     Assertion with === failed
     code:  assert l === r
     left:  "two"
     right: 2
     stacktrace:
       parameterized_test.exs:11: (test)



  4) test one convert to 1 (ParameterizedTest)
     parameterized_test.exs:9
     Assertion with === failed
     code:  assert l === r
     left:  "one"
     right: 1
     stacktrace:
       parameterized_test.exs:11: (test)



Finished in 0.04 seconds (0.04s on load, 0.00s on tests)
4 tests, 4 failures

Randomized with seed 936081

Use ex_parameterized

https://github.com/KazuCocoa/ex_parameterized makes allow parameterized testing following:

defmodule MyExampleTest do
  use ExUnit.Case, async: true
  use ExUnit.Parameterized        # Required

  test_with_params "add params",  # description
    fn (a, b, expected) ->        # test case
      assert a + b == expected
    end do
      [
        {1, 2, 3},                 # parameters
        "description": {1, 4, 5},  # parameters with description
      ]
  end
end

I like this idea. But this time, I have wanted to write ExUnit test cases with only standard libraries.

Use ExUnit.Case.register_test/4

ExUnit.Case.register_test/4.

Registers a new attribute to be used during ExUnit.Case tests.

The attribute values will be available as a key/value pair in context.registered. The key/value pairs will be cleared after each ExUnit.Case.test/3 similar to @tag.

ExUnit.start()

defmodule ParameterizedTest do
  use ExUnit.Case, async: true

  ExUnit.Case.register_attribute __ENV__, :pair

  for {lhs, rhs} <- [{"one", 1}, {"two", 2}, {"three", 3}] do
    @pair {lhs, rhs}

    test "#{lhs} convert to #{rhs}", context do
      {l, r} = context.registered.pair
      assert l === r
    end
  end

  test "pair should not have any value" do
    assert nil === @pair
  end
end

warning: undefined module attribute @pair, please remove access to @pair or explicitly set it before access
  parameterized_test.exs:18: ParameterizedTest (module)

.

  1) test two convert to 2 (ParameterizedTest)
     parameterized_test.exs:11
     Assertion with === failed
     code:  assert l === r
     left:  "two"
     right: 2
     stacktrace:
       parameterized_test.exs:13: (test)



  2) test one convert to 1 (ParameterizedTest)
     parameterized_test.exs:11
     Assertion with === failed
     code:  assert l === r
     left:  "one"
     right: 1
     stacktrace:
       parameterized_test.exs:13: (test)



  3) test three convert to 3 (ParameterizedTest)
     parameterized_test.exs:11
     Assertion with === failed
     code:  assert l === r
     left:  "three"
     right: 3
     stacktrace:
       parameterized_test.exs:13: (test)



Finished in 0.05 seconds (0.05s on load, 0.00s on tests)
4 tests, 3 failures

Randomized with seed 385498

It works well, outline of tests, scope about variables and only using standard libraries. Actually, this solution is written on the top of this article before I update. But I feel codes a little bit verbose. for example: {l, r} = context.registered.pair.